Target detection method, intelligent device and storage medium
By preprocessing the images to be detected by smart devices and inputting multiple object detection models for fusion, the problems of low detection efficiency and poor accuracy of existing detection methods are solved, and more efficient and accurate detection results are achieved.
Patent Information
- Application Number
- CN202411689905.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-22
- Publication Date
- 2025-02-28
AI Technical Summary
The existing detection methods have low detection efficiency and poor detection accuracy, resulting in low personnel utilization, high cost and high risk.
A target detection method is provided, by acquiring the image to be detected by the intelligent device, preprocessing and inputting the image into multiple object detection models, and fusing the detection results to obtain the final detection results.
It improves the accuracy and robustness of detection, reduces the false detection rate and missed detection rate, and improves the detection speed and efficiency.
Smart Images
Figure CN119540512A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and specifically provides a target detection method, an intelligent device and a storage medium. Background Art
[0002] At present, in the manufacturing process, various defects may occur in products due to human operational errors. Generally, visual inspection and traditional visual inspection are used to inspect products, but both methods have certain limitations.
[0003] The use of visual inspection methods will lead to low utilization of personnel and increase labor costs. At the same time, personnel are prone to fatigue when doing repetitive inspections for a long time, which will lead to missed inspections or false inspections. Overall, visual inspection has low efficiency, high risks and high costs.
[0004] The stable operation of traditional visual inspection has high requirements, such as stable lighting environment, stable product features and high requirements for acquired images, which makes the entire installation and debugging process complicated and long. Summary of the invention
[0005] In order to overcome the above defects, the present application is proposed to provide a solution or at least partially solve the technical problems of low detection efficiency and poor detection accuracy of existing detection methods. The present application provides a target detection method, a smart device and a storage medium.
[0006] In a first aspect, the present application provides a target detection method, the method comprising:
[0007] Acquire the image to be detected of the smart device;
[0008] Preprocessing the image to be detected to obtain N preprocessed images;
[0009] Input the preprocessed images into N target detection models respectively, and output N detection results;
[0010] A final detection result is obtained based on N of the detection results, where N is an integer greater than or equal to 2.
[0011] In one embodiment of the present application, the preprocessing of the image to be detected includes: performing N downsampling operations on the image to be detected to obtain N preprocessed images, wherein the sizes of the N preprocessed images are different.
[0012] In one embodiment of the present application, the obtaining of the final test result based on the N test results includes:
[0013] Counting the number of all categories in the N detection results;
[0014] Determine the category corresponding to the maximum number based on the number of all categories;
[0015] The maximum number of categories is taken as the final detection result.
[0016] In one embodiment of the present application, before acquiring the image to be detected, the method further includes:
[0017] Obtaining location coding information of the smart device;
[0018] Determine whether the smart device enters or leaves a preset area based on the location code information;
[0019] When the smart device enters the preset area, using a sensor at a preset position to collect the image to be detected;
[0020] When the smart device leaves the preset area, the sensor at the preset position is controlled to stop collecting the image to be detected.
[0021] In one embodiment of the present application, the obtaining of the location coding information of the smart device includes:
[0022] Establishing a communication connection with an industrial control device, wherein the industrial control device is used to control the sensor;
[0023] The location coding information of the smart device is obtained from the industrial control device.
[0024] In one embodiment of the present application, the industrial control device includes a programmable logic controller;
[0025] The establishing of a communication connection with the industrial control device comprises:
[0026] Create a client object;
[0027] The client object is used to connect to the programmable logic controller to establish a communication connection with the industrial control device.
[0028] In one embodiment of the present application, determining whether the smart device enters or leaves a preset area based on the location code information includes:
[0029] Parsing the position coding information;
[0030] Determining whether the parsed position coding information is within a preset coding range;
[0031] If so, it is determined that the smart device enters the preset area, otherwise, it is determined that the smart device leaves the preset area.
[0032] In one embodiment of the present application, the method further includes:
[0033] Determine whether the final test result is consistent with the preset standard information;
[0034] If not, the final test result is sent to relevant personnel.
[0035] In a second aspect, a smart device is provided, comprising:
[0036] at least one processor;
[0037] and, a memory communicatively coupled to the at least one processor;
[0038] Wherein, a computer program is stored in the memory, and the computer program is the aforementioned target detection method when executed by the at least one processor.
[0039] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored in the computer-readable storage medium, wherein the program codes are suitable for being loaded and run by a processor to execute any of the target detection methods described above.
[0040] The above one or more technical solutions of this application have at least one or more of the following Beneficial effects:
[0041] The present application provides a target detection method, which specifically includes: obtaining an image to be detected of a smart device; preprocessing the image to be detected to obtain N preprocessed images; inputting the preprocessed images into N target detection models respectively, and outputting N detection results; obtaining the final detection result based on the N detection results, where N is an integer greater than or equal to 2. By generating multiple preprocessed images from each original image, the diversity of the data is increased, which helps the model capture more details and features, thereby improving the accuracy of detection. Using multiple models for detection, even if a model has a false detection or missed detection in a specific situation, other models can still provide correct detection results, thereby reducing the false detection rate and missed detection rate of the overall system. Through the fusion of multiple images and multiple models, not only the accuracy and robustness of the detection are improved, but also the detection speed and efficiency are improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The disclosure of the present application will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. In addition, similar numbers in the drawings are used to represent similar components, among which:
[0043] Figure 1 It is a schematic diagram of the main steps of a target detection method in one embodiment of the present application;
[0044] Figure 2 This is a complete flowchart of a target detection method in one embodiment of the present application;
[0045] Figure 3 It is a schematic diagram of the main structure of a target detection device in one embodiment of the present application;
[0046] Figure 4 It is a schematic diagram of the structure of a smart device in one embodiment of the present application. DETAILED DESCRIPTION
[0047] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.
[0048] In the description of the present application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, memory, and may also include software parts, such as program code, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Non-temporary computer-readable storage media include any suitable medium that can store program code, such as a disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B or A and B. The singular terms "one" and "the" may also include plural forms.
[0049] Currently, traditional detection methods have low detection efficiency and poor detection accuracy. To this end, the present application provides a target detection method, an intelligent device and a storage medium.
[0050] See attached Figure 1 , Figure 1 It is a flowchart of the main steps of a target detection method according to an embodiment of the present application.
[0051] like Figure 1 As shown, the target detection method in the embodiment of the present application mainly includes the following steps S10 to S40.
[0052] Step S10: Acquire the image to be detected of the smart device.
[0053] The smart devices described in this application may include driving devices, smart vehicles, robots, aircraft and other devices.
[0054] The image to be detected may be acquired by a sensor. For example, a camera, a laser radar, etc. may be used as examples of the sensor.
[0055] The number of the acquired images to be detected may be one or more, and there is no specific limitation on this.
[0056] In one example, the image to be detected may be an image captured by a sensor when a vehicle is undergoing relevant detection on a production line.
[0057] Step S20: pre-process the image to be detected to obtain N pre-processed images.
[0058] Preprocessing may include operations such as image cropping, image scaling, normalization, data enhancement, etc., but is not limited thereto.
[0059] Step S30: input the preprocessed images into N target detection models respectively, and output N detection results.
[0060] The target detection model is used to detect objects in the image, so as to obtain the category and position of the object. The target detection model in this embodiment can be a deep learning model, such as a YOLO model, an SSD model, a Faster R-CNN model, etc., or a different version or configuration of the same model, which is not specifically limited, and the target detection model can be selected according to actual needs.
[0061] Step S40: Obtain a final detection result based on N detection results, where N is an integer greater than or equal to 2.
[0062] Specifically, the final detection result can be obtained through voting decision, weighted average and other methods.
[0063] Based on the above steps S10 to S40, first obtain the image to be detected of the smart device; preprocess the image to be detected to obtain N preprocessed images; input the preprocessed images into N target detection models respectively, and output N detection results; obtain the final detection result based on the N detection results, where N is an integer greater than or equal to 2. By generating multiple preprocessed images from each original image, the diversity of the data is increased, which helps the model capture more details and features, thereby improving the accuracy of detection. When using multiple models for detection, even if a model has a false detection or missed detection in a specific situation, other models can still provide correct detection results, thereby reducing the false detection rate and missed detection rate of the overall system. Through the fusion of multiple images and multiple models, not only the accuracy and robustness of the detection are improved, but also the detection speed and efficiency are improved.
[0064] The above steps S10 to S40 are further explained below.
[0065] Specifically, the above step S10 can be implemented through the following steps S101 to S104.
[0066] Step S101: Acquire location coding information of a smart device.
[0067] The location encoding information of the smart device refers to the encoding data used to describe the specific location of the smart device in the physical space. These data can be absolute locations (such as longitude and latitude coordinates) or relative locations (such as distance and direction relative to a reference point).
[0068] Specifically, the above step S101 can be implemented through the following steps S1011 to S1012.
[0069] Step S1011: Establishing a communication connection with an industrial control device.
[0070] Industrial control equipment refers to equipment used to monitor and control various equipment (such as control lines and sensors, where the line can be a production line, transmission line, etc.) in the industrial production process and record relevant data. For example, the industrial control equipment can be a programmable logic controller (PLC).
[0071] In a specific implementation of the present application, establishing a communication connection with the industrial control device includes: creating a client object; and connecting to a programmable logic controller using the client object.
[0072] A client object is a programming entity (usually an instance of a class) used to communicate with industrial control equipment. It encapsulates the specific implementation of the communication protocol and provides a simple and easy-to-use interface.
[0073] Exemplarily, the system may first install the Snap7 library of Python, use snap7.client.Client() to create a client object, and then use client.connect(ip, rack, slot) to connect to the industrial control device.
[0074] Step S1012: Acquire location coding information of the smart device from the industrial control device.
[0075] Specifically, the client object can use client.db_read(db_number,start_address,length) to read the data block of the industrial control device, convert the read byte data into an integer, and thus obtain the position coding information of the smart device.
[0076] Step S102: determining whether the smart device enters or leaves a preset area based on the location code information.
[0077] Specifically, the above step S102 can be implemented through the following steps S1021 to S1023.
[0078] Step S1021: parse the position coding information.
[0079] The process of parsing position-encoded information is to transform the raw data into an understandable and actionable form, which usually involves converting the data from binary, string, or other formats into specific numerical values or coordinates.
[0080] Step S1022: Determine whether the parsed position coding information is within a preset coding range.
[0081] The preset coding range is a predefined coding range used to determine whether the smart device enters or leaves the area, and can be adaptively adjusted according to the actual scenario.
[0082] Specifically, based on the parsed code value, it is determined whether it is within a preset code range.
[0083] Step S1023: If yes, it is determined that the smart device enters the preset area; otherwise, it is determined that the smart device leaves the preset area.
[0084] The preset area is a pre-set area. For example, the preset area may refer to the field of view of a sensor at a preset position. The preset position may be adaptively adjusted according to the actual scene. For example, the preset position may be the best shooting position of a production line or a transmission line.
[0085] Exemplarily, taking [1000, 5000] as an example of a preset coding range, when the parsed coding value is within the coding range, it indicates that the smart device enters the preset area.
[0086] Step S103: When the smart device enters the preset area, the sensor at the preset position is used to collect the image to be detected.
[0087] Step S104: When the smart device leaves the preset area, the sensor at the preset position is controlled to stop collecting the image to be detected.
[0088] When the smart device enters a preset area, the sensor at the preset position starts to collect images to be detected including the smart device, until the smart device leaves the preset area, at which time the sensor at the preset position stops collecting images to be detected, thereby obtaining at least one image to be detected including the smart device.
[0089] Through the position coding information of the smart device and the judgment of the preset area, intelligent control of image acquisition is achieved, which can reduce erroneous acquisition caused by misjudgment and improve the accuracy and reliability of the system.
[0090] The above is a further description of step S10 , and the following is a further description of step S20 .
[0091] In one embodiment of the present application, the preprocessing of the image to be detected includes: performing N downsampling operations on the image to be detected to obtain N preprocessed images, wherein the sizes of the N preprocessed images are different.
[0092] Specifically, the images to be detected can be downsampled by different programs to obtain preprocessed images of different sizes. For example, the preprocessed images can be (1200, 1200), (960, 960), (640, 640), etc., where (1200, 1200) means that the width of the image is 1200 pixels and the height is 1200 pixels. When there are multiple images to be detected, each photo can be downsampled to (1200, 1200), (960, 960), (640, 640) to obtain a preprocessed image.
[0093] The above is a further description of step S20 , and the following is a further description of step S30 .
[0094] For step S30, the target detection model may be a pre-trained deep learning model. For example, during the training phase, the first target detection model may be trained with an image of size (1200, 1200), the second target detection model may be trained with an image of size (960, 960), and the third target detection model may be trained with an image of size (640, 640).
[0095] Specifically, after obtaining N trained target detection models, the N pre-processed images obtained by downsampling are respectively input into the corresponding target detection models to obtain N detection results.
[0096] The above is a further description of step S30 , and the following is a further description of step S40 .
[0097] Specifically, the above step S40 can be implemented through the following steps S401 to S403.
[0098] Step S401: Count the number of all categories in N detection results.
[0099] Each detection result usually contains one or more detection boxes and their corresponding categories and confidence levels. Specifically, all detection results are traversed to count the number of occurrences of each category.
[0100] Step S402: Determine the category corresponding to the maximum number based on the number of all categories.
[0101] Specifically, the category with the largest number of categories is determined based on the number of all categories.
[0102] Step S403: taking the category with the largest number as the final detection result.
[0103] By fusing the detection results of multiple models, the overall detection accuracy can be improved. By counting the number of all categories, determining the category corresponding to the maximum number, and using the majority voting method, the reliability of the detection results can be improved.
[0104] In a specific embodiment of the present application, the method further includes: determining whether the final test result is consistent with the preset standard information; if not, sending the final test result to relevant personnel.
[0105] The preset standard information is information that has been pre-configured before the detection begins, and can be adaptively adjusted according to the actual scenario. For example, for the detection of vehicle tires, the relevant configuration information of the tires (pre-configured standard information) has been pre-configured in the system by the user.
[0106] Specifically, after obtaining the final test results, the final test results are compared with the preset standard information to verify whether the final test results are consistent with the preset standard information. If they are inconsistent, the final test results can be pushed to the relevant responsible personnel so that the relevant responsible personnel can modify or improve them.
[0107] Figure 2 It can be used as a complete process diagram for detecting vehicle appearance defects. Appearance defects generally include defects in appearance shape, color and material. Figure 2 As shown, it can be specifically implemented through the following steps S1-S8.
[0108] Step S1: monitor the PLC signal, which contains the position coding information of the vehicle on the production line.
[0109] Step S2: Parse the position coding information and determine whether the parsed position coding information is within a preset coding range to determine whether the target vehicle is within the camera's field of view.
[0110] Step S3: When the target vehicle is within the camera's field of view, the camera collects the image to be detected containing the target vehicle until the target vehicle leaves the camera's field of view. The camera and the sensor may be installed at a visual inspection station.
[0111] Step S4: downsample the image to be detected to obtain preprocessed images with sizes of (1200, 1200), (960, 960), and (640, 640), respectively.
[0112] Step S5: input the preprocessed images with sizes of (1200, 1200), (960, 960), and (640, 640) into model 1, model 2, and model 3, respectively, to obtain three detection results.
[0113] Step S6: Voting is performed on the three test results, and the category with the largest number of categories is taken as the final test result.
[0114] Step S7: Determine whether the final test result is consistent with the standard configuration. If so, end; otherwise, execute step S8.
[0115] Step S8: Push the final test results to relevant staff via SMS, email, etc., so that the staff can know them in time and take corresponding measures.
[0116] Install cameras, controllers and other hardware at the visual inspection station to build a visual inspection site. Through visual inspection of product appearance defects, the risk of missed detection and false detection caused by visual inspection can be eliminated. The use of a deep learning-based visual inspection algorithm can effectively reduce the image quality requirements of visual inspection. The installation distance from the camera to the product ranges from centimeters to meters, the stability of lighting conditions also ranges from high to low, and the debugging cycle also ranges from months to days, effectively reducing the implementation cost of the visual station. Through this embodiment, stable and reliable detection results can be obtained for the detection of appearance shape, appearance color and material, avoiding the unreliability of single detection of the product.
[0117] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present application.
[0118] Furthermore, the present application also provides a target detection device.
[0119] See attached Figure 3 , Figure 3 It is a main structural block diagram of a target detection device according to an embodiment of the present application.
[0120] like Figure 3 As shown, the target detection device in the embodiment of the present application mainly includes a first acquisition module 11, a preprocessing module 12, a detection module 13 and a second acquisition module 14. In some embodiments, one or more of the first acquisition module 11, the preprocessing module 12, the detection module 13 and the second acquisition module 14 can be combined into one module.
[0121] In some embodiments, the first acquisition module 11 may be configured to acquire an image to be detected of a smart device.
[0122] The preprocessing module 12 may be configured to preprocess the image to be detected to obtain N preprocessed images.
[0123] The detection module 13 may be configured to input the pre-processed images into N target detection models respectively and output N detection results.
[0124] The second acquisition module 14 may be configured to acquire a final detection result based on N detection results, where N is an integer greater than or equal to 2.
[0125] In one implementation, the description of the specific implementation functions can refer to steps S10-S40.
[0126] The target detection device is used to perform Figure 1 The target detection method embodiments shown in the figure have similar technical principles, technical problems solved and technical effects produced. Technical personnel in this technical field can clearly understand that for the convenience and conciseness of description, the specific working process and related instructions of the target detection device can refer to the contents described in the embodiment of the target detection method, which will not be repeated here.
[0127] Further, it should be understood that since the setting of each module is only for illustrating the functional units of the device of the present application, the physical devices corresponding to these modules may be the processor itself, or a part of the software in the processor, a part of the hardware, or a part of the combination of software and hardware. Therefore, the number of each module in the figure is only schematic.
[0128] It is understood by those skilled in the art that each module in the device can be adaptively split or merged. Such splitting or merging of specific modules will not cause the technical solution to deviate from the principle of the present application, and therefore, the technical solutions after splitting or merging will fall within the protection scope of the present application.
[0129] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.
[0130] Furthermore, the present application also provides an intelligent device, which may include at least one processor; and a memory connected to the at least one processor; wherein the memory stores a computer program, and when the computer program is executed by the at least one processor, the target detection method described in any of the above embodiments is implemented. Figure 4 As shown, Figure 4 exemplarily shows the structure of a smart device, which includes a processor 100 and a memory 200.
[0131] Furthermore, the present application also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present application, the computer-readable storage medium can be configured to store a program for executing the target detection method of the above-mentioned method embodiment, and the program can be loaded and run by the processor to implement the above-mentioned target detection method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium can be a memory device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.
[0132] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.
[0133] The user personal information processed by this application will vary depending on the specific product / service scenario, and will be based on the specific scenario in which the user uses the product / service. It may involve the user's account information, device information, driving information, equipment information or other related information. The applicant will treat the user's personal information and its processing with a high degree of diligence.
[0134] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.
[0135] So far, the technical solutions of the present application have been described in conjunction with the specific implementations shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific implementations. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.
Claims
1. A target detection method, characterized in that: The method comprises: Acquire the image to be detected of the smart device; Preprocessing the image to be detected to obtain N preprocessed images; Input the preprocessed images into N target detection models respectively, and output N detection results; A final detection result is obtained based on N of the detection results, where N is an integer greater than or equal to 2.
2. The target detection method according to claim 1, characterized in that: The preprocessing of the image to be detected includes: performing N downsampling operations on the image to be detected respectively to obtain N preprocessed images, wherein the sizes of the N preprocessed images are different.
3. The target detection method according to claim 1, characterized in that: The obtaining of the final detection result based on the N detection results includes: Counting the number of all categories in the N detection results; Determine the category corresponding to the maximum number based on the number of all categories; The maximum number of categories is taken as the final detection result.
4. The target detection method according to claim 1, characterized in that: The step of obtaining the image to be detected of the smart device includes: Obtaining location coding information of the smart device; Determine whether the smart device enters or leaves a preset area based on the location code information; When the smart device enters the preset area, using a sensor at a preset position to collect the image to be detected; When the smart device leaves the preset area, the sensor at the preset position is controlled to stop collecting the image to be detected.
5. The target detection method according to claim 4, characterized in that: The obtaining of the position coding information of the smart device includes: Establishing a communication connection with an industrial control device, wherein the industrial control device is used to control the sensor; The location coding information of the smart device is obtained from the industrial control device.
6. The target detection method according to claim 5, characterized in that: The industrial control equipment includes a programmable logic controller; The establishing of a communication connection with the industrial control device comprises: Create a client object; The client object is used to connect to the programmable logic controller to establish a communication connection with the industrial control device.
7. The target detection method according to claim 4, characterized in that: The determining whether the smart device enters or leaves a preset area based on the location code information includes: Parsing the position coding information; Determining whether the parsed position coding information is within a preset coding range; If so, it is determined that the smart device enters the preset area, otherwise, it is determined that the smart device leaves the preset area.
8. The target detection method according to claim 1, characterized in that: The method further comprises: Determine whether the final test result is consistent with the preset standard information; If not, the final test result is sent to relevant personnel.
9. A smart device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores a computer program, and when the computer program is executed by the at least one processor, the target detection method according to any one of claims 1 to 8 is implemented.
10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the target detection method according to any one of claims 1 to 8.