Cascaded defect detection method, device, apparatus and computer readable storage medium

Abstract

The application discloses a kind of cascade defect detection method, device, equipment and computer readable storage medium, cascade defect detection method includes: the detection image of the object to be detected is obtained, whether there is defect in the detection image according to the preset defect detection model is detected;If there is defect in the detection image, the detection image is cropped to the target defect image;According to the preset classification model, the defect characteristics of the target defect image are classified and identified, and an identification result is obtained;The defect result of the object to be detected is determined based on the identification result.The application improves the accuracy of defect detection.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a cascade defect detection method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] In industrial settings, certain parts may develop various defects during the manufacturing process due to manufacturing procedures and environmental factors. These defects are mostly inspected manually, which can be affected by factors such as human fatigue and subjective standards, leading to high rates of missed and false detections, and ultimately inaccurate test results. Summary of the Invention

[0003] The main objective of this invention is to provide a cascade defect detection method, apparatus, device, and computer-readable storage medium, aiming to solve the technical problem of how to improve the accuracy of defect detection.

[0004] To achieve the above objectives, the present invention provides a cascade defect detection method, comprising the following steps:

[0005] Acquire a detection image of the object to be detected, and detect whether there are defects in the detection image according to a preset defect detection model;

[0006] If a defect exists in the detected image, the detected image is cropped to obtain a target defect image;

[0007] The defect features of the target defect image are classified and identified according to a preset classification model to obtain the identification result;

[0008] The defect result of the object to be detected is determined based on the recognition result.

[0009] Optionally, the step of cropping defects from the detected image to obtain a target defect image includes:

[0010] Determine the bounding box corresponding to the defect in the detected image, and construct a cropping box based on the bounding box;

[0011] The detected image is cropped according to the cropping box to obtain the target defect image.

[0012] Optionally, the step of constructing a clipping box based on the bounding box includes:

[0013] If the length of the bounding box is greater than the width of the bounding box, then a first product between a preset length multiple and the length is determined, and a first square is constructed with the first product as the side length. If the bounding box is located in the middle of the first square, then the first square is used as the clipping frame.

[0014] Optionally, the step of constructing a clipping box based on the bounding box includes:

[0015] If the width of the bounding box is greater than the length of the bounding box, then a second product between a preset width multiple and the width is determined, and a second square is constructed with the second scene as the side length. If the bounding box is located in the middle of the second square, then the second square is used as the clipping frame.

[0016] Optionally, the step of constructing a clipping box based on the bounding box includes:

[0017] The bounding box can be used as a clipping box, or the bounding box can be enlarged by a preset multiple to obtain a clipping box.

[0018] Optionally, the step of classifying and identifying the defect features of the target defect image according to a preset classification model to obtain the identification result includes:

[0019] If there are multiple target defect images, the defect features of each target defect image are determined, and each defect feature is classified to obtain multiple defect features belonging to the same category.

[0020] Based on a preset classification model, a target classification model is determined for the defect features belonging to the same class. The target defect images corresponding to the defect features belonging to the same class are then classified and identified according to the target classification model to obtain the identification result.

[0021] Optionally, before the step of detecting whether a defect exists in the detected image according to a preset defect detection model, the following steps are included:

[0022] The detected image is input into a preset defect detection model, and the confidence level and cross-union ratio in the defect detection model are adjusted to preset thresholds.

[0023] Furthermore, to achieve the above objectives, the present invention also provides a cascaded defect detection device, comprising:

[0024] The acquisition module is used to acquire a detection image of the object to be detected and to detect whether there are defects in the detection image according to a preset defect detection model.

[0025] The cropping module is used to crop the detected image to obtain a target defect image if a defect exists in the detected image.

[0026] The identification module is used to classify and identify the defect features of the target defect image according to a preset classification model, and obtain the identification result;

[0027] The determination module is used to determine the defect result of the object to be detected based on the recognition result.

[0028] In addition, to achieve the above objectives, the present invention also provides a cascade defect detection device, which includes a memory, a processor, and a cascade defect detection program stored in the memory and executable on the processor. When the cascade defect detection program is executed by the processor, it implements the steps of the cascade defect detection method as described above.

[0029] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a cascade defect detection program, which, when executed by a processor, implements the steps of the cascade defect detection method as described above.

[0030] This invention acquires an image of the object to be inspected, and when a defect is detected in the image based on a defect detection model, the image is cropped to obtain a target defect image. Then, a classification model is used to classify and identify the defect features of the target defect image. Based on the identification results, the defect of the object to be inspected is determined. This avoids the missed and false detections that occur with manual inspection in existing technologies. Furthermore, the dual detection using both a defect detection model and a classification model improves the accuracy of defect detection. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the terminal / device structure of the hardware operating environment involved in the embodiments of the present invention;

[0032] Figure 2 This is a flowchart illustrating the first embodiment of the cascaded defect detection method of the present invention;

[0033] Figure 3 This is a schematic diagram of the device modules of the cascaded defect detection device of the present invention;

[0034] Figure 4 This is a schematic diagram of the target defect image in the cascaded defect detection method of the present invention;

[0035] Figure 5 This is a schematic diagram of a defect standard in the cascaded defect detection method of the present invention;

[0036] Figure 6 This is a schematic diagram of the OK and NG classes in the cascaded defect detection method of the present invention;

[0037] Figure 7 This is a schematic diagram of a process in the cascaded defect detection method of the present invention;

[0038] Figure 8 This is a schematic diagram of the clipping frame in the cascaded defect detection method of the present invention;

[0039] Figure 9 This is a schematic diagram illustrating the defect category classification and identification in the cascaded defect detection method of the present invention;

[0040] Figure 10 This is a schematic diagram illustrating the defect size classification and identification in the cascaded defect detection method of the present invention.

[0041] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0042] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0043] like Figure 1 As shown, Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of the present invention.

[0044] In this embodiment of the invention, the terminal is a cascaded defect detection device.

[0045] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0046] Optionally, the terminal may also include a camera, RF (Radio Frequency) circuitry, sensors, audio circuitry, a WiFi module, and so on. These sensors may include light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display screen according to the ambient light level, while the proximity sensor can turn off the display screen and / or backlight when the terminal device is moved to the ear. Of course, the terminal device may also be equipped with other sensors such as a gyroscope, barometer, hygrometer, thermometer, and infrared sensor, which will not be elaborated upon here.

[0047] Those skilled in the art will understand that Figure 1 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0048] like Figure 1 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a cascaded defect detection program.

[0049] exist Figure 1 In the terminal shown, network interface 1004 is mainly used to connect to the backend server and communicate with it; user interface 1003 is mainly used to connect to the client (user terminal) and communicate with it; while processor 1001 can be used to call the cascaded defect detection program stored in memory 1005 and perform the following operations:

[0050] Reference Figure 2 This invention provides a cascade defect detection method. In a first embodiment of the cascade defect detection method, the cascade defect detection method includes the following steps:

[0051] Step S10: Obtain the detection image of the object to be detected, and detect whether there is a defect in the detection image according to the preset defect detection model;

[0052] Currently, defect detection is performed manually, resulting in high false negative and false positive rates, leading to low accuracy in the final defect detection results. Furthermore, technologies such as anomaly detection, target classification, and target detection are difficult to implement in complex and ever-changing industrial scenarios. Therefore, this embodiment employs a technology cascading approach, using multiple models to jointly detect defects and allowing for flexible adjustments for different industrial scenarios. Specifically, firstly, target detection is used to detect defects. Then, defects are cropped using various bounding boxes. Finally, multiple classification models are used to re-judge defects based on their characteristics, thereby reducing false positives, improving the detection performance of individual models, and increasing the accuracy of defect detection. This also avoids the false negatives and false positives that can occur when a single detection model performs defect detection.

[0053] Therefore, in this embodiment, it is necessary to first determine which objects need to be detected, designating these objects as the objects to be detected. Then, an image of the objects to be detected is captured by a pre-installed camera to obtain a detection image of the objects. The objects to be detected can be various parts, such as earphone shells, fabric, and data cables.

[0054] Once the image to be detected is determined, defects can be directly detected using a pre-set defect detection model. For example, when the defect detection model is YOLOv5, the image to be detected can be input into the YOLOv5 model for training. The confidence score and IOU (Intersection over Union) in the YOLOv5 model can be adjusted to achieve a low false negative rate and a high false positive rate. The confidence score represents the degree to which the network believes in the detected target. Low confidence scores result in more bounding boxes, while high confidence scores may filter out correct predictions. IOU is a common evaluation metric in object detection. Defects can be undesirable features on the object being detected, such as color differences or stains.

[0055] Step S20: If there is a defect in the detected image, the detected image is cropped to obtain a target defect image;

[0056] When a defect is detected in the image, it can be cropped from the target defect image based on the bounding box detected in the image. This cropping is done using the bounding box of the defect's location. The bounding box is determined by the defect detection model; the module generates detection boxes during image detection, marking any detected defects or similar areas. These detection boxes then serve as the bounding boxes for cropping. The resulting target defect image must include at least the content contained within the bounding box, such as the defect itself.

[0057] Step S30: Classify and identify the defect features of the target defect image according to the preset classification model to obtain the identification result;

[0058] Step S40: Determine the defect result of the object to be detected based on the identification result.

[0059] In this embodiment, after acquiring a target defect image, it can be directly classified using a pre-set classification model. Furthermore, when multiple target defect images exist, different classification models can be used simultaneously to classify each image. Specifically, when there is only one target defect image, the defect features, such as defect type or defect size, are first determined. Then, the standard corresponding to the defect is determined, and the defect in the target defect image is judged according to a classification model, such as a binary classification model. If the defect in the target defect image matches the standard, an NG image is generated, and the object to be detected is confirmed to have the defect feature. However, if the defect in the target defect image does not match the standard, it can be considered a false detection result of the defect detection model, and an OK image is generated, indicating that the object to be detected does not have the defect feature. Here, the NG image represents a correct defect image, and the OK image represents a falsely detected defect image.

[0060] When there are multiple target defect images, each target defect image can be detected simultaneously. This involves detecting defect features in each target defect image and classifying and summarizing these features. For example, if the defect feature is a defect type, defects belonging to the same defect type and their corresponding target defect images are identified. Then, a classification model corresponding to that defect type is determined, and all target defect images corresponding to defects belonging to the same defect type are input into this classification model for training to obtain the training results. The training results for all defect images are then summarized to obtain the summary result, which is used as the recognition result. During model training, the classification model needs to first determine the standard corresponding to the defect type. Based on the standard, it is determined whether the defects in the target defect images meet the standard. If they meet the standard, it is determined that the object to be detected has a defect of that type. The standard corresponding to the defect type can be a user-defined standard. Alternatively, it can be based on defect size or special defect types, and the processing method is the same as that for defect types, which will not be elaborated here.

[0061] After determining the recognition results corresponding to all target defect images, the presence of a defect in the object to be detected can be determined based on the recognition results, thus compensating for false detections that may occur during defect detection by the defect detection model. If the defect detection model determines that a defect exists in the target detection image corresponding to the object to be detected, and the standard match between the defect and its features is determined again by the classification model, then the object to be detected is confirmed to have a defect, and this is taken as the defect result of the object to be detected.

[0062] In addition, to aid in understanding the principle of the cascaded defect detection method in this embodiment, examples are provided below.

[0063] For example, first, acquire a specific industrial dataset and determine the corresponding detection images. Then, use the YOLOv5 model to detect defects. When defects are found, crop them. The cropping method can be to crop the image by 1.1 times the larger of the width and height of the bounding box. This ensures that the target defect image obtained after defect cropping retains some background information while maintaining a square shape. Next, determine the defect features of the target defect image. Based on these features, select multiple binary classification models for training. Summarize the training results to obtain the final defect detection result. If, after evaluation, the 03 class defect features in the target defect image are found to be special, then these 03 class defect features are processed separately. For example, ... Figure 4 As shown, all defects of type 03 are distributed on the outermost ring of the circle. The standard corresponding to defect type 03 is as follows: Figure 5 As stated, if a Class 03 defect is located within one-third of the dashed line, it is classified as NG; if a Class 03 defect is not located within the dashed line, it is classified as OK. Therefore, based on the detection results, all Class 03 defects can be rotated to the 9 o'clock position of the industrial dataset, and 100*100 defect boxes can be cropped. These boxes are then classified into OK and NG categories according to the criteria corresponding to Class 03 defects, forming a binary classification model. The image corresponding to the NG type can be as follows: Figure 6 (a) and Figure 6 As shown in (b) above, the image corresponding to the OK type can be as follows: Figure 6 (c) and Figure 6 As shown in (d) in the figure.

[0064] For example, such as Figure 7 As shown, after acquiring the detection image, it is first processed by a single detection model. This involves inputting the detection image into a YOLOv5 model for training, resulting in a low false negative rate and high false positive rate. Then, the detection image undergoes defect cropping and defect feature determination. Defect cropping can include direct bounding box cropping, adding part of the background for cropping, or adding part of the background for cropping to a square. Defect feature determination is performed by judging the defect category or defect size. After defect feature determination, the image is then trained using multiple classification models. Specifically, defect feature 1 is input into classification model 1; defect feature 2 into classification model 2; defect feature n-1 into classification model n-1; and defect feature n into classification model n. The training results of all models (1-n) are then summarized to obtain a summary result. Based on this summary result, it is determined whether there is a defect. If no defect is found, the image is considered OK; otherwise, it is considered NG (Not Detected).

[0065] In this embodiment, by acquiring a detection image of the object to be detected, and upon determining the presence of a defect in the detection image based on the defect detection model, the detection image is cropped to obtain a target defect image. Then, a classification model is used to classify and identify the defect features of the target defect image. Based on the identification results, the defect status of the object to be detected is determined. This avoids the missed and false detections that occur due to manual inspection in existing technologies. Furthermore, the dual detection using both the defect detection model and the classification model improves the accuracy of defect detection.

[0066] Furthermore, based on the first embodiment of the present invention described above, a second embodiment of the cascaded defect detection method of the present invention is proposed. In this embodiment, the step S20 of the above embodiment, which involves cropping defects from the detected image to obtain a target defect image, is refined to include:

[0067] Step a: Determine the bounding box corresponding to the defect in the detected image, and construct a cropping box based on the bounding box;

[0068] Step b: Crop the detected image according to the cropping box to obtain the target defect image.

[0069] In this embodiment, when performing defect detection on the image, it is necessary to first determine the bounding boxes corresponding to the detected defects in the image. These bounding boxes can be determined based on the detection boxes generated by the defect detection model. Then, cropping boxes are constructed based on these bounding boxes, and the detected image is then cropped to remove defects. For example, as... Figure 8 As shown in (a), the target defect image is obtained by directly cropping using the bounding box as the cropping frame. This method results in a more accurate and precise image of the target defect. For example, as... Figure 8 As shown in (b), by adding 21% background to the bounding box for cropping, the cropping box now includes the bounding box, and its length and width are greater than the bounding box. Therefore, the target defect image obtained by defect cropping in this way includes both the defect and some background information. For example, as... Figure 8 As shown in (c), the maximum of the length and width of the bounding box is determined, and the clipping box is constructed at 1.1 times the maximum. At this time, the clipping box includes the bounding box, and its length and width are greater than the bounding box. The target defect image obtained by defect clipping at this time includes the defect and ensures that there is a certain background, and it is square.

[0070] In this embodiment, a cropping box is constructed based on the bounding box corresponding to the defect in the detected image, and then the detected image is cropped based on the cropping box to obtain the target defect image, thereby ensuring the accuracy of the obtained target defect image.

[0071] Specifically, the step of constructing a clipping box based on the bounding box includes:

[0072] Step a1: If the length of the bounding box is greater than the width of the bounding box, then determine the first product between the preset length multiple and the length, and construct a first square with the first product as the side length. If the bounding box is located in the middle of the first square, then use the first square as the clipping frame.

[0073] In this embodiment, when constructing the clipping box, the length and width of the acquisition bounding box can be obtained first, and the length and width can be compared. If the length of the bounding box is greater than the width, the length of the bounding box can be enlarged to ensure that the target defect image obtained by clipping has a certain background. When enlarging the length, the enlargement factor is determined first, that is, the length factor is determined, such as 1.1 times. The length factor can be adjusted and modified according to user needs. Then, the length is enlarged according to the length factor, that is, the length factor is multiplied by the length to obtain a first product, and then a first square is constructed using the first product as the side length. The side length of the first square is the first product, and the defect and the bounding box are located at the middle position of the first square. Then, this first square is used as the clipping box.

[0074] In this embodiment, when the length of the bounding box is greater than the width of the bounding box, a first square is constructed with the first product between the length and the length multiple as the side length, and then the first square is used as the clipping box, thereby ensuring the effectiveness of the clipping box.

[0075] Specifically, the step of constructing a clipping box based on the bounding box includes:

[0076] Step a2: If the width of the bounding box is greater than the length of the bounding box, then determine the second product between the preset width multiple and the width, and construct a second square with the second scene as the side length. If the bounding box is located in the middle of the second square, then use the second square as the clipping frame.

[0077] In this embodiment, when constructing the clipping box, the length and width of the acquisition bounding box can be obtained first, and the length and width can be compared. If the width of the bounding box is greater than the length, the width of the bounding box can be enlarged to ensure that the target defect image obtained by clipping has a certain background. When enlarging the width, the enlargement factor is determined first, that is, the width factor, such as 1.1 times. The width factor can be adjusted and modified according to user needs. Then, the width is enlarged according to the width factor, that is, the width factor is multiplied by the width to obtain a second product, and then a second square is constructed with the second product as the side length. The side length of the second square is the second product, and the defect and the bounding box are located at the middle position of the second square. Then, this second square is used as the clipping box.

[0078] In this embodiment, when the width of the bounding box is greater than the length of the bounding box, a second square is constructed with the second product between the width and a multiple of the width as the side length, and then the second square is used as the clipping box, thereby ensuring the effectiveness of the clipping box.

[0079] Specifically, the step of constructing a clipping box based on the bounding box includes:

[0080] Step a3: Use the bounding box as a clipping box, or enlarge the bounding box by a preset bounding box multiple to obtain a clipping box.

[0081] In this embodiment, when constructing the clipping frame, the bounding box can be used directly as the clipping frame for defect clipping. Alternatively, the bounding box can be enlarged by a certain multiple (i.e., the bounding box multiple). For example, if the bounding box multiple is set to 1.1, the length and width of the bounding box can be multiplied by 1.1 to obtain the final clipping frame.

[0082] In this embodiment, by directly using the bounding box as the clipping box, or by enlarging the bounding box according to the bounding box multiplier, a clipping box is obtained, so that the target defect image obtained by clipping with the clipping box contains at least the identified defect.

[0083] Further, the step of classifying and identifying the defect features of the target defect image according to a preset classification model to obtain the identification result includes:

[0084] Step c: If there are multiple target defect images, determine the defect features of each target defect image and classify each defect feature to obtain multiple defect features belonging to the same category.

[0085] Step d: Determine the target classification model corresponding to the defect features belonging to the same class according to the preset classification model, and classify and identify the target defect images corresponding to the defect features belonging to the same class according to the target classification model to obtain the identification result.

[0086] In this embodiment, when classifying and recognizing a target defect image, it is necessary to first determine the defect features of the target defect image, such as defect category or defect size, or special type of defect (such as a defect type specially marked by the user). The defect features are compared with multiple preset defect features to determine the matching defect features among the preset defect features. Then, the classification model corresponding to the matching defect feature is used as the target classification model corresponding to the defect feature. The target defect image is then input into the target classification model for classification and recognition to obtain the recognition result.

[0087] In a scenario where multiple target defect images exist, each image needs to be classified. This classification can be based on the defect features of each image. For example, when the defect feature is a defect category, features belonging to the same category can be grouped together, such as all belonging to category 1. Then, from among several pre-set classification models, one corresponding to category 1 is selected as the target classification model. All target defect images corresponding to category 1 are then classified and identified using this target model. After classifying and identifying all target defect images, the results are summarized. If an NG (Not Detected) image is found in the summary results, the object to be detected is confirmed to have a defect. The identification results include OK (OK) images and NG images. NG images are correct defect images, while OK images are false positives.

[0088] For example, when the defect feature is a defect category, such as Figure 9 As shown, after determining the defect category, if the defect category matches the first type of defect, the target defect image is classified and identified using binary classification model 1; if the defect category matches the second type of defect, the target defect image is classified and identified using binary classification model 2; if the defect category matches the (n-1)th type of defect, the target defect image is classified and identified using binary classification model n-1; if the defect category matches the nth type of defect, the target defect image is classified and identified using binary classification model n. Finally, the classification and identification results corresponding to all target defect images of the object to be detected are summarized to obtain the summary result, and then the object to be detected is determined to have a defect based on the summary result.

[0089] When the defect feature is the defect size, such as Figure 10 As shown, after determining the defect size, if the defect size matches defect size 1, the target defect image is classified and identified using classification model 1; if the defect size matches defect size 2, the target defect image is classified and identified using classification model 2; if the defect size matches defect size n-1, the target defect image is classified and identified using classification model n-1; if the defect size matches defect size n, the target defect image is classified and identified using classification model n. Finally, the classification and identification results corresponding to all target defect images of the object to be detected are summarized to obtain the summary result, and then the object to be detected is determined to have a defect based on the summary result.

[0090] In this embodiment, the target classification model corresponding to the defect features of the target defect image is determined from multiple preset classification models, and then the target defect image is input into the target classification model for classification and recognition to obtain the recognition result, thereby ensuring the accuracy of the obtained recognition result.

[0091] Further, before the step of detecting whether a defect exists in the detected image according to a preset defect detection model, the following steps are included:

[0092] Step e: Input the detected image into a preset defect detection model, and adjust the confidence level and crossover ratio in the defect detection model to a preset threshold.

[0093] In this embodiment, after acquiring the detection image and inputting it into the pre-set defect detection model, the confidence level and cross-union ratio (CUI) in the defect detection model need to be lowered, such as by modulating a pre-set preset threshold. This preset threshold needs to be sufficiently small to generate enough detection boxes during defect detection, thus avoiding missed detections.

[0094] In this embodiment, by adjusting the confidence level and cross-union ratio to a preset threshold after inputting the detected image into the defect detection model, the phenomenon of missed detection is avoided and the accuracy of defect detection is improved.

[0095] In addition, refer to Figure 3 The present invention also provides a cascaded defect detection device, comprising:

[0096] The acquisition module A10 is used to acquire the detection image of the object to be detected, and to detect whether there is a defect in the detection image according to the preset defect detection model;

[0097] The cropping module A20 is used to crop the detected image to obtain a target defect image if a defect exists in the detected image.

[0098] The recognition module A30 is used to classify and recognize the defect features of the target defect image according to a preset classification model, and obtain the recognition result;

[0099] The determination module A40 is used to determine the defect result of the object to be detected based on the recognition result.

[0100] Optionally, the cropping module A20 is used for:

[0101] Determine the bounding box corresponding to the defect in the detected image, and construct a cropping box based on the bounding box;

[0102] The detected image is cropped according to the cropping box to obtain the target defect image.

[0103] Optionally, the cropping module A20 is used for:

[0104] If the length of the bounding box is greater than the width of the bounding box, then a first product between a preset length multiple and the length is determined, and a first square is constructed with the first product as the side length. If the bounding box is located in the middle of the first square, then the first square is used as the clipping box.

[0105] Optionally, the cropping module A20 is used for:

[0106] If the width of the bounding box is greater than the length of the bounding box, then a second product between a preset width multiple and the width is determined, and a second square is constructed with the second scene as the side length. If the bounding box is located in the middle of the second square, then the second square is used as the clipping frame.

[0107] Optionally, the cropping module A20 is used for:

[0108] The bounding box can be used as a clipping box, or the bounding box can be enlarged by a preset multiple to obtain a clipping box.

[0109] Optionally, the identification module A30 is used for:

[0110] If there are multiple target defect images, the defect features of each target defect image are determined, and each defect feature is classified to obtain multiple defect features belonging to the same category.

[0111] Based on a preset classification model, a target classification model is determined for the defect features belonging to the same class. The target defect images corresponding to the defect features belonging to the same class are then classified and identified according to the target classification model to obtain the identification result.

[0112] Optionally, module A10 is used for:

[0113] The detected image is input into a preset defect detection model, and the confidence level and cross-union ratio in the defect detection model are adjusted to preset thresholds.

[0114] The steps for implementing each functional module of the cascade defect detection device can be referred to in the various embodiments of the cascade defect detection method of the present invention, and will not be repeated here.

[0115] Furthermore, the present invention also provides a cascade defect detection device, the cascade defect detection device comprising: a memory, a processor, and a cascade defect detection program stored in the memory; the processor is used to execute the cascade defect detection program to implement the steps of the above embodiments of the cascade defect detection method.

[0116] The present invention also provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the steps of the embodiments of the cascaded defect detection method described above.

[0117] The specific implementation of the computer-readable storage medium of the present invention is basically the same as the embodiments of the cascaded defect detection method described above, and will not be repeated here.

[0118] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0119] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0120] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0121] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for detecting cascaded defects, characterized in that, The cascaded defect detection method includes the following steps: Acquire a detection image of the object to be detected, and detect whether there are defects in the detection image according to a preset defect detection model; If a defect exists in the detected image, the detected image is cropped to obtain a target defect image; The defect features of the target defect image are classified and identified according to a preset classification model to obtain the identification result; wherein, the defect features include defect category, defect size, or special class defect; the preset classification model includes a binary classification model. When the binary classification model is trained, and the defect features in the target defect image include a 03 class defect belonging to the special class defect, the 03 class defect is rotated to the 9 o'clock position of the industrial dataset, and a 100*100 defect box is cropped. According to the standard corresponding to the 03 class defect, it is divided into OK class and NG class to form a binary classification model. The defect result of the object to be detected is determined based on the recognition result; The step of cropping defects from the detected image to obtain a target defect image includes: Determine the bounding box corresponding to the defect in the detected image, and construct a cropping box based on the bounding box; The detected image is cropped according to the cropping box to obtain the target defect image; The step of constructing a clipping box based on the bounding box includes at least one of the following: If the length of the bounding box is greater than the width of the bounding box, then a first product between a preset length multiple and the length is determined, and a first square is constructed with the first product as the side length. If the bounding box is located in the middle of the first square, then the first square is used as the clipping box. If the width of the bounding box is greater than the length of the bounding box, then a second product between a preset width multiple and the width is determined, and a second square is constructed with the second product as the side length. If the bounding box is located in the middle of the second square, then the second square is used as the clipping box. The bounding box is enlarged by a preset bounding box multiple to obtain a clipping box; The step of classifying and identifying the defect features of the target defect image according to a preset classification model to obtain the identification result includes: If there are multiple target defect images, the defect features of each target defect image are determined, and each defect feature is classified to obtain multiple defect features belonging to the same category. The target classification model corresponding to the defect features belonging to the same class is determined according to the preset classification model. The target defect images corresponding to the defect features belonging to the same class are classified and identified according to the target classification model to obtain the identification result. The target classification model corresponding to each class of defect features is different.

2. The cascaded defect detection method as described in claim 1, characterized in that, The step of constructing a clipping box based on the bounding box includes: Use the bounding box as the clipping box.

3. The cascaded defect detection method as described in any one of claims 1-2, characterized in that, Before the step of detecting whether there is a defect in the detection image according to the preset defect detection model, the following steps are included: The detected image is input into a preset defect detection model, and the confidence level and cross-union ratio in the defect detection model are adjusted to preset thresholds.

4. A cascaded defect detection device, characterized in that, The cascaded defect detection device includes: The acquisition module is used to acquire a detection image of the object to be detected and to detect whether there are defects in the detection image according to a preset defect detection model. A cropping module is used to crop the detected image to obtain a target defect image if a defect exists in the detected image. The cropping process includes: determining the bounding box corresponding to the defect in the detected image and constructing a cropping box based on the bounding box; cropping the detected image based on the cropping box to obtain the target defect image; wherein constructing the cropping box based on the bounding box includes at least one of the following: if the length of the bounding box is greater than the width of the bounding box, determining a first product between a preset length multiple and the length, constructing a first square with the first product as the side length, and using the first square as the cropping box if the bounding box is located in the middle of the first square; if the width of the bounding box is greater than the length of the bounding box, determining a second product between a preset width multiple and the width, constructing a second square with the second product as the side length, and using the second square as the cropping box if the bounding box is located in the middle of the second square; and enlarging the bounding box by a preset bounding box multiple to obtain the cropping box. The recognition module is used to classify and recognize the defect features of the target defect image according to a preset classification model, and obtain the recognition result; wherein, the defect features include defect category, defect size, or special class defect; the preset classification model includes a binary classification model. When training the binary classification model, and the defect features in the target defect image include a Class 03 defect belonging to the special class defect, the Class 03 defect is rotated to the 9 o'clock position of the industrial dataset, and a 100*100 defect box is cropped. It is then divided into OK and NG classes according to the criteria corresponding to Class 03 defects, forming a binary classification model; wherein, the The method describes classifying and identifying the defect features of the target defect image according to a preset classification model to obtain identification results, including: if there are multiple target defect images, determining the defect features of each target defect image and classifying each defect feature to obtain multiple defect features belonging to the same category; determining the target classification model corresponding to the defect features belonging to the same category according to the preset classification model; classifying and identifying the target defect images corresponding to the defect features belonging to the same category according to the target classification model to obtain identification results; wherein, the target classification model corresponding to each category of defect features is different; The determination module is used to determine the defect result of the object to be detected based on the recognition result.

5. A cascaded defect detection device, characterized in that, The cascaded defect detection device includes: a memory, a processor, and a cascaded defect detection program stored in the memory and executable on the processor. When the cascaded defect detection program is executed by the processor, it implements the steps of the cascaded defect detection method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a cascade defect detection program, which, when executed by a processor, implements the steps of the cascade defect detection method as described in any one of claims 1 to 3.

Images