Detection method and device, electronic equipment and storage medium

Abstract

The invention provides a detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-detected screw hole image, inputting the to-be-detected screw hole image into a pre-trained foreign matter detection model, and obtaining a detection result. By adopting the method, the first to-be-trained model is trained through the high-quality sample screw hole image data and the non-high-quality sample screw hole image data to obtain the foreign matter detection model, so that the generalization of the foreign matter detection model is improved, and the foreign matter detection model can adapt to various detection environments and different screw holes; and the foreign matter detection model is adopted to detect the foreign matter in the screw hole, so that the accuracy of a screw hole detection result is improved, and false detection and missing detection are reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technologies, and in particular, to a detection method, apparatus, electronic device, and storage medium. Background Art

[0002] During the production process of laptop computers, it is necessary to detect foreign objects in the screw holes. However, due to the different materials and detection environments of the screw holes, it is difficult to generalize the detection method for screw holes. For example, when an automatic screw locking machine locks screws, in order to avoid damaging the material plate by repeatedly screwing, it is necessary to detect whether the screw hole is occupied by foreign objects. However, the materials and colors of different material plates may be different, and the working environments and light source conditions of different automatic screw locking machines may also be different, resulting in large differences in the imaging of screw holes. The accuracy of the detection results based on the imaging of screw holes is also affected, and false detections and missed detections are likely to occur. Therefore, how to improve the accuracy of the detection results of screw holes has become an urgent technical problem to be solved. Summary of the Invention

[0003] The present disclosure provides a detection method, apparatus, electronic device, and storage medium.

[0004] According to a first aspect of the present disclosure, a detection method is provided. The method includes:

[0005] Obtain an image of a screw hole to be detected;

[0006] Input the image of the screw hole to be detected into a pre-trained foreign object detection model to obtain a detection result;

[0007] Wherein, the foreign object detection model is obtained by training a first model to be trained based on high-quality sample screw hole image data and non-high-quality sample screw hole image data.

[0008] In an implementable manner, the method for training the foreign object detection model includes:

[0009] Obtain high-quality sample screw hole image data and non-high-quality sample screw hole image data;

[0010] Determine the labels corresponding to the high-quality sample screw hole image data and the labels corresponding to the non-high-quality sample screw hole image data;

[0011] Preprocess the high-quality sample screw hole image data and the non-high-quality sample screw hole image data to obtain multiple processed images as target sample screw hole images. The preprocessing is used to reduce the image quality;

[0012] Input the target sample screw hole image into the first model to be trained to obtain a detection result of foreign objects in the screw hole;

[0013] Determine the loss function value of the first model to be trained based on the label corresponding to the target sample screw hole image and the screw hole foreign object detection result;

[0014] If the loss function value is less than the preset loss threshold, determine the current first model to be trained as the foreign object detection model;

[0015] If the loss function value is not less than the preset loss threshold, obtain a new target sample screw hole image, and return to execute the step of inputting the target sample screw hole image into the first model to be trained until the loss function value is less than the preset loss threshold or the number of iterations reaches the preset number of iterations, and determine the first model to be trained as the foreign object detection model.

[0016] In an implementable manner, the high-quality sample screw hole image data includes at least one high-quality sample screw hole image carrying a label. Determining the label corresponding to the high-quality sample screw hole image data and the label corresponding to the non-high-quality sample screw hole image data includes:

[0017] Train a second model to be trained based on the high-quality sample screw hole image carrying the label and the label to obtain a label annotation model;

[0018] Based on the label annotation model, determine the label of the high-quality sample screw hole image data without a label and the label of the non-high-quality sample screw hole image data.

[0019] In an implementable manner, preprocessing the high-quality sample screw hole image data and the non-high-quality sample screw hole image data includes:

[0020] Determine the first pixel distribution feature of the high-quality sample screw hole image data and the second pixel distribution feature of the non-high-quality sample screw hole image data;

[0021] Based on the first pixel distribution feature and the second pixel distribution feature, perform first preprocessing on the high-quality sample screw hole image data and the non-high-quality sample screw hole image data, or perform second preprocessing on the high-quality sample screw hole image data.

[0022] In an implementable manner, performing second preprocessing on the high-quality sample screw hole image data includes:

[0023] Screen the high-quality sample screw hole images to be processed from the high-quality sample screw hole image data according to a first preset ratio;

[0024] Perform brightness enhancement processing on the high-quality sample screw hole images to be processed.

[0025] In one implementable manner, the first preprocessing of the high-quality sample screw hole image data and the non-high-quality sample screw hole image data includes:

[0026] Screen the to-be-processed high-quality sample screw hole images from the high-quality sample screw hole image data, and screen the to-be-processed non-high-quality sample screw hole images from the non-high-quality sample screw hole image data according to a second preset ratio;

[0027] Perform rotation processing on the to-be-processed high-quality sample screw hole images and the to-be-processed non-high-quality sample screw hole images; and / or,

[0028] Screen the to-be-processed high-quality sample screw hole images from the high-quality sample screw hole image data, and screen the to-be-processed non-high-quality sample screw hole images from the non-high-quality sample screw hole image data according to a third preset ratio;

[0029] Perform flipping processing on the to-be-processed high-quality sample screw hole images and the to-be-processed non-high-quality sample screw hole images.

[0030] In one implementable manner, the inputting the to-be-detected screw hole image into a pre-trained foreign object detection model to obtain a detection result includes:

[0031] Input the to-be-detected screw hole image into a pre-trained foreign object detection model;

[0032] The foreign object detection model determines the intersection over union ratio between the first region containing foreign objects and the second region not containing foreign objects in the to-be-detected screw hole image. If the intersection over union ratio is not less than a preset ratio threshold, output a detection result that there are foreign objects in the screw hole.

[0033] According to a second aspect of the present disclosure, there is provided a detection device, the device includes:

[0034] An image acquisition module, configured to acquire a to-be-detected screw hole image;

[0035] A detection module, configured to input the to-be-detected screw hole image into a pre-trained foreign object detection model to obtain a detection result; wherein, the foreign object detection model is obtained by training a first to-be-trained model based on high-quality sample screw hole image data and non-high-quality sample screw hole image data.

[0036] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0037] At least one processor; and

[0038] A memory communicatively connected to the at least one processor;

[0039] Wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the present disclosure.

[0040] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method described in the present disclosure.

[0041] By using the detection method of the present disclosure, an image of a screw hole to be detected is obtained, and the image of the screw hole to be detected is input into a pre-trained foreign object detection model to obtain a detection result. The first model to be trained is trained with high-quality sample screw hole image data and non-high-quality sample screw hole image data to obtain a foreign object detection model, which improves the generalization of the foreign object detection model, enables the foreign object detection model to adapt to various detection environments and different screw holes, and uses the foreign object detection model to detect foreign objects in screw holes, improving the accuracy of the screw hole detection result and reducing the occurrence of false detection and missed detection.

[0042] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] By referring to the accompanying drawings and reading the following detailed description, the above and other objects, features, and advantages of the exemplary embodiments of the present disclosure will become easily understood. In the drawings, several embodiments of the present disclosure are shown in an exemplary rather than restrictive manner, wherein:

[0044] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0045] Figure 1 Shows a schematic implementation flow diagram of an information processing method provided by an embodiment of the present application;

[0046] Figure 2 Shows an image of a screw hole to be detected provided by an embodiment of the present disclosure;

[0047] Figure 3 Shows a schematic training flow diagram of a foreign object detection model provided by an embodiment of the present application;

[0048] Figure 4 Shows a schematic diagram of pixel distribution provided by an embodiment of the present application;

[0049] Figure 5 Shows another schematic diagram of pixel distribution provided by an embodiment of the present application;

[0050] Figure 6 Shows a schematic diagram of a detection result of applying the detection method provided by an embodiment of the present disclosure;

[0051] Figure 7 Shows a schematic structural diagram of an information processing device provided by an embodiment of the present application;

[0052] Figure 8 Shows a schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure. Detailed implementation manners

[0053] To make the objectives, features, and advantages of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present disclosure.

[0054] Due to the different materials and detection environments of the screw holes, it is difficult to generalize the detection method for screw holes. At present, the accuracy of the results of screw hole detection based on screw hole imaging is also affected, and false detection and missed detection are likely to occur. Therefore, in order to improve the accuracy of the results of screw hole detection, the present application provides a detection method, device, electronic device, and storage medium. The electronic device provided by the present application can be devices such as mobile phones, computers, and tablet computers.

[0055] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application.

[0056] Figure 1 Shows a schematic implementation flowchart of an information processing method provided by an embodiment of the present application. As Figure 1 shown, the method includes:

[0057] S101, obtain an image of a screw hole to be detected.

[0058] In the present disclosure, electronic devices such as mobile phones, computers, and tablet computers include one or more screw holes. An image acquisition device such as a mobile phone camera or a camera can be used to acquire an image including the screw holes of the electronic device. The grayscale image of the acquired image including the screw holes of the electronic device can be used as the image of the screw hole to be detected. Figure 2 Shows an image of a screw hole to be detected provided by an embodiment of the present disclosure. As Figure 2 shown, Figure 2 (a), Figure 2 (b), Figure 2 (c) and Figure 2(d) shows a grayscale image of each screw hole of the laptop case, where each grayscale image of the screw hole shows the situation where the screw is occupied by foreign objects.

[0059] S102, input the image of the screw hole to be detected into a pre-trained foreign object detection model to obtain a detection result.

[0060] Among them, the foreign object detection model is obtained by training a first model to be trained based on high-quality sample screw hole image data and non-high-quality sample screw hole image data.

[0061] Using the detection method of the present disclosure, an image of a screw hole to be detected is obtained, and the image of the screw hole to be detected is input into a pre-trained foreign object detection model to obtain a detection result. The foreign object detection model is obtained by training a first model to be trained with high-quality sample screw hole image data and non-high-quality sample screw hole image data, which improves the generalization of the foreign object detection model, enabling the foreign object detection model to adapt to various detection environments and different screw holes. Detecting foreign objects in screw holes using the foreign object detection model improves the accuracy of the screw hole detection results and reduces the occurrence of false detections and missed detections.

[0062] In a possible implementation manner, Figure 3 shows a schematic diagram of a training process of the foreign object detection model provided by an embodiment of the present application, as Figure 3 shown, the foreign object detection model training method includes:

[0063] S301, obtain high-quality sample screw hole image data and non-high-quality sample screw hole image data.

[0064] In the present disclosure, each sample screw hole image can be classified into high-quality sample screw hole image data and non-high-quality sample screw hole image data according to the quality of the collected image. The high-quality sample screw hole image data includes screw hole images that meet one or more high-quality image conditions. Among them, the high-quality image conditions include: the brightness is within a preset brightness interval; the average image gray value is within a preset numerical range. The exposure time is within a preset duration range, and the preset brightness interval, preset duration range, and preset numerical range can be set according to the application scenario. For example, the preset numerical range can be set to [120, 150] or [110, 140], etc.

[0065] In the present disclosure, it is also possible to determine whether the collected image is high-quality sample screw hole image data or non-high-quality sample screw hole image data according to the magnitude of the image entropy. If the image entropy of the collected image is greater than a preset entropy value, it can be determined that the image belongs to the high-quality sample screw hole image data; otherwise, it belongs to the non-high-quality sample screw hole image data. The image entropy can be calculated using the following formula:

[0066]

[0067] Wherein, H is the image entropy, and p(i) is the probability that a pixel point with a gray value of i appears. Wherein, n i is the number of pixel points with a gray value of i, and N is the total number of pixels in the image.

[0068] S302. Determine the labels corresponding to the high-quality sample screw hole image data and the labels corresponding to the non-high-quality sample screw hole image data.

[0069] In a possible implementation manner, the high-quality sample screw hole image data may include at least one high-quality sample screw hole image carrying a label. The determining the labels corresponding to the high-quality sample screw hole image data and the labels corresponding to the non-high-quality sample screw hole image data may include steps A1 - A2:

[0070] Step A1. Train a second model to be trained based on the high-quality sample screw hole image carrying the label and the label to obtain a label annotation model.

[0071] Step A2. Based on the label annotation model, determine the labels of the high-quality sample screw hole image data without carrying a label and the labels of the non-high-quality sample screw hole image data.

[0072] In the present disclosure, the screw hole defects of some images in the high-quality sample screw hole image data can be annotated. For example, the screw hole defects of all images in the high-quality sample screw hole image data can be annotated, or the screw hole defects of 10% of the images in the high-quality sample screw hole image data can be annotated. The screw hole defects may include that the screw hole is blocked by foreign objects and the screw hole is not standard, etc. In the present disclosure, a rectangular frame or a circular frame can be used to annotate the screw holes with defects in some images of the high-quality sample screw hole image data, and the screw holes without defects can be not annotated.

[0073] In the present disclosure, the screw hole image that has been annotated can be used as the first high-quality screw hole image. Train a deep learning model according to the first high-quality screw hole image that has been annotated to obtain a pre-annotation model. Use the pre-annotation model to annotate the screw hole images that have not been annotated in the high-quality sample screw hole image data and the screw holes in each non-high-quality sample screw hole image data to obtain the labels of other screw hole images in the high-quality sample screw hole image data and the labels of each non-high-quality sample screw hole image data. Among them, the number of images in the non-high-quality sample screw hole image data is usually greater than the number of images in the high-quality sample screw hole image data. In the present disclosure, for the screw hole images annotated by the pre-annotation model, the annotation information can also be made more accurate by means of manual adjustment.

[0074] In the present disclosure, manual annotation can also be used to label the tags corresponding to the high-quality sample screw hole image data and the tags corresponding to the non-high-quality sample screw hole image data.

[0075] S303. Preprocess the high-quality sample screw hole image data and the non-high-quality sample screw hole image data to obtain multiple processed images, all of which are used as target sample screw hole images. The preprocessing is used to reduce the image quality.

[0076] In the present disclosure, by preprocessing the high-quality sample screw hole image data and the non-high-quality sample screw hole image data, the image quality is reduced.

[0077] In a possible implementation manner, the preprocessing of the high-quality sample screw hole image data and the non-high-quality sample screw hole image data may include steps B1 - B2:

[0078] Step B1. Determine the first pixel distribution feature of the high-quality sample screw hole image data and the second pixel distribution feature of the non-high-quality sample screw hole image data.

[0079] In the present disclosure, histogram statistics can be performed on the high-quality sample screw hole image data. By the number of pixel points in each pixel interval in the image, the pixel point distribution feature of the image is obtained as the first pixel distribution feature. Similarly, histogram statistics can be performed on the non-high-quality sample screw hole image data. By the number of pixel points in each pixel interval in the image, the pixel point distribution feature of the image is obtained as the second pixel distribution feature. For example, the 0 - 255 pixels can be divided into 16 pixel intervals, and each pixel interval is an interval including 16 pixels. Figure 4 FIG. shows a pixel distribution schematic diagram provided by an embodiment of the present application. As Figure 4 shown, the abscissa of the image represents the pixel interval, Bins is the pixel interval, the ordinate of the image represents the probability that the pixel point falls into the pixel interval, and "#of pixels" represents the distribution probability of the pixels. Figure 4 FIG. shows the pixel distribution feature of the high-quality sample screw hole image data. As Figure 4 shown, there are more pixels distributed in the 10th Bin and the 1st Bin in the high-quality sample screw hole image data. Figure 5 FIG. shows another pixel distribution schematic diagram provided by an embodiment of the present application. As Figure 5 shown, the abscissa of the image represents the pixel interval, Bins is the pixel interval, the ordinate of the image represents the probability that the pixel point falls into the pixel interval, and "#of pixels" represents the distribution probability of the pixels. Figure 5 FIG. shows the pixel distribution feature of the non-high-quality sample screw hole image data on the surface of the collected metal device. As Figure 5As shown, the pixel distribution in this image is mainly concentrated in the 16th Bin, which is caused by the overexposure easily generated by the surface reflection of the metal in the meantime.

[0080] Step B2, based on the first pixel distribution feature and the second pixel distribution feature, perform a first preprocessing on the high-quality sample screw hole image data and the non-high-quality sample screw hole image data, or perform a second preprocessing on the high-quality sample screw hole image data.

[0081] In a possible implementation manner, the performing the second preprocessing on the high-quality sample screw hole image data may include steps C1-C2:

[0082] Step C1, screen the high-quality sample screw hole images to be processed from the high-quality sample screw hole image data according to a first preset ratio.

[0083] Step C2, perform a brightness enhancement process on the high-quality sample screw hole images to be processed.

[0084] In the present disclosure, the first preset ratio may be set to 30% or 40%, etc. The high-quality sample screw hole image data of the first preset ratio can be randomly selected as the high-quality sample screw hole images to be processed, and a brightness enhancement process is performed on the high-quality sample screw hole images to be processed. Optionally, the following formula can be used to perform a brightness enhancement process on the high-quality sample screw hole images to be processed:

[0085]

[0086] Wherein, I(i,j) is the pixel at the (i,j) position of the enhanced sample screw hole image I, α is the scaling coefficient of the pixel gray value at the corresponding position of the high-quality sample screw hole image to be processed, and β is the coefficient for left and right translation of the histogram.

[0087] The magnitudes of α and β can be set according to the pixel distribution of the image. α takes any number between 0.8 and 1.5, and β takes any integer between -50 and 50. The value of β satisfies: For example, if the image is overexposed, α can be set to 1.5 and β takes 50.

[0088] In a possible implementation manner, the performing the first preprocessing on the high-quality sample screw hole image data and the non-high-quality sample screw hole image data may include steps D1-D3:

[0089] Step D1, screen the high-quality sample screw hole images to be processed from the high-quality sample screw hole image data according to a second preset ratio, and screen the non-high-quality sample screw hole images to be processed from the non-high-quality sample screw hole image data.

[0090] In the present disclosure, the second preset ratio can be set to 30% or 35%, etc.

[0091] Step D2, perform rotation processing on the to-be-processed high-quality sample screw hole image and the to-be-processed non-high-quality sample screw hole image; and / or,

[0092] Step D3, screen the to-be-processed high-quality sample screw hole image from the high-quality sample screw hole image data and screen the to-be-processed non-high-quality sample screw hole image from the non-high-quality sample screw hole image data according to a third preset ratio.

[0093] In the present disclosure, since the characteristic information of foreign object detection in the screw hole focuses on the circular hole, in order to improve the accuracy of screw hole defect detection, rotation processing can be performed on the to-be-processed high-quality sample screw hole image and the to-be-processed non-high-quality sample screw hole image. The image obtained after rotating the circular screw hole is distributed the same as the real image, but images at different angles can enhance the robustness of the trained foreign object detection model.

[0094] In the present disclosure, for the to-be-processed high-quality sample screw hole image and the to-be-processed non-high-quality sample screw hole image with the second preset ratio, half of the images can be randomly rotated 90 degrees to the left or right, and half of the images can be randomly rotated 15 degrees to the left or right. Optionally, the getRotationMatrix2D in the Opencv library can be used to obtain the 2D coordinate transformation matrix of each pixel point in the image, and the image is rotated through the 2D coordinate transformation matrix, and linear interpolation is used for the pixel values. Among them, the 2D coordinate transformation matrix R:

[0095]

[0096] where γ is the radian value of the rotation angle.

[0097] Step D4, perform flipping processing on the to-be-processed high-quality sample screw hole image and the to-be-processed non-high-quality sample screw hole image.

[0098] In the present disclosure, the third preset ratio can be set to 10% or 15%. The to-be-processed high-quality sample screw hole image is screened from the high-quality sample screw hole image data and the to-be-processed non-high-quality sample screw hole image is screened from the non-high-quality sample screw hole image data according to the third preset ratio. Since the screw hole is an axisymmetric shape, flipping processing can be performed on the to-be-processed high-quality sample screw hole image and the to-be-processed non-high-quality sample screw hole image. Flipping processing can further enhance the robustness of the trained foreign object detection model. Optionally, for the to-be-processed high-quality sample screw hole image and the to-be-processed non-high-quality sample screw hole image with the third preset ratio, half of the images can be randomly rotated up and down, and half of the images can be randomly flipped left and right.

[0099] Both the rotated and flipped images are determined as the target screw hole images.

[0100] S304, Input the target sample screw hole image into the first model to be trained, and obtain the detection result of foreign objects in the screw hole.

[0101] S305, Based on the label corresponding to the target sample screw hole image and the detection result of foreign objects in the screw hole, determine the loss function value of the first model to be trained.

[0102] S306, If the loss function value is less than the preset loss threshold, determine the current first model to be trained as the foreign object detection model.

[0103] Among them, the preset loss threshold can be set according to the actual application scenario.

[0104] S307, If the loss function value is not less than the preset loss threshold, obtain a new target sample screw hole image, and return to execute the step of inputting the target sample screw hole image into the first model to be trained until the loss function value is less than the preset loss threshold or the number of iterations reaches the preset number of iterations, and then determine the first model to be trained as the foreign object detection model.

[0105] Among them, the preset number of iterations can be set to 500 times or 1000 times, etc.

[0106] In a possible implementation manner, the step of inputting the screw hole image to be detected into the pre-trained foreign object detection model to obtain the detection result may include steps E1 - E2:

[0107] Step E1, Input the screw hole image to be detected into the pre-trained foreign object detection model.

[0108] Step E2, The foreign object detection model determines the intersection over union ratio between the first area containing foreign objects and the second area not containing foreign objects in the screw hole image to be detected. If the intersection over union ratio is not less than the preset ratio threshold, output the detection result that there are foreign objects in the screw hole.

[0109] In the present disclosure, the preset ratio threshold can be set according to the actual application scenario. For example, it can be set to 0.6 or 0.7, etc.

[0110] Figure 6 Shows a schematic diagram of a detection result of applying the detection method provided by the embodiments of the present disclosure. As Figure 6 shown, Figure 6 (a), Figure 6 (b) and Figure 6 (c) show the detection effects under different image conditions, different scenarios, and different screw hole defects. Figure 6 (a) and Figure 6(b) shows the detection results of no foreign objects in the screw holes under different image conditions and different scenarios. Figure 6 (c) shows the detection results of foreign objects in the screw holes.

[0111] The foreign object detection model provided by the present disclosure can adapt to various detection environments and the detection of screw holes on different products. By annotating a small number of high-quality screw hole images, a large number of other screw hole images can be annotated, reducing the image annotation time and saving the costs of image acquisition and annotation. By preprocessing the images by simulating the conditions in various scenarios, the image data covering scenarios in different detection scenarios is enriched, further enhancing the generalization ability and robustness of the foreign object detection model and improving the detection accuracy.

[0112] Based on the same inventive concept, according to the detection method provided by the above embodiments of the present disclosure, correspondingly, another embodiment of the present disclosure further provides a detection device, the structural schematic diagram of which is as Figure 7 shown, and specifically includes:

[0113] An image acquisition module 701, configured to acquire an image of a screw hole to be detected;

[0114] A detection module 702, configured to input the image of the screw hole to be detected into a pre-trained foreign object detection model to obtain a detection result; wherein, the foreign object detection model is obtained by training a first model to be trained based on high-quality sample screw hole image data and non-high-quality sample screw hole image data.

[0115] By using the detection device of the present disclosure, an image of a screw hole to be detected is acquired, and the image of the screw hole to be detected is input into a pre-trained foreign object detection model to obtain a detection result. The foreign object detection model is obtained by training a first model to be trained based on high-quality sample screw hole image data and non-high-quality sample screw hole image data, improving the generalization of the foreign object detection model, so that the foreign object detection model can adapt to various detection environments and different screw holes. Using the foreign object detection model to detect foreign objects in the screw holes improves the accuracy of the screw hole detection results and reduces the occurrence of false detection and missed detection.

[0116] In an implementable manner, the device further includes:

[0117] A model training module (not shown in the figure) is used to obtain high-quality sample screw hole image data and non-high-quality sample screw hole image data; determine the labels corresponding to the high-quality sample screw hole image data and the labels corresponding to the non-high-quality sample screw hole image data; preprocess the high-quality sample screw hole image data and the non-high-quality sample screw hole image data to obtain multiple processed images, all of which are used as target sample screw hole images, and the preprocessing is used to reduce the image quality; input the target sample screw hole images into a first model to be trained to obtain screw hole foreign object detection results; determine the loss function value of the first model to be trained based on the labels corresponding to the target sample screw hole images and the screw hole foreign object detection results; if the loss function value is less than a preset loss threshold, determine the current first model to be trained as a foreign object detection model; if the loss function value is not less than the preset loss threshold, obtain new target sample screw hole images, and return to execute the step of inputting the target sample screw hole images into the first model to be trained until the loss function value is less than the preset loss threshold or the number of iterations reaches a preset number of iterations, and determine the first model to be trained as a foreign object detection model.

[0118] In an implementable manner, the high-quality sample screw hole image data includes at least one high-quality sample screw hole image with a label, and the model training module is specifically configured to train a second model to be trained based on the high-quality sample screw hole image with a label and the label to obtain a label annotation model; determine the labels of the high-quality sample screw hole image data without a label and the non-high-quality sample screw hole image data based on the label annotation model.

[0119] In an implementable manner, the model training module is specifically configured to determine the first pixel distribution feature of the high-quality sample screw hole image data and the second pixel distribution feature of the non-high-quality sample screw hole image data; based on the first pixel distribution feature and the second pixel distribution feature, perform first preprocessing on the high-quality sample screw hole image data and the non-high-quality sample screw hole image data, or perform second preprocessing on the high-quality sample screw hole image data.

[0120] In an implementable manner, the model training module is specifically configured to screen the high-quality sample screw hole images to be processed from the high-quality sample screw hole image data according to a first preset ratio; perform brightness enhancement processing on the high-quality sample screw hole images to be processed.

[0121] In one implementable embodiment, the model training module is specifically configured to screen the to-be-processed high-quality sample screw hole images from the high-quality sample screw hole image data and screen the to-be-processed non-high-quality sample screw hole images from the non-high-quality sample screw hole image data according to a second preset ratio; perform rotation processing on the to-be-processed high-quality sample screw hole images and the to-be-processed non-high-quality sample screw hole images; and / or screen the to-be-processed high-quality sample screw hole images from the high-quality sample screw hole image data and screen the to-be-processed non-high-quality sample screw hole images from the non-high-quality sample screw hole image data according to a third preset ratio; perform flipping processing on the to-be-processed high-quality sample screw hole images and the to-be-processed non-high-quality sample screw hole images.

[0122] In one implementable embodiment, the detection module is specifically configured to input the screw hole image to be detected into a pre-trained foreign object detection model; the foreign object detection model determines the intersection over union (IoU) between the first region containing foreign objects and the second region not containing foreign objects in the screw hole image to be detected, and if the intersection over union is not less than a preset ratio threshold, outputs a detection result that there is a foreign object in the screw hole.

[0123] According to the embodiments of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0124] Figure 8 FIG. shows a schematic block diagram of an exemplary electronic device 800 that can be used to implement the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0125] As Figure 8 shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0126] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as a keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as a disk, optical disc, etc.; and communication unit 809, such as a network card, modem, wireless communication transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0127] Computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 801 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 801 executes the various methods and processes described above, such as the detection method. For example, in some embodiments, the detection method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by computing unit 801, one or more steps of the detection method described above can be executed. Alternatively, in other embodiments, computing unit 801 can be configured to execute the detection method in any other suitable manner (e.g., by means of firmware).

[0128] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] The program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine, or entirely on the remote machine or server.

[0130] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] In order to provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input from the user may be received in any form (including acoustic input, speech input, or tactile input).

[0132] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0133] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0134] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. There is no limitation herein.

[0135] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" can explicitly or implicitly include at least one such feature. In the description of this disclosure, "a plurality" means two or more, unless otherwise specifically defined.

[0136] As described above, the above are only specific embodiments of this disclosure, but the protection scope of this disclosure is not limited thereto. Any person skilled in the art within the technical scope disclosed in this disclosure can easily think of changes or substitutions, which should all be covered by the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be subject to the protection scope of the claims.

Claims

1. A detection method, characterized in that, The method includes: Obtain an image of a screw hole to be detected; Input the image of the screw hole to be detected into a pre-trained foreign object detection model to obtain a detection result; Among them, the foreign object detection model is obtained by training a first model to be trained based on high-quality sample screw hole image data and non-high-quality sample screw hole image data.

2. The method according to claim 1, wherein The method for training the foreign object detection model includes: Obtain high-quality sample screw hole image data and non-high-quality sample screw hole image data; Determine the labels corresponding to the high-quality sample screw hole image data and the labels corresponding to the non-high-quality sample screw hole image data; Perform preprocessing on the high-quality sample screw hole image data and the non-high-quality sample screw hole image data to obtain multiple processed images, all of which are used as target sample screw hole images. The preprocessing is used to reduce the image quality; Input the target sample screw hole image into the first model to be trained to obtain a detection result of foreign objects in the screw hole; Based on the label corresponding to the target sample screw hole image and the detection result of foreign objects in the screw hole, determine the loss function value of the first model to be trained; If the loss function value is less than a preset loss threshold, determine the current first model to be trained as the foreign object detection model; If the loss function value is not less than the preset loss threshold, obtain a new target sample screw hole image, and return to execute the step of inputting the target sample screw hole image into the first model to be trained until the loss function value is less than the preset loss threshold or the number of iterations reaches a preset number of iterations, and determine the first model to be trained as the foreign object detection model.

3. The method according to claim 1, wherein The high-quality sample screw hole image data includes at least one high-quality sample screw hole image with a label. Determining the labels corresponding to the high-quality sample screw hole image data and the labels corresponding to the non-high-quality sample screw hole image data includes: Train a second model to be trained based on the high-quality sample screw hole image with a label and the label to obtain a label annotation model; Based on the label annotation model, determine the labels of the high-quality sample screw hole image data without a label and the labels of the non-high-quality sample screw hole image data.

4. The method according to claim 1, characterized in that Performing preprocessing on the high-quality sample screw hole image data and the non-high-quality sample screw hole image data includes: Determine the first pixel distribution feature of the high-quality sample screw hole image data and the second pixel distribution feature of the non-high-quality sample screw hole image data; Based on the first pixel distribution feature and the second pixel distribution feature, perform first preprocessing on the high-quality sample screw hole image data and the non-high-quality sample screw hole image data, or perform second preprocessing on the high-quality sample screw hole image data.

5. The method according to claim 4, characterized in that Performing second preprocessing on the high-quality sample screw hole image data includes: Screen the high-quality sample screw hole images to be processed from the high-quality sample screw hole image data according to a first preset ratio; Perform brightness enhancement processing on the high-quality sample screw hole images to be processed.

6. The method according to claim 4, wherein Performing first preprocessing on the high-quality sample screw hole image data and the non-high-quality sample screw hole image data includes: Screen the to-be-processed high-quality sample screw hole images from the high-quality sample screw hole image data, and screen the to-be-processed non-high-quality sample screw hole images from the non-high-quality sample screw hole image data according to a second preset ratio; Perform rotation processing on the to-be-processed high-quality sample screw hole images and the to-be-processed non-high-quality sample screw hole images; and / or, Screen the to-be-processed high-quality sample screw hole images from the high-quality sample screw hole image data, and screen the to-be-processed non-high-quality sample screw hole images from the non-high-quality sample screw hole image data according to a third preset ratio; Perform flipping processing on the to-be-processed high-quality sample screw hole images and the to-be-processed non-high-quality sample screw hole images.

7. The method according to claim 1, characterized in that, The inputting the to-be-detected screw hole image into a pre-trained foreign object detection model to obtain a detection result includes: Input the to-be-detected screw hole image into a pre-trained foreign object detection model; The foreign object detection model determines the intersection over union between the first region containing foreign objects and the second region not containing foreign objects in the to-be-detected screw hole image. If the intersection over union is not less than a preset ratio threshold, output a detection result that there are foreign objects in the screw hole.

8. A detection device, characterized in that, The device includes: An image acquisition module for acquiring a to-be-detected screw hole image; A detection module for inputting the to-be-detected screw hole image into a pre-trained foreign object detection model to obtain a detection result; wherein, the foreign object detection model is obtained by training a first to-be-trained model based on high-quality sample screw hole image data and non-high-quality sample screw hole image data.

9. An electronic device, comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in any one of claims 1-7 is implemented.

10. A storage medium containing computer-executable instructions, where the computer-executable instructions are used to execute the method described in any one of claims 1-7 when executed by a computer processor.

Images