Watch component defect detection method and device, computer equipment and storage medium

Abstract

The invention relates to a watch component defect detection method and device, computer equipment and a storage medium. The method comprises the following steps: performing component region positioning on an acquired to-be-detected watch component image by adopting a trained component positioning model to obtain at least one component region image of the same or different component types; respectively inputting each component region image into a trained scratch detection model matched with the component type of the component region image for processing, and outputting a scratch defect detection result; respectively inputting each component region image into a trained part segmentation model for processing, and outputting a predicted part contour image; performing part defect detection on at least one part contour in the predicted part contour image to obtain a part defect detection result; and generating a target defect detection result including the scratch defect detection result and the part defect detection result. According to the invention, the accuracy and efficiency of defect detection of the watch component can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a watch component defect detection method, device, computer equipment and storage medium. Background Art

[0002] During the production process of watch accessories, defects such as bruises, scratches, solder point offset and cable deformation may affect the function of the accessories. Therefore, it is necessary to detect the defects in the watch accessories to ensure the quality of the watch accessories.

[0003] In the current defect detection process of watch accessories, deep learning is usually used to complete defect detection. However, since the types of defects generated by different components in the accessories may be the same or different, and the defect detection method of the existing technology does not take into account the differences in component defects, the accuracy of component defect detection is low. In addition, due to the large number of defects on the surface of the component, the defect detection efficiency of the existing defect detection method is low. Summary of the invention

[0004] Based on this, it is necessary to provide a watch component defect detection method, device, computer equipment and storage medium to address the above-mentioned technical problems, which can improve the accuracy and efficiency of watch component defect detection.

[0005] In a first aspect, the present application provides a method for detecting defects in watch components, comprising:

[0006] Using the trained component positioning model to locate the component area of ​​the acquired watch component image to be inspected, and obtaining at least one component area image of the same or different component type;

[0007] Input each component area image into a trained scratch detection model that matches its component type for processing, and output the scratch defect detection result;

[0008] Input each component area image into the trained part segmentation model for processing, and output the predicted part contour image;

[0009] Performing part defect detection on at least one part contour in the predicted part contour image to obtain a part defect detection result;

[0010] Generate target defect detection results including scratch defect detection results and part defect detection results.

[0011] In a second aspect, the present application provides a watch component defect detection device, comprising:

[0012] A positioning module, used to use a trained component positioning model to locate the component area of ​​the acquired watch component image to be detected, and obtain at least one component area image of the same or different component type;

[0013] A scratch detection module is used to input the images of each component area into a trained scratch detection model that matches its component type for processing and output scratch defect detection results;

[0014] A contour prediction module is used to input each component area image into the trained part segmentation model for processing and output a predicted part contour image;

[0015] A part detection module, used to perform part defect detection on at least one part contour in the predicted part contour image to obtain a part defect detection result;

[0016] A generation module is used to generate target defect detection results including scratch defect detection results and part defect detection results.

[0017] In a third aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method when executing the computer program.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0019] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and the computer program implements the steps in the above method when executed by a processor.

[0020] Compared with the prior art, the above-mentioned watch component defect detection method, device, computer equipment and storage medium realize the automatic detection of surface defects of watch components; different scratch detection models are used to detect different types of component defects respectively, thereby improving the accuracy of component defect detection; the component positioning model is used to pre-position the component area, and the corresponding scratch detection model is used to perform defect detection on the positioned components, thereby improving the defect detection efficiency. In addition, by performing part defect detection on the parts deployed on the components or between components, the comprehensiveness of the defect detection of watch components is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 An application environment diagram of a watch component defect detection method provided in an embodiment of the present application;

[0022] Figure 2A flowchart of a watch component defect detection method provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of component area positioning of a watch component image to be detected by a component positioning model provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of part contour prediction for a component area image by a part segmentation model provided in an embodiment of the present application;

[0025] Figure 5 A schematic flow chart of another watch component defect detection method provided in an embodiment of the present application;

[0026] Figure 6 A structural block diagram of a watch component defect detection device provided in an embodiment of the present application;

[0027] Figure 7 An internal structure diagram of a computer device provided in an embodiment of the present application;

[0028] Figure 8 An internal structure diagram of another computer device provided in an embodiment of the present application;

[0029] Fig. 9 An internal structure diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0031] The watch component defect detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through a communication network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0032] like Figure 2As shown, the embodiment of the present application provides a method for detecting defects in watch components, which is applied to Figure 1 The terminal 102 or the server 104 in the example is used for explanation. It is understandable that the computer device may include at least one of the terminal and the server. The method includes the following steps:

[0033] S202: Use the trained component positioning model to locate the component area of ​​the acquired component image of the watch to be inspected, and obtain at least one component area image of the same or different component type.

[0034] S204, inputting each component area image into a trained scratch detection model that matches its component type for processing, and outputting a scratch defect detection result.

[0035] S206, inputting each component area image into the trained part segmentation model for processing, and outputting a predicted part contour image.

[0036] S208: Perform part defect detection on at least one part contour in the predicted part contour image to obtain a part defect detection result.

[0037] S210, generating target defect detection results including scratch defect detection results and part defect detection results.

[0038] There may be one or more watch components in the watch component image to be detected; the component types of each watch component may be the same or different; for example, there are connectors, cables and brackets in the watch component image A to be detected. The connectors, cables and brackets are all watch components.

[0039] The component localization model is used to locate the area where the component is located in the component image of the watch to be detected. The component localization model can be pre-trained by relevant technical personnel, or an existing localization model can be used, such as SSD (Single Shot MultiBox Detector).

[0040] For example, Figure 3 A schematic diagram of component area positioning of a watch component image to be detected by a component positioning model is shown. In the watch component image M to be detected, there are watch components of component type a, watch components of component type b, and watch components of component type c; the watch component image M to be detected is input into the trained component positioning model, and the component positioning model performs component positioning on watch components of different component types, and generates component area images corresponding to the corresponding component types, which are component area images of component type a, component area images of component type b, and component area images of component type c.

[0041] In order to further improve the model prediction accuracy of the component positioning model, the watch component positioning scenario can be combined to generate a sample data set related to the scenario, and the sample data set related to this scenario can be used for model training to obtain the component positioning model.

[0042] Specifically, a sample image of a watch component is obtained; wherein the sample image of the watch component includes at least two watch components of different component types. The area where the watch component is located in the sample image of the watch component is marked with a position and a type, and a sample label of the sample image of the watch component is obtained. The sample image of the watch component is input into a preset component positioning model for processing to obtain a model output result; according to the model output result and the sample label, the component positioning model is trained until the model training end condition is met to obtain a trained component positioning model. Among them, the model training end condition can be pre-set by relevant technical personnel. For example, the model training end condition can be that the number of training times reaches a set number threshold, or it can also be that the loss value tends to be stable, etc. This embodiment does not limit this.

[0043] It should be noted that, since the types of scratches or scratches existing in watch components of different component types are different, in order to further improve the detection accuracy of component defects, different scratch detection models can be used to perform defect detection on component scratches of different component types.

[0044] For the scratch detection model of any component type, the model training method may specifically be: obtain a component defect sample image under the component type; wherein the component defect sample image includes at least one scratch defect. The component defect sample image is labeled with the scratch defect position and defect type to obtain a sample label of the component defect sample image. The component defect sample image is input into a preset scratch detection model for processing to obtain a model output result; according to the model output result and the sample label, the scratch detection model is trained until the model training end condition is met to obtain a trained scratch detection model. The model training end condition may be pre-set by relevant technical personnel. For example, the model training end condition may be that the number of training times reaches a set number threshold, or that the loss value tends to be stable, etc. This embodiment does not limit this.

[0045] It should be noted that in addition to defects in the components themselves, defects in watch components also include defects in parts deployed on the components or located on the solder joints between components. Among them, the parts contained in the solder joints on the components or between components can be patches, tapes or copper sheets, etc. Therefore, in order to ensure the comprehensiveness of watch component defect detection, it is also necessary to perform part defect detection on the parts located on the surface of the watch components. Specifically, the area where the parts are located in the above-mentioned component area images can be located, and part defect detection can be performed on the contours of the located parts.

[0046] Among them, the part segmentation model is used to perform part area segmentation, which can be obtained by relevant technicians through pre-training based on a large number of part contour extraction samples. For any component area image, the part segmentation model segments the part contour in the component area image to obtain a predicted part contour image. It should be noted that the number of contours of the part contours contained in the predicted part contour image can be one or more, and the positions and shapes of different types of parts in the component area image are different.

[0047] like Figure 4 The schematic diagram of part contour prediction of a component region image by a part segmentation model is shown. The part segmentation model predicts the part contour of the component region image M to obtain a predicted part contour image Mm. The predicted part contour image Mm includes part contour a, part contour b and part contour c.

[0048] Part defect detection is performed on different part contours respectively. Specifically, different part detection methods can be used according to the part type to perform part defect detection on the part contour. For example, for patch-type parts, defect detection can be performed by template matching. Specifically, the part contour can be matched with the standard contour template of the part for similarity; if the similarity is higher than the preset similarity threshold, it is considered that the part defect detection has passed and the part has no defects; if the similarity is not higher than the preset similarity threshold, it is considered that the part defect detection has failed and the part has defects.

[0049] Generate a target defect detection result including a scratch defect detection result and a part defect detection result, and use the target defect detection result as the watch component defect detection result of the watch component image to be detected. The target defect detection result can be used by subsequent relevant technicians to perform defect analysis on the corresponding watch component product.

[0050] It can be seen that in the embodiments of the present application, different scratch detection models are used to perform defect detection on different types of component defects, thereby improving the accuracy of component defect detection. In the process of defect detection, the component area is pre-positioned by using the component positioning model, and then the positioned component is subjected to defect detection in combination with the corresponding scratch detection model, thereby improving the efficiency of component defect detection and realizing the automatic detection of surface defects of watch components. In addition, by using the part segmentation model to locate the contour area of ​​the parts deployed on the component or between components, and performing part defect detection on the contour of the positioned part, the comprehensiveness of the defect detection of the watch component is improved.

[0051] It should be noted that since watch components are non-appearance parts, some defects that do not affect their functions are acceptable defects. Therefore, in most cases, only some defects that affect the realization of their functions need to be detected. For some shallow scratches, they do not damage the function of the component and can be considered as good products; while for some deeper scratches, they will damage the function of the component and seriously affect the realization of the component function, and can be considered as defective products. However, deep scratches and shallow scratches are very similar in the image and are not easy to distinguish; and different types of components have different degrees of scratches that damage the function of the component. Therefore, in the model training process of scratch detection models for different component types, in order to enable the model to better distinguish deep scratches from shallow scratches, specific model training measures need to be adopted to appropriately improve the model training process or sample pre-processing process, so as to effectively avoid the model from overfitting background features. Improve the accuracy of model training and achieve effective recognition of deep and shallow scratches.

[0052] In some embodiments, before inputting each component region image into a trained scratch detection model matching its component type for processing and outputting a scratch defect detection result, the method further includes:

[0053] Obtain sample images of shallow scratched components with shallow scratched areas and sample images of deep scratched components with deep scratched areas under corresponding component types; the number of sample images of shallow scratched components and the number of sample images of deep scratched components meet a preset sample quantity balance condition; the sample images of shallow scratched components have shallow scratched sample label values; the sample images of deep scratched components have deep scratched sample label values;

[0054] generating a scratch sample training set including component sample images with shallow scratches and component sample images with deep scratches;

[0055] Inputting the scratch sample training set into a preset scratch detection model for processing, and outputting a first prediction result for a component sample image with shallow scratches and a second prediction result for a component sample image with deep scratches;

[0056] Determining a first cross entropy loss value according to a first prediction result of the shallow scratch component sample image and a shallow scratch sample label value;

[0057] Determining a second cross entropy loss value according to a second prediction result of the deep scratch component sample image and the deep scratch sample label value;

[0058] Determining a confusion loss value according to the first prediction result and the second prediction result;

[0059] The scratch detection model is trained according to the first cross entropy loss value, the second cross entropy loss value, and the confusion loss value until the model training end condition is met, thereby obtaining a trained scratch detection model for the corresponding component type.

[0060] The sample quantity balance condition may be that the sample quantity of the deep scratch component sample image and the shallow scratch component sample image is the same or the quantity difference is not greater than a preset difference range. The difference range may be preset by relevant technical personnel, for example, it may be 2% to 4% of the total number of samples.

[0061] The shallow scratch sample label values ​​of the shallow scratch component sample images and the deep scratch sample label values ​​of the deep scratch component sample images can be pre-labeled by relevant technicians. The shallow scratch sample label values ​​may include the position coordinates of the shallow scratch contour area and the shallow scratch type; the deep scratch sample label values ​​may include the position coordinates of the deep scratch contour area and the deep scratch type.

[0062] According to the first prediction result of the shallow scratch component sample image and the shallow scratch sample label value, based on the preset cross entropy loss function, a first cross entropy loss value can be determined; and, according to the second prediction result of the deep scratch component sample image and the deep scratch sample label value, based on the preset cross entropy loss function, a second cross entropy loss value can be determined.

[0063] It should be noted that in order to more effectively distinguish the pixel features of deep scratches and shallow scratches during the training process, the pixel-level confusion loss is added. For the sample data in a batch during the training process, the confusion loss is expressed as follows:

[0064]

[0065] Among them, p θ (y i |x 1 ) represents the probability of predicting the shallow scratch component sample image as the i-th defect type in the first prediction result; p θ (y i |x 2) represents the probability of predicting the shallow scratch component sample image as the i-th defect type in the second prediction result; θ is the network model parameter; n is the defect category, including shallow scratch defects and deep scratch defects.

[0066] The first cross entropy loss value, the second cross entropy loss value and the confusion loss value are used as target loss values, and based on the target loss values, the scratch detection model is trained until the model training end condition is met, and a trained scratch detection model for the corresponding component type is obtained. The model training end condition can be pre-set by relevant technical personnel according to actual needs, for example, the target loss value tends to be stable or reaches a set loss threshold, etc.

[0067] It can be seen that in this embodiment, the scratch detection model is trained by using sample-balanced deep scratch component sample images and shallow scratch component sample images, and pixel-level confusion loss is introduced during the model training process. The sample data set and loss function levels are optimized, so that the model can better distinguish between deep scratches and shallow scratches, thereby improving the scratch detection model's recognition accuracy for deep and shallow scratches, and further improving the scratch detection model's recognition accuracy for scratch defects of corresponding component types.

[0068] In order to avoid overfitting the background area features outside the scratch area during the model training process, and thus further improve the accuracy of the scratch defect detection model training, the sample images in the scratch sample training set can be appropriately improved.

[0069] In some embodiments, after acquiring a sample image of a shallow scratched component with a shallow scratched area and a sample image of a deep scratched component with a deep scratched area under a corresponding component type, the method further comprises:

[0070] Performing regional image extraction on the shallow scratch area in the shallow scratch component sample image to obtain the shallow scratch area image; performing regional image extraction on the deep scratch area in the deep scratch component sample image to obtain the deep scratch area image;

[0071] Using at least one background image generation method to generate a plurality of background images;

[0072] The shallow scratch area image is fused with each background image to obtain a fused shallow scratch component sample image; the deep scratch area image is fused with each background image to obtain a fused deep scratch component sample image;

[0073] Accordingly, a scratch sample training set including a component sample image with shallow scratches and a component sample image with deep scratches is generated, including:

[0074] A scratch sample training set is generated, which includes a shallow scratch component sample image, a deep scratch component sample image, a fused shallow scratch component sample image, and a fused deep scratch component sample image.

[0075] The background image generation method may be a random color modification method, a random noise addition method, or the like.

[0076] It can be seen that in this embodiment, by performing image augmentation on the sample image and fusing different background areas to generate a fused sample image, overfitting of background features outside the scratch area during the model training process is reduced to a certain extent, thereby improving the accuracy of model training.

[0077] Figure 5 A flowchart of another watch component defect detection method provided in an embodiment of the present application. This embodiment is further optimized on the basis of the above embodiment. Figure 5 As shown, the method includes:

[0078] S502: Use the trained component positioning model to locate the component area of ​​the acquired watch component image to be inspected, and obtain at least one component area image of the same or different component type.

[0079] S504: Input each component area image to a scratch detection model that matches its component type for processing, and output a scratch defect detection result.

[0080] S506, inputting each component area image into the trained part segmentation model for processing, and outputting a predicted part contour image.

[0081] S508: Determine the part types corresponding to the respective part contours in the predicted part contour image.

[0082] S510 , performing appearance defect detection on each part contour according to the part type corresponding to each part contour, and obtaining an appearance defect detection result.

[0083] S512. Perform position defect detection on each part contour according to the part type corresponding to each part contour to obtain a position defect detection result.

[0084] S514, generating a part defect detection result including an appearance defect detection result and a position defect detection result.

[0085] S516: Generate target defect detection results including scratch defect detection results and part defect detection results.

[0086] The part type may include a patch part type and a tape part type, etc., which may be specifically determined by performing edge detection on the part contour.

[0087] Different methods of appearance defect detection and position defect detection are used for different types of parts, so defect detection can be carried out in a targeted manner, thereby improving the accuracy of defect detection. Among them, the appearance defect detection of parts can include template matching detection, edge detection and curvature detection.

[0088] In some embodiments, according to the part type corresponding to each part contour, each part contour is subjected to appearance defect detection to obtain an appearance defect detection result, including:

[0089] According to the part type corresponding to each part contour, the part contour template, edge detection condition, arc detection condition and angle detection condition corresponding to each part contour are determined respectively;

[0090] Perform similarity matching on each part contour and its corresponding part contour template respectively to obtain the template matching detection result;

[0091] According to the edge detection conditions corresponding to the contours of each part, edge detection is performed on the contours of each part to obtain edge detection results;

[0092] According to the curvature detection conditions corresponding to the contours of each part, the curvature detection is performed on the contours of each part to obtain the curvature detection results;

[0093] According to the angle detection conditions corresponding to the contours of each part, the angle detection is performed on the contours of each part to obtain the angle detection results;

[0094] The appearance defect detection results are determined based on the template matching detection results, edge detection results, curvature detection results and angle detection results corresponding to the contours of each part.

[0095] The part contour template may be a template pre-constructed based on a standard part contour, and different types of parts may correspond to different part contour templates.

[0096] For any part contour, a template matching the part type can be selected as the part contour template of the part contour according to the part type of the part contour; for example, part type A corresponds to part contour template a, and part type B corresponds to part contour template b. The part contour is matched with the part contour template by similarity, for example, cosine similarity or Euclidean distance can be used to match the similarity to obtain template matching similarity; if the template matching similarity is not less than a preset similarity threshold, it can be determined that the template matching detection of the part contour template has passed; if the template matching similarity is less than the preset similarity threshold, it can be determined that the template matching detection of the part contour template has failed.

[0097] Among them, edge detection can be a gradient detection of the contour edge of the part contour. Different part types correspond to different edge detection conditions. For example, for the contour of a tape type part, the edge detection condition can be whether the tape edge is horizontal or vertical; for the contour of a patch type part, the edge detection condition can be whether there is a convexity or concaveness on the contour edge.

[0098] Specifically, the Hough transform method or the least squares fitting method can be used to detect whether the part contour is horizontal or vertical. The edge pixel point analysis method can be used to detect whether the part contour has convexities or concavities.

[0099] Among them, the arc detection condition can be to detect the existence of an arc contour in the part contour for arc detection, which can be obtained by fitting the arc using the least squares method. The angle detection condition can be to detect whether the contour angle in the part contour meets the angle judgment condition. For example, for a tape part, the arc angle formed by its four contour edges needs to be vertical, or within a preset angle range judgment interval.

[0100] It should be noted that the curvature detection conditions and angle detection conditions are selected according to the part type. There may be a part type that only needs to be detected in curvature or angle, or there may be a part type that needs to be detected in curvature and angle.

[0101] For any part contour, if the template matching detection, edge detection, curvature detection and angle detection of the part contour all pass, the appearance defect detection of the part contour passes; if any of the template matching detection results, edge detection results, curvature detection results and angle detection results of the part contour fails, the appearance defect detection of the part contour fails.

[0102] It can be seen that in this embodiment, by performing contour detection on the part contour in multiple dimensions such as template matching detection, edge detection, curvature detection and angle detection, the comprehensiveness and accuracy of part appearance defect detection are improved, thereby improving the comprehensiveness and accuracy of part defect detection, and further improving the comprehensiveness and accuracy of component surface defects.

[0103] It is understandable that in order to improve the accuracy of position defect detection, at least one method can be used to perform position defect detection on the part contour.

[0104] In some embodiments, according to the part type corresponding to each part contour, position defect detection is performed on each part contour to obtain a position defect detection result, including:

[0105] According to the part types corresponding to the contours of each part, the part position detection models corresponding to the contours of each part and the relative positions of the standard parts are determined respectively;

[0106] Input each part contour into the corresponding part position detection model for processing, and output a first position detection result;

[0107] Determine the center point coordinates corresponding to each part contour, and determine the predicted part relative position of each part contour in the corresponding predicted part contour image according to the center point coordinates;

[0108] Determining a second position detection result according to the standard part relative position and the predicted part relative position of each part contour;

[0109] The position defect detection result is determined according to the first position detection result and the second position detection result respectively corresponding to each part contour.

[0110] The part position detection model can be pre-trained by relevant technical personnel. Specifically, a large number of part contour image samples corresponding to the corresponding part types can be obtained, and the position of the part contours in the part contour image samples can be marked; the part position detection model is trained using the part contour image samples to obtain a trained part position detection model.

[0111] It should be noted that, since the positions of part contours of different part types in the predicted part contour image are relatively fixed, the part contours of different part types correspond to their own relative positions of standard parts, that is, the exact position of a part contour in the part contour image to which it belongs.

[0112] Specifically, for any part contour, the part contour is input into the part position detection model corresponding to its part type, and the part position detection model predicts the position of the part contour to obtain a first position detection result. The center point coordinates of the part contour are determined, and according to the center point coordinates, the predicted part relative position of the part contour in the predicted part contour image to which it belongs can be determined. The position difference between the standard part relative position and the predicted part relative position of the part contour is determined, and it is determined whether the position difference is not greater than the preset difference threshold; if so, the second position detection result is determined to be a position detection pass; if so, the second position detection result is determined to be a position detection failure. If the first position detection result and the second position detection result corresponding to the part contour are both detected through, the position defect detection result of the part contour can be determined to be a detection pass; if the first position detection result or the second position detection result corresponding to the part contour is a detection failure, the position defect detection result of the part contour can be determined to be a detection failure.

[0113] It can be seen that in this embodiment, by adopting multiple methods to detect part position defects, the comprehensiveness and accuracy of part position defect detection are improved, thereby improving the comprehensiveness and accuracy of part defect detection, and further improving the comprehensiveness and accuracy of component surface defects.

[0114] The above technical scheme improves the comprehensiveness of part contour detection by performing position defect detection and appearance defect detection on part contour; it adopts a combination of deep learning and machine learning to perform part defect detection, specifically using deep learning technology to perform fine segmentation of part appearance, and based on deep learning, using machine vision technology to accurately locate and identify the part position; and, in the process of position defect detection and appearance defect detection, the difference in part type is taken into account, so that based on the difference characteristics of part type, different methods are used for defect detection in a targeted manner, thereby improving the accuracy of part defect detection.

[0115] It should be noted that during the iteration process of watch products, since the product is in the early stage of production, there may be a need to detect new defects, and the original scratch detection model cannot effectively detect the new defects of watch components. In addition, the demand for detection of new defects may appear frequently, and the detection of new defects needs to be quickly launched. Therefore, for such problems, if one model is used to detect all new defects, the accuracy of each new defect detection is low, and the model training time is very long, and it cannot meet the needs of rapid launch. If multiple models are used to detect different types of new defects, the model defect detection accuracy meets the needs, but the efficiency is low. Therefore, a model training method that can quickly complete model iteration and launch is needed.

[0116] In some embodiments, after generating target defect detection results including scratch defect detection results and part defect detection results, the method includes:

[0117] In response to a newly added scratch defect detection request, a newly added scratch defect sample image is generated; the newly added scratch defect sample image has a newly added type scratch sample label value;

[0118] Inputting the newly added scratch defect sample image into a preset newly added scratch detection model for processing, and outputting the newly added scratch prediction result of the newly added scratch defect sample image;

[0119] The newly added scratch detection model is trained according to the newly added scratch prediction results and the corresponding newly added type scratch sample label values ​​until the preset model training end conditions are met to obtain a trained newly added scratch detection model; the newly added scratch detection model is used to detect scratch defects of the newly added scratch type in the watch component image;

[0120] Among them, the newly added scratch detection model includes at least one residual module; the residual module includes four serially connected convolutional layers; the convolution kernel sizes of the two middle convolutional layers are set to the same.

[0121] Among them, the newly added scratch defect detection request can be a request from the defect detection demander to detect a newly added type of scratch defect. The newly added scratch defect detection request can carry the newly added scratch defect. According to the newly added scratch defect, a newly added scratch defect sample image is generated. Specifically, the sample image can be synthesized manually or automatically by fusing the background image with the newly added defect image.

[0122] The newly added type scratch sample label value of the newly added scratch defect sample image may include a newly added scratch defect position label and a newly added scratch category label.

[0123] Among them, the newly added scratch detection model can be pre-set by relevant technical personnel. For example, the newly added scratch detection model is ResNet (residual network). Taking ResNet as an example, in order to further improve the model training efficiency, during the model training process, a low-rank matrix fine-tuning method is adopted to increase the low-rank matrix parameters and only fine-tune the newly added parameters, so as to quickly complete the model iteration update during the training process, and at the same time, it can also avoid the overfitting problem caused by fewer new defects.

[0124] Specifically, the network structure of the residual module in the newly added scratch detection module is adjusted; the original residual network consists of three serially connected convolutional layers, the convolution kernel size of the convolution layer connected to the input and output is 1×1, and the convolution kernel size of the middle convolution layer is 3×3. In order to further improve the model training efficiency, the middle 3×3 convolution kernel is expanded to two 1×1 convolution kernels. The expression of the fine-tuning process is as follows:

[0125] y=(W 0 x+ΔWx)+x=(W 0 x+BAx)+x;

[0126] Among them, W 0 Represents the model pre-training weight; ΔW represents the fine-tuning parameter for the newly added defects; but since ΔW itself has too many parameters, it is decomposed into two low-rank matrices A and B to reduce the number of parameters.

[0127] Among them, the end condition of model training can be that the number of model iterations reaches a threshold, or the loss value tends to be stable, etc.

[0128] It can be seen that in this implementation, by making slight structural adjustments to the residual module in the newly added scratch detection model, the model's training efficiency for new defects is greatly improved, thereby facilitating the subsequent rapid launch of the new scratch detection model and improving the detection efficiency of new scratch defects.

[0129] It should be understood that, although the steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0130] Based on the same inventive concept, the embodiment of the present application also provides a watch component defect detection device. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more watch component defect detection device embodiments provided below can refer to the limitations of the watch component defect detection method above, and will not be repeated here.

[0131] like Figure 6 As shown, the embodiment of the present application provides a watch component defect detection device 600, comprising:

[0132] A positioning module 602 is used to use a trained component positioning model to perform component area positioning on the acquired component image of the watch to be detected, and obtain at least one component area image of the same or different component type;

[0133] The scratch detection module 604 is used to input each component area image into a trained scratch detection model that matches its component type for processing, and output a scratch defect detection result;

[0134] The contour prediction module 606 is used to input each component area image into the trained part segmentation model for processing and output a predicted part contour image;

[0135] A part detection module 608 is used to perform part defect detection on at least one part contour in the predicted part contour image to obtain a part defect detection result;

[0136] The generating module 610 is used to generate target defect detection results including scratch defect detection results and part defect detection results.

[0137] In some embodiments, in performing part defect detection on at least one part contour in the predicted part contour image to obtain a part defect detection result, the part detection module 608 is specifically used to:

[0138] Determine the part type corresponding to each part contour in the predicted part contour image;

[0139] According to the part type corresponding to each part contour, each part contour is subjected to appearance defect detection to obtain an appearance defect detection result;

[0140] According to the part type corresponding to each part contour, position defect detection is performed on each part contour to obtain a position defect detection result;

[0141] Generate part defect detection results including appearance defect detection results and position defect detection results.

[0142] In some embodiments, in terms of performing appearance defect detection on each part contour according to the part type corresponding to each part contour to obtain the appearance defect detection result, the part detection module 608 is specifically used to:

[0143] According to the part type corresponding to each part contour, the part contour template, edge detection condition, arc detection condition and angle detection condition corresponding to each part contour are determined respectively;

[0144] Perform similarity matching on each part contour and its corresponding part contour template respectively to obtain the template matching detection result;

[0145] According to the edge detection conditions corresponding to the contours of each part, edge detection is performed on the contours of each part to obtain edge detection results;

[0146] According to the curvature detection conditions corresponding to the contours of each part, the curvature detection is performed on the contours of each part to obtain the curvature detection results;

[0147] According to the angle detection conditions corresponding to the contours of each part, the angle detection is performed on the contours of each part to obtain the angle detection results;

[0148] The appearance defect detection results are determined based on the template matching detection results, edge detection results, curvature detection results and angle detection results corresponding to the contours of each part.

[0149] In some embodiments, in terms of performing position defect detection on each part contour according to the part type corresponding to each part contour to obtain the position defect detection result, the part detection module 608 is specifically used to:

[0150] According to the part types corresponding to the contours of each part, the part position detection models corresponding to the contours of each part and the relative positions of the standard parts are determined respectively;

[0151] Input each part contour into the corresponding part position detection model for processing, and output a first position detection result;

[0152] Determine the center point coordinates corresponding to each part contour, and determine the predicted part relative position of each part contour in the corresponding predicted part contour image according to the center point coordinates;

[0153] Determining a second position detection result according to the standard part relative position and the predicted part relative position of each part contour;

[0154] The position defect detection result is determined according to the first position detection result and the second position detection result respectively corresponding to each part contour.

[0155] In some embodiments, the watch component defect detection device 600 further includes:

[0156] An image acquisition module is used to acquire sample images of shallow scratched components with shallow scratched areas and sample images of deep scratched components with deep scratched areas under corresponding component types; the number of sample images of shallow scratched components and the number of sample images of deep scratched components meet a preset sample quantity balance condition; the sample images of shallow scratched components have shallow scratched sample label values; the sample images of deep scratched components have deep scratched sample label values;

[0157] A sample generation module, used for generating a scratch sample training set including a component sample image with shallow scratches and a component sample image with deep scratches;

[0158] A result prediction module, used for inputting the scratch sample training set into a preset scratch detection model for processing, and outputting a first prediction result for a sample image of a component with a shallow scratch, and a second prediction result for a sample image of a component with a deep scratch;

[0159] A first loss determination module, used to determine a first cross entropy loss value according to a first prediction result of a shallow scratch component sample image and a shallow scratch sample label value;

[0160] A second loss determination module, used for determining a second cross entropy loss value according to a second prediction result of the deep scratch component sample image and a deep scratch sample label value;

[0161] A confusion loss defect module, used to determine a confusion loss value according to the first prediction result and the second prediction result;

[0162] The model training module is used to train the scratch detection model according to the first cross entropy loss value, the second cross entropy loss value and the confusion loss value until the model training end condition is met, so as to obtain the trained scratch detection model under the corresponding component type.

[0163] In some embodiments, the watch component defect detection device 600 further includes:

[0164] A region extraction module is used to extract a region image of a shallow scratch region in a sample image of a shallow scratch component to obtain a shallow scratch region image; and to extract a region image of a deep scratch region in a sample image of a deep scratch component to obtain a deep scratch region image;

[0165] A background generation module, used to generate a plurality of background images by using at least one background image generation method;

[0166] An image fusion module is used to fuse the shallow scratch area image with each background image to obtain a fused shallow scratch component sample image; and to fuse the deep scratch area image with each background image to obtain a fused deep scratch component sample image;

[0167] Accordingly, in terms of generating a scratch sample training set including component sample images with shallow scratches and component sample images with deep scratches, the sample generation module is specifically used to:

[0168] A scratch sample training set is generated, which includes a shallow scratch component sample image, a deep scratch component sample image, a fused shallow scratch component sample image, and a fused deep scratch component sample image.

[0169] In some embodiments, the watch component defect detection device 600 further includes:

[0170] A sample adding module is used to generate a new scratch defect sample image in response to a new scratch defect detection request; the new scratch defect sample image has a new type scratch sample label value;

[0171] An image processing module is used to input the newly added scratch defect sample image into a preset newly added scratch detection model for processing, and output the newly added scratch prediction result of the newly added scratch defect sample image;

[0172] A new model training module is added, which is used to train the new scratch detection model according to the new scratch prediction results and the corresponding new type of scratch sample label values ​​until the preset model training end conditions are met to obtain the trained new scratch detection model; the new scratch detection model is used to detect scratch defects of the new scratch type in the watch component image;

[0173] Among them, the newly added scratch detection model includes at least one residual module; the residual module includes four serially connected convolutional layers; the convolution kernel sizes of the two middle convolutional layers are set to the same.

[0174] Each module in the above-mentioned watch component defect detection device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0175] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data related to the defect detection of watch components. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above-mentioned watch component defect detection method are implemented.

[0176] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, the steps in the above-mentioned watch component defect detection method are implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0177] Those skilled in the art will understand that Figure 7 or Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0178] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0179] In some embodiments, Fig. 9 The figure shows an internal structure diagram of a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0180] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0181] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0182] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0183] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0184] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting defects in watch components, characterized in that: include: Using the trained component positioning model to locate the component area of ​​the acquired watch component image to be inspected, and obtaining at least one component area image of the same or different component type; Inputting each of the component area images into a trained scratch detection model that matches its component type for processing, and outputting a scratch defect detection result; Inputting each of the component area images into a trained part segmentation model for processing, and outputting a predicted part contour image; Performing part defect detection on at least one part contour in the predicted part contour image to obtain a part defect detection result; A target defect detection result including the scratch defect detection result and the part defect detection result is generated.

2. The method according to claim 1, characterized in that The performing part defect detection on at least one part contour in the predicted part contour image to obtain a part defect detection result includes: Determine the part type corresponding to each part contour in the predicted part contour image; According to the part type corresponding to each of the part contours, performing appearance defect detection on each of the part contours to obtain an appearance defect detection result; According to the part type corresponding to each of the part contours, position defect detection is performed on each of the part contours to obtain a position defect detection result; Generate a part defect detection result including the appearance defect detection result and the position defect detection result.

3. The method according to claim 2, characterized in that The performing appearance defect detection on each of the part contours according to the part type corresponding to each of the part contours to obtain the appearance defect detection result includes: According to the part type corresponding to each of the part contours, respectively determine the part contour template, edge detection condition, arc detection condition and angle detection condition corresponding to each of the part contours; Performing similarity matching on each of the part contours and its corresponding part contour template to obtain a template matching detection result; According to the edge detection conditions corresponding to the contours of each part, edge detection is performed on the contours of each part to obtain edge detection results; According to the curvature detection conditions corresponding to the contours of each part, the curvature detection is performed on the contours of each part to obtain a curvature detection result; According to the angle detection conditions corresponding to the contours of each part, angle detection is performed on the contours of each part to obtain angle detection results; The appearance defect detection result is determined according to the template matching detection result, the edge detection result, the curvature detection result and the angle detection result respectively corresponding to each of the part contours.

4. The method according to claim 2, characterized in that: The step of performing position defect detection on each of the part contours according to the part type corresponding to each of the part contours to obtain a position defect detection result includes: According to the part types corresponding to the part contours, respectively determine the part position detection models and the relative positions of the standard parts corresponding to the part contours; Inputting each of the part contours into a corresponding part position detection model for processing, and outputting a first position detection result; Determine the center point coordinates corresponding to each of the part contours, and determine the predicted part relative position of each of the part contours in the corresponding predicted part contour image based on the center point coordinates; Determining a second position detection result according to the standard part relative position and the predicted part relative position of each of the part contours; The position defect detection result is determined according to the first position detection result and the second position detection result respectively corresponding to each of the part contours.

5. The method according to claim 1, characterized in that Before inputting the component area images to the trained scratch detection model matching the component type for processing and outputting the scratch defect detection result, the method further includes: Obtaining a shallow scratch component sample image with a shallow scratch area and a deep scratch component sample image with a deep scratch area under a corresponding component type; the number of the shallow scratch component sample images and the number of the deep scratch component sample images satisfy a preset sample quantity balance condition; the shallow scratch component sample image has a shallow scratch sample label value; the deep scratch component sample image has a deep scratch sample label value; generating a scratch sample training set including the shallow scratch component sample image and the deep scratch component sample image; Inputting the scratch sample training set into a preset scratch detection model for processing, and outputting a first prediction result for the shallow scratch component sample image and a second prediction result for the deep scratch component sample image; Determining a first cross entropy loss value according to a first prediction result of the shallow scratch component sample image and the shallow scratch sample label value; Determining a second cross entropy loss value according to a second prediction result of the deep scratch component sample image and the deep scratch sample label value; Determining a confusion loss value according to the first prediction result and the second prediction result; The scratch detection model is trained according to the first cross entropy loss value, the second cross entropy loss value, and the confusion loss value until a model training end condition is met, thereby obtaining the trained scratch detection model under the corresponding component type.

6. The method according to claim 5, characterized in that After obtaining the shallow scratch component sample image with shallow scratch area and the deep scratch component sample image with deep scratch area under the corresponding component type, the method further includes: Performing regional image extraction on the shallow scratch area in the shallow scratch component sample image to obtain a shallow scratch area image; performing regional image extraction on the deep scratch area in the deep scratch component sample image to obtain a deep scratch area image; Using at least one background image generation method to generate a plurality of background images; The shallow scratch area image is fused with each of the background images to obtain a fused shallow scratch component sample image; the deep scratch area image is fused with each of the background images to obtain a fused deep scratch component sample image; Accordingly, the generating of the scratch sample training set including the shallow scratch component sample image and the deep scratch component sample image comprises: A scratch sample training set including the shallow scratch component sample image, the deep scratch component sample image, the fused shallow scratch component sample image and the fused deep scratch component sample image is generated.

7. The method according to claim 1, characterized in that After generating the target defect detection result including the scratch defect detection result and the part defect detection result, the method further includes: In response to a newly added scratch defect detection request, a newly added scratch defect sample image is generated; the newly added scratch defect sample image has a newly added type scratch sample label value; Inputting the newly added scratch defect sample image into a preset newly added scratch detection model for processing, and outputting a newly added scratch prediction result for the newly added scratch defect sample image; The newly added scratch detection model is trained according to the newly added scratch prediction results and the corresponding newly added type scratch sample label values ​​until the preset model training end condition is met, thereby obtaining the trained newly added scratch detection model; the newly added scratch detection model is used to detect scratch defects of the newly added scratch type in the watch component image; Among them, the newly added scratch detection model includes at least one residual module; the residual module includes four serially connected convolutional layers; the convolution kernel sizes of the two middle convolutional layers are set to be the same.

8. A watch component defect detection device, characterized in that: include: A positioning module, used to use a trained component positioning model to locate the component area of ​​the acquired watch component image to be detected, and obtain at least one component area image of the same or different component type; A scratch detection module, used for inputting each component area image into a trained scratch detection model matching its component type for processing, and outputting a scratch defect detection result; A contour prediction module, used for inputting each of the component area images into a trained part segmentation model for processing, and outputting a predicted part contour image; A part detection module, used to perform part defect detection on at least one part contour in the predicted part contour image to obtain a part defect detection result; A generation module is used to generate a target defect detection result including the scratch defect detection result and the part defect detection result.

9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images