Training method and detection method of defect detection model, equipment and program product

Abstract

The invention provides a training method of a defect detection model, a detection method, equipment and a program product. The training method comprises the following steps: acquiring labeled sample image data; based on each piece of sample image data, obtaining a mixed loss function corresponding to each piece of sample image data through a jointly trained label distribution coding network and an original defect detection model; wherein the mixed loss function is in positive correlation with a first loss function of the label distribution coding network and a second loss function of the original defect detection model; and responding to the condition that the mixed loss function meets a training stop condition, and taking the current original defect detection model as a trained defect detection model. The label distribution coding network is adopted to assist in training the defect detection model, and global information in the image is effectively captured; the combination of domain knowledge and data features is realized through the mixed loss function, the complex mode in the image can be better understood and explained, and the accuracy and robustness of classification are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular, to a training method, a detection method, a device, and a program product for a defect detection model. Background Art

[0002] Computer vision technology obtains images of the product surface through a camera and uses image processing algorithms to analyze and process the images. Commonly used image processing techniques include edge detection, texture analysis, morphological processing, etc. These techniques can effectively identify and classify surface defects, improving the accuracy and efficiency of detection.

[0003] In recent years, with the rapid development of deep learning technology, surface defect detection methods based on deep learning have gradually attracted attention. Deep learning models, especially convolutional neural networks (CNNs), have shown excellent performance in image classification and object detection tasks. Through training with a large number of defect samples, deep learning models can automatically extract image features and achieve high-precision defect detection.

[0004] However, existing image classification and recognition technologies still face some challenges. First, most methods mainly focus on local features of images and ignore the global structure and relationships between regions in the images. For example, in steel surface defect detection, a small defect may cause a series of changes in the nearby regions. If only individual regions are focused on while ignoring the relationships between regions, it may lead to inaccurate detection results. Second, most methods rely on a large amount of labeled data for training, which is often difficult to obtain in practical applications. Generally speaking, there are mainly the following points:

[0005] (1) Ignoring the global structure and relationships between regions: Existing image classification and recognition technologies often focus too much on local features and ignore the global structure and relationships between regions. This method may lead to inaccurate classification results when dealing with complex image problems, such as object surface classification.

[0006] (2) Relying on a large amount of labeled data: Existing methods usually require a large amount of labeled data for training. However, in practical applications, obtaining a large amount of high-quality labeled data is an extremely difficult task. This limits the effectiveness of these methods in practical applications.

[0007] (3) Lack of effective knowledge representation and data fusion methods: In image classification and recognition, effective knowledge representation and data fusion are the keys to improving classification accuracy and robustness. However, existing methods often lack effective knowledge representation and data fusion methods. Summary of the Invention

[0008] The technical problem to be solved by the present disclosure is to overcome the above-mentioned defects in the prior art, and provide a training method, a detection method, a device and a program product for a defect detection model.

[0009] The present disclosure solves the above technical problem through the following technical solutions:

[0010] The present disclosure provides a training method for a defect detection model, and the training method includes:

[0011] Obtain labeled sample image data;

[0012] Based on each of the sample image data, obtain a mixed loss function corresponding to each of the sample image data through a jointly trained label distribution encoding network and an original defect detection model; wherein, the mixed loss function is positively correlated with a first loss function of the label distribution encoding network and a second loss function of the original defect detection model;

[0013] In response to the mixed loss function satisfying the training stop condition, use the current original defect detection model as the trained defect detection model.

[0014] Optionally, the obtaining of the labeled sample image data includes:

[0015] Obtain a sample set of images;

[0016] Label the sample set of images; wherein, the label content includes the position box of the defect on the object surface and the defect category.

[0017] Optionally, the obtaining of the mixed loss function corresponding to each of the sample image data through a jointly trained label distribution encoding network and an original defect detection model includes:

[0018] For each of the sample image data, generate a label distribution map, input the label distribution map into the to-be-trained label distribution encoding network to obtain a reconstructed label distribution map and the value of the corresponding first loss function, and input the sample image data into the to-be-trained original defect detection model to obtain the value of the corresponding second loss function;

[0019] Calculate the value of the second loss function corresponding to each of the sample image data based on the values of the first loss function and the second loss function corresponding to each of the sample image data.

[0020] Optionally, the obtaining of the mixed loss function corresponding to each of the sample image data through a jointly trained label distribution encoding network and an original defect detection model further includes:

[0021] In response to the value of the first loss function being greater than a preset threshold, the corresponding sample image data does not conform to the real scene.

[0022] Optionally, the generating the label distribution map includes:

[0023] Dividing the sample image data into rectangular color patches that are equally divided into a parts in length and b parts in width; where a and b are both positive integers;

[0024] For each of the rectangular color patches, if the rectangular color patch has no overlapping part with the position box of the defect or the proportion of the overlapping part in the rectangular color patch is less than a preset proportion threshold, the rectangular color patch is labeled as defect-free; if the proportion is greater than or equal to the proportion threshold, the rectangular color patch is labeled as the defect category corresponding to the position box.

[0025] Optionally, the following formula is used to calculate the hybrid loss function:

[0026] Hybrid loss function = hyperparameter * first loss function + second loss function.

[0027] Optionally, the training stop condition includes that the fluctuation of the hybrid loss function is within a preset range.

[0028] The present disclosure also provides a method for detecting defects on an object surface, the detection method including:

[0029] Obtaining an image to be detected;

[0030] Inputting the image to be detected into a defect detection model to obtain a defect detection result; wherein, the defect detection model is trained by using the training method of the foregoing defect detection model, and the defect detection result includes the position box where the defect is located and the defect category.

[0031] The present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and used for running on the processor. When the processor executes the computer program, the training method of the foregoing defect detection model and / or the detection method of the foregoing defects on the object surface are implemented.

[0032] The present disclosure also provides a computer program product, including a computer program. When the computer program is executed by a processor, the training method of the foregoing defect detection model and / or the detection method of the foregoing defects on the object surface are implemented.

[0033] On the basis of conforming to the common knowledge in the art, the above preferred conditions can be combined arbitrarily to obtain various preferred examples of the present disclosure.

[0034] The positive and progressive effects of the present disclosure are as follows: A label distribution encoding network is used to assist in training a defect detection model. The global structure and the relationships between regions in the image are characterized by the label distribution map, effectively capturing the global information in the image and making up for the deficiency of traditional methods that overly focus on local features. Training the label distribution encoding network based on the label distribution map can more accurately judge the nature and scope of influence of defects, introduce a global perspective, and improve the accuracy and reliability of the detection results. By means of a hybrid loss function that combines a first loss function and a second loss function, the integration of domain knowledge and data features is achieved, enabling a better understanding and interpretation of complex patterns in the image. This not only improves the accuracy of classification but also enhances the robustness of the defect detection model, enabling it to handle various complex image recognition tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flowchart of a method for training a defect detection model provided in Embodiment 1 of the present disclosure;

[0036] Figure 2 It is a flowchart of a specific implementation manner of step S11 of a method for training a defect detection model provided in Embodiment 1 of the present disclosure;

[0037] Figure 3 It is a flowchart of a specific implementation manner of step S12 of a method for training a defect detection model provided in Embodiment 1 of the present disclosure;

[0038] Figure 4 It is a flowchart of a specific implementation manner of step S121 of a method for training a defect detection model provided in Embodiment 1 of the present disclosure;

[0039] Figure 5 It is a flowchart of a method for detecting object surface defects provided in Embodiment 2 of the present disclosure;

[0040] Figure 6 It is a structural diagram of an electronic device provided in Embodiment 3 of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] The present disclosure will be further described below by way of embodiments, but the present disclosure is not limited to the scope of the described embodiments.

[0042] In the embodiments of the present disclosure, prefix words such as "first" and "second" are only used to distinguish different described objects, and have no restrictive effect on the position, order, priority, quantity, content, etc. of the described objects. In the embodiments of the present disclosure, the use of prefix words such as ordinal numbers to distinguish described objects does not constitute a restriction on the described objects. For the statements of the described objects, refer to the descriptions in the claims or the context of the embodiments. Unnecessary restrictions should not be imposed because of the use of such prefix words. In addition, in the description of this embodiment, unless otherwise specified, the meaning of "a plurality" is two or more.

[0043] Embodiment 1

[0044] Figure 1 The flowchart of a training method for a defect detection model provided by an exemplary embodiment of the present disclosure. The training method includes:

[0045] S11. Obtain labeled sample image data.

[0046] S12. Based on each sample image data, obtain the mixed loss function corresponding to each sample image data through the jointly trained label distribution encoding network and the original defect detection model. Among them, the mixed loss function is positively correlated with the first loss function of the label distribution encoding network and the second loss function of the original defect detection model.

[0047] S13. In response to the mixed loss function satisfying the training stop condition, use the current original defect detection model as the trained defect detection model.

[0048] Among them, surface defect detection mainly includes defect detection of various types such as scratches, pits, cracks, and stains.

[0049] The sample image data is a labeled real-scene picture, and the label includes the position box where the defect is located and the defect category.

[0050] Generally, the training method for defect detection is to select a detection model, such as yolov7, input the picture x to obtain the output prediction result y, calculate the loss function L according to the prediction result y and the true label y_true of this picture, and adjust the parameters of the detection model yolov7 by minimizing L, so as to complete the training of the model. The above is the defect detection modeling process based on data driving.

[0051] However, there will be a problem with this approach. Since the prediction result y classifies each pixel, it is very likely that adjacent pixels are assigned to different defect categories. However, in the real scene, according to domain knowledge, if adjacent pixels are both defective, then their defect types are generally the same. Therefore, the defect detection model obtained by the data-driven method may violate domain knowledge.

[0052] To solve this problem, this embodiment introduces a label distribution encoding network, whose function is to present domain knowledge in the form of a label distribution encoding network. The label distribution encoding network adopts an Encoder-Decoder structure. The input is a label distribution map, and the output is also a label distribution map. The purpose of the network is to perform reconstruction, and the Mean Squared Error (MSE) can be used as the first loss function L1. Specifically, if the defect classification of an image conforms to the domain knowledge, the value of the first loss function L1 obtained by inputting it into the label distribution encoding network will be relatively small; conversely, if the defect classification of an image does not conform to the domain knowledge (for example, adjacent pixels are assigned to different defect categories), the value of the first loss function L1 obtained by inputting it into the label distribution encoding network will be relatively large.

[0053] Each sample image data can be equally divided in terms of length and width to obtain several rectangular color blocks. Then, the rectangular color blocks are labeled based on their relative positions to the position boxes of the defects, resulting in a label distribution map. The label distribution map is more refined than the sample image data, characterizing the global structure and the relationships between regions in the image, effectively capturing the global information in the image, and making up for the deficiency of traditional methods that overly focus on local features.

[0054] Then, the label distribution encoding network is trained based on the label distribution map. The label distribution map can help identify the changes caused by a small defect in its surrounding area. By analyzing these changes, the nature and the influence range of the defect can be judged more accurately, introducing a global perspective and improving the accuracy and reliability of the detection results.

[0055] The original defect detection model is a commonly used defect detection model in the prior art, which can include SSD, Faster R-CNN, YOLO series (yolov1 - yolov7), etc., and will not be elaborated here.

[0056] The sample image data is input into the original defect detection model for training. Based on the second loss function (objective function) L of the original defect detection model, a constant multiple of L1 is added as the new objective function, that is, the hybrid loss function. The hybrid loss function is used to control the training processes of both the label distribution encoding network and the original defect detection model simultaneously. L is used for the training of the original defect detection model (such as yolov7), and L1 ensures that the output defect distribution conforms to the domain knowledge. When the model is trained, both L and L1 will be very small, thus ensuring that the output defect classification conforms to the domain knowledge.

[0057] The hybrid loss function combines the first loss function and the second loss function. Through the hybrid loss function, the combination of domain knowledge and data features is achieved, enabling a better understanding and interpretation of complex patterns in images. By constructing a hybrid objective function for knowledge-data fusion, it assists in training the defect detection model, effectively representing the global structure and relationships between regions in the image. This not only improves the accuracy of classification but also enhances the robustness of the defect detection model, enabling it to handle various complex image recognition tasks.

[0058] In this embodiment, a label distribution encoding network is used to assist in training the defect detection model. The global structure and relationships between regions in the image are characterized by the label distribution map, effectively capturing the global information in the image and making up for the deficiency of traditional methods that overly focus on local features. Training the label distribution encoding network based on the label distribution map can more accurately judge the nature and scope of influence of defects, introducing a global perspective and improving the accuracy and reliability of the detection results. Through the hybrid loss function that combines the first loss function and the second loss function, the combination of domain knowledge and data features is achieved, enabling a better understanding and interpretation of complex patterns in images. This not only improves the accuracy of classification but also enhances the robustness of the defect detection model, enabling it to handle various complex image recognition tasks.

[0059] In one embodiment, referring to Figure 2 , step S11 includes:

[0060] S111. Obtain the sample set images.

[0061] S112. Label the sample set images. Among them, the label content includes the position box of the defect on the object surface and the defect category.

[0062] In one embodiment, referring to Figure 3 , step S12 includes:

[0063] S121. For each sample image data, generate a label distribution map, input the label distribution map into the label distribution encoding network to be trained to obtain the reconstructed label distribution map and the corresponding value of the first loss function, and input the sample image data into the original defect detection model to be trained to obtain the corresponding value of the second loss function.

[0064] S122. Calculate the corresponding value of the second loss function based on the values of the first loss function and the second loss function corresponding to each sample image data.

[0065] In one embodiment, step S12 further includes:

[0066] In response to the value of the first loss function being greater than the preset threshold, the corresponding sample image data represents non-conformance to the real scenario.

[0067] Among them, the preset threshold can be set according to actual needs.

[0068] In one embodiment, referring to Figure 4 , "generating a label distribution map" in S121 includes:

[0069] S1211. Divide the sample image data to obtain rectangular color blocks that are equally divided into a parts in length and b parts in width. Among them, both a and b are positive integers.

[0070] S1212. For each rectangular color block, if there is no overlapping part between the rectangular color block and the position box of the defect or the proportion of the overlapping part in the rectangular color block is less than the preset proportion threshold, label the rectangular color block as defect-free; if the proportion is greater than or equal to the proportion threshold, label the rectangular color block as the defect category corresponding to the position box.

[0071] Among them, when dividing the sample image data, assuming the original picture is , divide the length and width of the picture successively equally divided and equally divided to obtain small rectangular color blocks, and the size of each small rectangular color block is , where M, N, m, and n are all positive integers.

[0072] Assume the proportion threshold is 50%. For each rectangular color block, if there is no overlapping part between it and the defect annotation box (i.e., the position box) or the proportion of the overlapping part does not exceed 50% of the rectangular color block, label the rectangular color block as defect-free; otherwise, if the proportion of the overlapping part with a certain defect annotation box is greater than 50% of the area of the rectangular color block, label it as the defect category of that defect annotation box. Assume there are types of defects, then for any picture, a graph is obtained, and the value of each pixel of this graph is , k}, where k is a positive integer, 0 represents no defect, and i represents the i-th type of defect. The obtained graph is called a label distribution map.

[0073] The proportion threshold can be set according to actual needs, such as 50%.

[0074] In one embodiment, the following formula is used to calculate the hybrid loss function:

[0075] Hybrid loss function = hyperparameter * first loss function + second loss function.

[0076] That is, hybrid loss function = L + λ * L1.

[0077] Among them, λ represents a hyperparameter, L1 represents the first loss function, and L represents the second loss function. The hyperparameter λ generally takes values in the range of [0.1, 0.5], and the hyperparameter can be set according to actual needs.

[0078] In one embodiment, the training stop condition includes that the fluctuation of the mixed loss function is within a preset range.

[0079] Among them, the preset range can be set according to actual needs.

[0080] Embodiment 2

[0081] Figure 5 The figure is a flowchart of a method for detecting object surface defects provided by an exemplary embodiment of the present disclosure. The detection method includes:

[0082] S21. Obtain an image to be detected;

[0083] S22. Input the image to be detected into the defect detection model to obtain a defect detection result. Among them, the defect detection model is trained by using the training method of the defect detection model in Embodiment 1, and the defect detection result includes the position box where the defect is located and the defect category.

[0084] In this embodiment, a label distribution encoding network is used to assist in training the defect detection model. The global structure and the relationship between regions in the image are characterized by the label distribution map, effectively capturing the global information in the image and making up for the deficiency of traditional methods that overly focus on local features; training the label distribution encoding network based on the label distribution map can more accurately judge the nature and influence range of the defect, introducing a global perspective and improving the accuracy and reliability of the detection result; through the mixed loss function that combines the first loss function and the second loss function, the combination of domain knowledge and data features is realized, and the complex patterns in the image can be better understood and explained, not only improving the classification accuracy, but also enhancing the robustness of the defect detection model, enabling it to handle various complex image recognition tasks.

[0085] Embodiment 3

[0086] Figure 6 The figure is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and configured to run on the processor. When the processor executes the computer program, the method for detecting object surface defects and / or the method for detecting object surface defects described in any of the above embodiments are implemented. Figure 6 The shown electronic device 90 is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0087] Such as Figure 6As shown, the electronic device 90 may be presented in the form of a general-purpose computing device. For example, it may be a server device. The components of the electronic device 90 may include, but are not limited to: at least one of the above-mentioned processors 91, at least one of the above-mentioned memories 92, and a bus 93 that connects different system components (including the memory 92 and the processor 91).

[0088] The bus 93 includes a data bus, an address bus, and a control bus.

[0089] The memory 92 may include volatile memory, such as random access memory (RAM) 921 and / or cache memory 922, and may further include read-only memory (ROM) 923.

[0090] The memory 92 may also include a program tool 925 (or utility) having a set (at least one) of program modules 924. Such program modules 924 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0091] The processor 91 executes various functional applications and data processing by running computer programs stored in the memory 92, such as the object surface defect detection method and / or the object surface defect detection method provided in any of the above embodiments.

[0092] The electronic device 90 may also communicate with one or more external devices 94 (such as a keyboard, a pointing device, etc.). Such communication may be carried out through an input / output (I / O) interface 95. Moreover, the electronic device 90 may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 96. As shown in the figure, the network adapter 96 communicates with other modules of the electronic device 90 through the bus 93. It should be understood that although not shown in the figure, other hardware and / or software modules may be used in combination with the electronic device 90, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0093] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more of the above-mentioned units / modules may be embodied in one unit / module. Conversely, the features and functions of one unit / module described above may be further divided and embodied by multiple units / modules.

[0094] Embodiment 4

[0095] An embodiment of the present disclosure also provides a computer program product, including a computer program, where when the computer program is executed by a processor, the detection method for object surface defects and / or the detection method for object surface defects described in any one of the above are implemented.

[0096] Among them, the program code for executing the computer program product of the present disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on the user device, partially on the user device, executed as an independent software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0097] Although the specific implementation manners of the present disclosure have been described above, those skilled in the art should understand that this is only an example, and the protection scope of the present disclosure is defined by the appended claims. Without departing from the principles and essence of the present disclosure, those skilled in the art can make various changes or modifications to these implementation manners, but these changes and modifications all fall within the protection scope of the present disclosure.

Claims

1. A method for training a defect detection model, characterized in that: The training method comprises: Obtain labeled sample image data; Based on each of the sample image data, a hybrid loss function corresponding to each of the sample image data is obtained by jointly training the label distribution coding network and the original defect detection model; wherein the hybrid loss function is positively correlated with the first loss function of the label distribution coding network and the second loss function of the original defect detection model; In response to the hybrid loss function satisfying a training stop condition, the current original defect detection model is used as a trained defect detection model.

2. The defect detection model training method according to claim 1, characterized in that: The step of obtaining labeled sample image data includes: Get sample set images; The sample set images are labeled, wherein the label content includes a location frame of the defect on the surface of the object and a defect category.

3. The defect detection model training method according to claim 1, characterized in that: The hybrid loss function corresponding to each of the sample image data is obtained by jointly training the label distribution encoding network and the original defect detection model based on each of the sample image data, including: For each of the sample image data, a label distribution map is generated, and the label distribution map is input into the label distribution encoding network to be trained to obtain a reconstructed label distribution map and a corresponding value of the first loss function, and the sample image data is input into the original defect detection model to be trained to obtain a corresponding value of the second loss function; The value of the second loss function corresponding to each of the sample image data is obtained based on the numerical calculation of the first loss function and the second loss function corresponding to each of the sample image data.

4. The defect detection model training method according to claim 3, characterized in that: The method of obtaining a hybrid loss function corresponding to each sample image data by jointly training a label distribution coding network and an original defect detection model based on each sample image data also includes: In response to the value of the first loss function being greater than a preset threshold, the corresponding sample image data representation does not conform to the real scene.

5. The defect detection model training method according to claim 3, characterized in that: The generating of the label distribution graph comprises: Performing image division on the sample image data to obtain rectangular color blocks divided into equal parts by length a and equal parts by width b; wherein a and b are both positive integers; For each of the rectangular color blocks, if there is no overlapping part between the rectangular color block and the defect location box or the proportion of the overlapping part in the rectangular color block is less than a preset proportion threshold, the rectangular color block is marked as non-defective; if the proportion is greater than or equal to the proportion threshold, the rectangular color block is marked as the defect category corresponding to the location box.

6. The defect detection model training method according to claim 1, characterized in that: The mixed loss function is calculated using the following formula: Mixed loss function = hyperparameters * first loss function + second loss function.

7. The defect detection model training method according to claim 1, characterized in that: The training stop condition includes that the fluctuation of the mixed loss function is within a preset range.

8. A method for detecting surface defects of an object, characterized in that: The detection method comprises: Acquire the image to be detected; The image to be detected is input into a defect detection model to obtain a defect detection result; wherein the defect detection model is trained using the defect detection model training method as described in any one of claims 1 to 7, and the defect detection result includes a location box of the defect and a defect category.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, characterized in that: When the processor executes the computer program, it implements the defect detection model training method described in any one of claims 1 to 7 and / or implements the object surface defect detection method described in claim 8.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the defect detection model training method described in any one of claims 1 to 7 and / or implements the object surface defect detection method described in claim 8.

Images