Method, apparatus and computer-readable storage medium for generating defect samples

Abstract

This invention discloses a method, apparatus, and computer-readable storage medium for generating defect samples, relating to the field of deep learning technology. The method includes the following steps: inputting a preset defect image and a preset target image into a trained convolutional neural network to obtain defect image samples; the convolutional neural network includes a wavelet filtering module, a downsampling layer, and an upsampling layer connected in sequence, wherein the wavelet filtering module is used to perform preset wavelet filtering processing on the input image; wherein the downsampling layer includes at least one downsampling module and at least one wavelet filtering module alternately arranged; the upsampling layer includes at least one upsampling module and at least one wavelet filtering module alternately arranged; the number of downsampling modules in the downsampling layer is equal to and corresponds to the number of upsampling modules in the upsampling layer. This invention provides effective defect samples for training defect detection models.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method, apparatus and computer-readable storage medium for generating defect samples. Background Technology

[0002] With the increasing application of deep learning in industrial products, defect detection is a crucial industrial application. In recent years, research on industrial defects has primarily focused on using various mainstream neural frameworks, making simple modifications to these frameworks for defect classification and detection. However, in practical applications, defect samples used for training are often scarce and difficult to obtain, resulting in a significant imbalance between positive and negative samples during training. This severely limits the performance of defect detection models, and can even render the models completely unusable.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide a method for generating defect samples, aiming to solve the technical problem of the scarcity of defect samples.

[0005] To achieve the above objectives, the present invention provides a defect sample generation method, which includes the following steps:

[0006] Input the preset defect image and the preset target image into the trained convolutional neural network to obtain defect image samples;

[0007] The convolutional neural network includes a wavelet filtering module, a downsampling layer, and an upsampling layer connected in sequence. The wavelet filtering module is used to perform preset wavelet filtering processing on the input image; wherein...

[0008] The downsampling layer includes at least one downsampling module and at least one wavelet filtering module arranged alternately. The downsampling module includes at least one downsampling unit and at least one downsampling convolutional layer.

[0009] The upsampling layer includes at least one upsampling module and at least one wavelet filtering module arranged alternately. The upsampling module includes at least one upsampling unit and at least one upsampling convolutional layer.

[0010] The downsampling modules in the downsampling layer are equal in number to the upsampling modules in the upsampling layer and correspond to each other.

[0011] Optionally, the step of inputting the preset defect image and the preset target image into a trained convolutional neural network to obtain defect image samples includes:

[0012] Obtain the convolutional neural network to be trained;

[0013] The preset defect training image and the preset target training image are input into the convolutional neural network to be trained to obtain the target defect image;

[0014] The convolutional neural network to be trained is trained based on the preset defect training image and the target defect image until the convolutional neural network to be trained converges, thereby obtaining a trained convolutional neural network model.

[0015] Optionally, the step of inputting the preset defect training image and the preset target training image into the convolutional neural network to be trained to obtain the target defect image includes:

[0016] Based on the wavelet filtering module, the preset defect training image and the preset target training image are subjected to preset wavelet filtering processing to output the first defect feature image.

[0017] After processing the first defect feature image based on the downsampling unit, downsampling convolutional layer and wavelet filtering module of the downsampling layer, the second defect feature image is output.

[0018] After processing the output second defect feature image based on the upsampling unit, upsampling convolutional layer and wavelet filtering module of the upsampling layer, the target defect image is output.

[0019] Optionally, the step of performing preset wavelet filtering on the preset defect training image and the preset target training image to output the first defect feature image includes:

[0020] Perform a preset number of wavelet transforms on the preset defect training images and the preset target training images to obtain low-frequency feature maps and high-frequency feature maps;

[0021] Obtain the discard probability of the high-frequency feature map;

[0022] The low-frequency feature map and the high-frequency feature map with a discard probability lower than a preset discard threshold are subjected to corresponding inverse wavelet transforms to obtain the first defect feature image.

[0023] Optionally, the step of obtaining the discard probability of the high-frequency feature map includes:

[0024] The pixel intensity of the high-frequency feature map is modeled based on the Gaussian distribution to obtain the pixel intensity variance of the high-frequency feature map;

[0025] The discard probability of the high-frequency feature map is obtained based on the pixel intensity variance and the transformation level of the high-frequency feature map, wherein the discard probability is negatively correlated with the pixel intensity variance and positively correlated with the transformation level.

[0026] Optionally, the step of training the convolutional neural network to be trained based on the preset defect training image and the target defect image until the convolutional neural network to be trained converges includes:

[0027] The preset defect training image and the target defect image are input into the convolutional neural network to be trained to obtain a reference defect image, wherein the reference defect image includes a first defect of the preset defect training image and a third defect generated based on the second defect of the target defect image;

[0028] Based on the first similarity information between the first defect and the second defect, and the second similarity information between the first defect and the third defect, the loss value of the preset loss function is obtained until the convolutional neural network to be trained converges.

[0029] Optionally, the preset loss function is:

[0030]

[0031] Where d1 is the first defect, d2 is the second defect, d3 is the third defect, NCC is pixel correlation, SSIM is structural similarity, ce is edge similarity, and mse is mean square error.

[0032] Optionally, the step of obtaining a loss value of a preset loss function based on the first similarity information between the first defect and the second defect, and the second similarity information between the first defect and the third defect, until the convolutional neural network to be trained converges includes:

[0033] The loss value of the preset loss function is calculated based on the pixel correlation, structural similarity, and edge similarity between the first defect and the second defect, as well as the pixel mean square error, structural similarity, and edge similarity between the first defect and the third defect.

[0034] Based on the loss value, update the network parameters of the convolutional neural network to be trained;

[0035] If the loss value is greater than a preset loss threshold, the convolutional neural network to be trained is determined to have converged.

[0036] In addition, to achieve the above objectives, the present invention also provides a defect sample generation apparatus, the defect sample generation apparatus comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method described in any of the preceding claims.

[0037] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0038] This invention proposes a defect sample generation method. By inputting a preset defect image and a preset target image into a trained convolutional neural network (CNN), corresponding defect image samples are generated, thus solving the problem of defect sample scarcity. Furthermore, by inserting a wavelet filtering module into the CNN, which performs preset wavelet filtering on the input image, the CNN can better extract the structural features of the input image at each level. This ensures that the defects in the synthesized defect image samples based on the preset defect image and the preset target image retain the structural features of the defects in the preset defect image to the greatest extent, thereby improving the realism of the defects in the defect image samples. This invention ensures the realism of defects in the generated defect samples while simultaneously generating them, thus providing effective defect samples for training defect detection models. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention;

[0040] Figure 2 This is a schematic diagram of a scenario according to an embodiment of the defect sample generation method of the present invention;

[0041] Figure 3 This is a flowchart illustrating the second embodiment of the defect sample generation method of the present invention;

[0042] Figure 4 This is a schematic diagram illustrating a scenario where the convolutional neural network to be trained is being trained according to the present invention.

[0043] Figure 5 This is a schematic diagram of the preset wavelet filtering process in the wavelet filtering module of the present invention;

[0044] Figure 6 This is a flowchart illustrating the third embodiment of the defect sample generation method of the present invention.

[0045] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0046] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0047] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0048] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0049] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0050] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0051] Deep learning is increasingly widely used in industrial products, with defect detection being a crucial application. Due to the diverse nature of defects, traditional machine vision algorithms struggle to fully model and transfer defect features, resulting in limited reusability and wasting significant human resources by requiring differentiation based on working conditions. Recent research on industrial defects has primarily focused on using various mainstream neural frameworks, from CNNs (Convolutional Neural Networks) to SSDs (Single Shot MultiBox Detectors), and even semantic segmentation-based FCNs (Fully Convolutional Networks), to make simple improvements to these frameworks for defect classification and detection. However, in practical applications, defect samples used for training are often scarce and difficult to obtain, leading to a severe imbalance between positive and negative samples during training. This severely limits model performance and can even render the model unusable. Furthermore, supervised learning methods often fail to meet normal production demands in scenarios with highly variable defect appearances. Of course, some research works generate a large amount of training data by manually annotating it to meet the needs of agents. This requires a lot of human resources and may also have the problem that the artificial defects are not realistic enough, which affects the actual application effect of the defect detection model.

[0052] In actual production, the increasing advancement of industrial technology has led to fewer and fewer product defects, resulting in insufficient defect samples to train defect detection models. Existing generative adversarial network-based methods also suffer from problems such as training instability, difficulty in model convergence, and difficulty in adapting to changes in the target image. Therefore, defect samples remain consistently scarce.

[0053] like Figure 1 As shown, Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application.

[0054] like Figure 1As shown, the defect sample generation device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0055] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the defect sample generation device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0056] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and computer programs.

[0057] exist Figure 1 In the defect sample generation device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the defect sample generation device of the present invention can be set in the defect sample generation device, and the defect sample generation device calls the computer program stored in the memory 1005 through the processor 1001 and executes the defect sample generation method provided in the embodiment of this application.

[0058] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0059] Reference Figure 2 , Figure 2 This is a schematic diagram of a scenario in the first embodiment of the defect sample generation method of the present invention.

[0060] The first embodiment of the present invention provides a defect sample generation method, the defect sample generation method comprising the following steps:

[0061] Step S100: Input the preset defect image and the preset target image into the trained convolutional neural network to obtain defect image samples;

[0062] The convolutional neural network includes a wavelet filtering module, a downsampling layer, and an upsampling layer connected in sequence. The wavelet filtering module is used to perform preset wavelet filtering processing on the input image; wherein...

[0063] The downsampling layer includes at least one downsampling module and at least one wavelet filtering module arranged alternately. The downsampling module includes at least one downsampling unit and at least one downsampling convolutional layer.

[0064] The upsampling layer includes at least one upsampling module and at least one wavelet filtering module arranged alternately. The upsampling module includes at least one upsampling unit and at least one upsampling convolutional layer.

[0065] The downsampling modules in the downsampling layer are equal in number to the upsampling modules in the upsampling layer, and they correspond to each other. For example... Figure 2 As shown in the figure, the gray bars represent the input and output images. The downsampling layer in the figure includes two alternating downsampling modules and two wavelet filtering modules. It can be understood that the downsampling layer may include more or fewer downsampling modules and wavelet filtering modules. Similarly, the upsampling layer in the figure includes two alternating upsampling modules and two wavelet filtering modules. It can be understood that the upsampling layer may include more or fewer upsampling modules and wavelet filtering modules.

[0066] Specifically, the preset defect image is an existing image containing defects, and the defects in the preset defect image have been labeled. The preset target image is a target image from which the defects in the preset defect image are to be transferred. By inputting the preset defect image and the preset target image into a trained convolutional neural network, the defects in the preset defect image are synthesized into the preset target image, thereby obtaining a defect image sample.

[0067] A convolutional neural network (CNN) is a deep feedforward artificial neural network that includes multiple convolutional layers, upsampling layers, and downsampling layers. Each convolutional layer can process an image, while upsampling and downsampling layers can change the image size (scale). This allows different convolutional layers to process the image at different scales, enabling the addition or extraction of features at different scales. Through appropriate training, the network parameters, such as the convolutional kernel, bias, and weights, in each convolutional layer of the CNN can be adjusted.

[0068] The convolutional neural network includes a wavelet filtering module, a downsampling layer, and an upsampling layer connected in sequence. The wavelet filtering module is used to perform preset wavelet filtering processing on the input image. The downsampling layer includes alternating downsampling modules and wavelet filtering modules. Each downsampling module includes at least one downsampling unit (such as a pooling layer) and at least one downsampling convolutional layer. The downsampling units and downsampling convolutional layers are also alternately arranged; that is, there cannot be consecutive downsampling units or downsampling convolutional layers, but this does not mean that the downsampling units or downsampling convolutional layers must have a one-to-one relationship.

[0069] Different downsampling convolutional layers (which implicitly include activation and normalization layers, not described in detail here) have different scales, used to process images at specific scales through convolutional kernels. This is because different image features (such as texture, edges, and objects) behave differently at different scales, thus requiring separate processing at different scales for different features. Downsampling units (such as pooling layers) reduce the image size, making the output image scale smaller and meeting the requirements of the next scale of convolutional layers. Pooling is a specific form of downsampling, and specific pooling layers that can be used include max-pooling, average-pooling, decimation, demuxout, and others.

[0070] Similarly, the upsampling layer includes at least one upsampling module and at least one wavelet filtering module arranged alternately. Each upsampling module includes at least one upsampling unit and at least one upsampling convolutional layer. The upsampling units and convolutional layers are also arranged alternately; that is, there cannot be consecutive upsampling units or convolutional layers. However, this does not mean that the upsampling units or convolutional layers must have a one-to-one relationship. Upsampling units (such as MUXER layers) are used to magnify the image to a scale that meets the requirements of the next convolutional layer.

[0071] Furthermore, the downsampling units and the upsampling units are equal in number and correspond to each other to ensure that the image output by the convolutional neural network has the same size as the input image.

[0072] In the first embodiment of the present invention, a corresponding defect image sample is generated by inputting a preset defect image and a preset target image into a trained convolutional neural network, thereby realizing the generation of defect samples and solving the problem of the scarcity of defect samples. Furthermore, by inserting a wavelet filtering module into the convolutional neural network, which performs preset wavelet filtering processing on the input image, the convolutional neural network can better extract the structural features of the input image at each level. This ensures that the defects in the defect image sample synthesized based on the preset defect image and the preset target image retain the structural features of the defects in the preset defect image to the greatest extent, thereby improving the realism of the defects in the defect image sample.

[0073] Furthermore, referring to Figure 3 The second embodiment of the present invention provides a defect sample generation method, which, based on the first embodiment described above, includes the following steps before step S100:

[0074] Step S200: Obtain the convolutional neural network to be trained;

[0075] Step S210: Input the preset defect training image and the preset target training image into the convolutional neural network to be trained to obtain the target defect image;

[0076] Step S220: Train the convolutional neural network to be trained according to the preset defect training image and the target defect image until the convolutional neural network to be trained converges, and obtain the trained convolutional neural network model.

[0077] Specifically, refer to Figure 4 , Figure 4This is a schematic diagram illustrating the training scenario of the convolutional neural network to be trained according to the present invention. The convolutional neural network to be trained is obtained. The structure of the convolutional neural network to be trained is the same as that of the already trained convolutional neural network, that is, the convolutional neural network to be trained also includes a wavelet filtering module, a downsampling layer, and an upsampling layer connected in sequence. The preset defect training image is an existing image containing a first defect, and the first defect in the preset defect training image has been labeled. The preset target image is the target image from which the first defect in the preset defect image is to be transferred. By inputting the preset defect training image (i.e., image A) and the preset target training image (i.e., image B) into the convolutional neural network to be trained, the image undergoes preset wavelet filtering processing by the wavelet filtering module in the convolutional neural network to obtain an initial synthesized image. Then, the initial synthesized image is alternately subjected to downsampling processing, convolution calculation, and preset wavelet filtering processing by the wavelet filtering module in the downsampling layer; then, it is alternately subjected to upsampling processing, convolution calculation, and preset wavelet filtering processing by the wavelet filtering module in the upsampling layer, finally obtaining the target defect image (i.e., image B'). Then, the preset defect training image and the target defect image are input into the convolutional neural network to be trained to obtain a reference defect image (i.e., image A'). It is understood that the preset defect training image includes a first defect (i.e., d1), the target defect image includes a second defect generated based on the first defect (i.e., d2), and the reference defect image includes the first defect and a third defect generated based on the second defect (i.e., d3). A corresponding preset loss function can be constructed based on defect edge information, pixel distribution information, and loss function terms such as reverse direction consistency. Then, the corresponding loss value is calculated using the preset loss function to determine the similarity between the first defect and the second defect, and the similarity between the first defect and the third defect, thereby judging the performance of the convolutional neural network to be trained. When the loss value of the preset loss function is greater than the preset function value, it indicates that the first defect and the second defect have a high similarity, and the first defect and the third defect also have a high similarity, indicating that the performance of the convolutional neural network to be trained is excellent. It can be determined that the convolutional neural network to be trained has converged, thus obtaining the trained convolutional neural network.

[0078] Furthermore, in another embodiment, step S210 further includes the following steps:

[0079] Step S211: Based on the wavelet filtering module, perform preset wavelet filtering on the preset defect training image and the preset target training image to output the first defect feature image.

[0080] Step S212: After processing the output first defect feature image based on the downsampling unit, downsampling convolutional layer and wavelet filtering module of the downsampling layer, the second defect feature image is output.

[0081] Step S213: After processing the output second defect feature image based on the upsampling unit, upsampling convolutional layer and wavelet filtering module of the upsampling layer, the target defect image is output.

[0082] Specifically, based on the wavelet filtering module, a preset wavelet filtering process is performed on the preset defect training image and the preset target training image to output a first defect feature image. In this embodiment, the preset wavelet filtering process is used to obtain the corresponding structural texture information of the preset defect training image and the preset target training image. Since the structural texture information of the image is relatively stable and is not affected by factors such as illumination and noise, the accuracy of the defect in the output first defect feature image can be effectively guaranteed. Then, the first defect feature image is downsampled sequentially by the downsampling unit in the downsampling layer, convolution is performed by the downsampling convolution layer, and preset wavelet filtering is performed by the wavelet filtering module. When there are two or more downsampling modules in the downsampling layer, the above steps of downsampling sequentially by the downsampling unit, convolution is performed by the downsampling convolution layer, and preset wavelet filtering is performed by the wavelet filtering module are repeated until the first defect feature image has been processed by all the downsampling modules and wavelet filtering modules in the downsampling layer, and then the second defect feature image is output.

[0083] Similarly, the second defect feature image is upsampled sequentially by upsampling units in the upsampling layer, convolved by upsampling convolutional layers, and then filtered by a pre-defined wavelet filter module. When there are two or more upsampling modules in the upsampling layer, the above steps of sequential upsampling by upsampling units, convolution by upsampling convolutional layers, and pre-defined wavelet filter processing are repeated until the second defect feature image has undergone processing by all upsampling and wavelet filter modules in the upsampling layer, at which point the target defect image is output. In this embodiment, by inserting a wavelet filter module into the convolutional neural network, the wavelet filter module is used to perform pre-defined wavelet filter processing on the input image. This allows the convolutional neural network to better extract the structural features of the input image at each level, resulting in a more complete and accurate extraction of the structural features of defects in the input image, thereby improving the realism of defects in the output image.

[0084] Furthermore, in another embodiment, step S211 includes the following steps:

[0085] Step A: Perform wavelet transform on the preset defect training image and the preset target training image a preset number of times to obtain low-frequency feature maps and high-frequency feature maps;

[0086] Step B: Obtain the discard probability of the high-frequency feature map;

[0087] Step C: Perform corresponding inverse wavelet transforms on the low-frequency feature map and the high-frequency feature map whose discard probability is lower than the preset discard threshold to obtain the first defect feature image.

[0088] Specifically, refer to Figure 5 , Figure 5This is a schematic diagram of the preset wavelet filtering process in the wavelet filtering module of the present invention. The preset wavelet filtering process of the wavelet filtering module involves performing a preset number of wavelet transforms on the input image to obtain corresponding low-frequency feature maps and high-frequency feature maps; then obtaining the discard probability of the high-frequency feature maps; and finally, performing corresponding inverse wavelet transforms on the low-frequency feature maps and the high-frequency feature maps whose discard probabilities are lower than a preset discard threshold to obtain corresponding reconstructed feature images, which are then used as output images. The corresponding low-frequency and high-frequency feature maps are obtained by performing a preset number of wavelet transforms on the preset defect training image and the preset target training image, respectively. It can be understood that performing one wavelet transform on an image yields one low-frequency feature map and three high-frequency feature maps, performing two wavelet transforms yields one low-frequency feature map and six high-frequency feature maps, and performing n wavelet transforms yields one low-frequency feature map and 3n high-frequency feature maps, where n is a positive integer greater than zero. The preset number of transforms can be set according to specific user needs, such as 1, 2, 3, or 4 transforms. The discard probability of a high-frequency feature map can be determined based on the pixel intensity distribution information and its transform level. The transform level refers to the number of wavelet transforms used to obtain the high-frequency feature map; for example, if the high-frequency feature map is obtained through the third wavelet transform, the corresponding transform level is 3. When the pixel intensity distribution of the high-frequency feature map is too concentrated, it indicates that the high-frequency feature map contains a lot of noise, thus increasing the discard probability. Conversely, when the transform level of the high-frequency feature map is high, it indicates that the high-frequency feature map contains fewer features, thus increasing the discard probability. Then, the low-frequency feature map corresponding to the preset defect training image and the high-frequency feature map with a discard probability lower than a preset discard threshold, as well as the low-frequency feature map corresponding to the preset target training image and the high-frequency feature map with a discard probability lower than the preset discard threshold, are reconstructed using the corresponding inverse wavelet transform to obtain the first defect feature image. The preset discard threshold is a threshold preset by the user to determine whether to discard the high-frequency feature map, and can be selected according to specific needs, such as 0.4, 0.5, 0.6, etc. In this embodiment, the preset defect training image and the preset target training image are fused by discarding wavelet transform and inverse wavelet transform to obtain the first defect feature image, thus realizing the initial fusion of the preset defect training image and the preset target training image. In the fusion process, some high-frequency feature images with more noise and / or fewer features are discarded by discarding probability, thereby improving the completeness and accuracy of the structural features of the defect on the final obtained first defect feature image.

[0089] Furthermore, in another embodiment, step B further includes the following steps:

[0090] Step B1: Model the pixel intensity of the high-frequency feature map based on the Gaussian distribution to obtain the pixel intensity variance of the high-frequency feature map;

[0091] Step B2: Based on the pixel intensity variance and the transformation level of the high-frequency feature map, obtain the discard probability of the high-frequency feature map, wherein the discard probability is negatively correlated with the pixel intensity variance and positively correlated with the transformation level.

[0092] Specifically, the pixel intensity of the high-frequency feature map can be modeled based on a Gaussian distribution to obtain a corresponding Gaussian distribution mathematical model. Then, the variance (i.e., the pixel intensity variance of the high-frequency feature map) is calculated based on this Gaussian distribution mathematical model. According to the pixel intensity variance and the transformation level of the high-frequency feature map, the discard probability of the high-frequency feature map is obtained through a preset discard probability function. It is understood that since a smaller pixel intensity indicates a more concentrated pixel intensity in the high-frequency feature map, meaning the high-frequency feature map contains more noise, the discard probability is negatively correlated with the pixel intensity variance. Conversely, a higher transformation level indicates fewer features in the high-frequency feature map, therefore the discard probability is positively correlated with the transformation level.

[0093] As an optional embodiment, the preset discard probability function is:

[0094] , a,k>0;

[0095] Where σ is the pixel intensity variance of the high-frequency feature map, L is the transformation level of the high-frequency feature map, and a and k are parameters greater than zero.

[0096] Furthermore, referring to Figure 6 , Figure 6 This is a flowchart illustrating a third embodiment of the defect sample generation method of the present invention. The third embodiment of the present invention provides a defect sample generation method based on the above... Figure 3 In the embodiment shown, step S220 includes the following steps:

[0097] Step S231: Input the preset defect training image and the target defect image into the convolutional neural network to be trained to obtain a reference defect image, wherein the reference defect image includes a first defect of the preset defect training image and a third defect generated based on the second defect of the target defect image;

[0098] Step S232: Based on the first similarity information between the first defect and the second defect, and the second similarity information between the first defect and the third defect, obtain the loss value of the preset loss function until the convolutional neural network to be trained converges.

[0099] Specifically, after obtaining the target defect image, the preset defect training image and the target defect image are input into the convolutional neural network to be trained to obtain a reference defect image. It is understood that the preset defect training image includes a first defect, the target defect image includes a second defect generated based on the first defect, and the reference defect image includes the first defect and a third defect generated based on the second defect. Then, based on the first similarity information between the first defect and the second defect, and the second similarity information between the first defect and the third defect, the loss value of a preset loss function is obtained, thereby determining the similarity between the first defect and the second defect generated by the convolutional neural network to be trained, and the similarity between the first defect and the second defect generated twice by the convolutional neural network to be trained. The first similarity information may include pixel correlation, structural similarity, edge similarity, etc., between the first defect and the second defect, and the first similarity information may include pixel mean square error, structural similarity, edge similarity, etc., between the first defect and the third defect. When the loss value meets the preset requirements, it indicates that the first defect is highly similar to the second defect, and the first defect is also highly similar to the third defect. This ensures the authenticity of the defects in the image output by the convolutional neural network to be trained, and thus it can be determined that the convolutional neural network to be trained has completed training and converged.

[0100] Furthermore, in another embodiment, the preset loss function is:

[0101]

[0102] Where d1 is the first defect, d2 is the second defect, d3 is the third defect, NCC is pixel correlation, SSIM is structural similarity, ce is edge similarity, and mse is mean square error.

[0103] Specifically, NCC (Normalized Cross Correlation) refers to pixel correlation.

[0104]

[0105] Where p represents the pixel coordinates of the image.

[0106] SSIM (Structural Similarity) is a measure of structural similarity.

[0107]

[0108] Where x and y are the input images, μ x It is the average value of x, μ y It is the average value of y, σ2 x The variance σ of x 2 y Let σ be the variance of y. xy Let c be the covariance of x and y, and c1 and c2 be constants used to avoid systematic errors caused by a denominator of 0.

[0109] mse (Mean Squared Error) is the mean squared error.

[0110] mse(x,y)=

[0111] Where x and y are the input images, and m is the number of samples taken from x and y.

[0112] ce represents edge similarity, where, , An offline-trained edge extraction model can be used to extract the edge information of the first, second, and third defects, where the edge information is binary information. The edge similarity between the first and second defects is obtained by comparing their edge information, and the edge similarity between the first and third defects is obtained by comparing their edge information. It is understood that other methods can also be used to extract edge information, such as OpenCV and Halcon, and this embodiment does not limit this approach.

[0113] In this embodiment, a preset loss function is constructed based on the pixel correlation, structural similarity, and edge similarity between the first and second defects, as well as the pixel mean square error, structural similarity, and edge similarity between the first and third defects. This function can be used to determine the similarity between the first defect and the second defect generated by the convolutional neural network to be trained, and the similarity between the first defect and the third defect generated twice by the convolutional neural network to be trained. This determines the loss of defect features in the image during the image generation process of the convolutional neural network to be trained, so as to adjust the network parameters of the convolutional neural network to be trained.

[0114] Furthermore, step S232 includes the following steps:

[0115] Step D: Calculate the loss value of the preset loss function based on the pixel correlation, structural similarity, and edge similarity between the first and second defects, and the pixel mean square error, structural similarity, and edge similarity between the first and third defects.

[0116] Step E: Update the network parameters of the convolutional neural network to be trained based on the loss value;

[0117] Step F: When the loss value is greater than a preset loss threshold, it is determined that the convolutional neural network to be trained has converged.

[0118] Specifically, after obtaining the pixel correlation, structural similarity, and edge similarity between the first and second defects, and the pixel mean square error, structural similarity, and edge similarity between the first and third defects, the corresponding loss values ​​can be calculated based on a preset loss function. Then, the network parameters of the convolutional neural network to be trained are updated according to the loss values. When the loss value is greater than a preset loss threshold, since the larger the loss value of the preset loss function, the higher the similarity between the first, second, and third defects, it can be determined that the convolutional neural network to be trained has converged, and the trained convolutional neural network is obtained.

[0119] Furthermore, this embodiment of the invention also proposes a computer storage medium storing a computer program. When the computer program is executed by a processor, it implements the operations in the defect sample generation method provided in the above embodiments. The specific steps will not be described in detail here.

[0120] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity / operation / object from another, and do not necessarily require or imply any such actual relationship or order between these entities / operations / objects; the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0121] For the apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and relevant details can be found in the description of the method embodiments. The apparatus embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present invention. Those skilled in the art can understand and implement this without any creative effort.

[0122] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0123] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, vehicle, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0124] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for generating defect samples, characterized in that, The defect sample generation method comprises the following steps: The preset defect image and the preset target image are input into the trained convolutional neural network to obtain a defect image sample; The convolutional neural network comprises a wavelet filtering module, a down-sampling layer and an up-sampling layer connected in sequence, wherein the wavelet filtering module is configured to perform preset wavelet filtering processing on an input image; wherein The down-sampling layer comprises at least one down-sampling module and at least one wavelet filtering module arranged alternately, and the down-sampling module comprises at least one down-sampling unit and at least one down-sampling convolutional layer; The up-sampling layer comprises at least one up-sampling module and at least one wavelet filtering module arranged alternately, and the up-sampling module comprises at least one up-sampling unit and at least one up-sampling convolutional layer; The number of down-sampling modules in the down-sampling layer is equal to the number of up-sampling modules in the up-sampling layer and corresponds to each other; The step of inputting the preset defect image and the preset target image into the trained convolutional neural network to obtain a defect image sample comprises: Obtaining a convolutional neural network to be trained; Inputting the preset defect training image and the preset target training image into the convolutional neural network to be trained to obtain a target defect image; Training the convolutional neural network to be trained according to the preset defect training image and the target defect image until the convolutional neural network to be trained converges to obtain a trained convolutional neural network model; The step of inputting the preset defect training image and the preset target training image into the convolutional neural network to be trained to obtain a target defect image comprises: Based on the wavelet filtering module, performing preset wavelet filtering processing on the preset defect training image and the preset target training image to output a first defect feature image; Based on the down-sampling unit, the down-sampling convolutional layer and the wavelet filtering module of the down-sampling layer, performing corresponding processing on the output first defect feature image to output a second defect feature image; Based on the up-sampling unit, the up-sampling convolutional layer and the wavelet filtering module of the up-sampling layer, performing corresponding processing on the output second defect feature image to output a target defect image.

2. The defect sample generation method of claim 1, wherein, The step of performing preset wavelet filtering processing on the preset defect training image and the preset target training image to output a first defect feature image comprises: Performing wavelet transform on the preset defect training image and the preset target training image for a preset number of times to obtain a low-frequency feature map and a high-frequency feature map; Obtaining a dropout probability of the high-frequency feature map; Performing corresponding inverse wavelet transform on the low-frequency feature map and the high-frequency feature map with a dropout probability lower than a preset dropout threshold to obtain the first defect feature image.

3. The defective sample generation method of claim 2, wherein, The step of obtaining the dropout probability of the high-frequency feature map comprises: Based on Gaussian distribution, modeling pixel intensity of the high-frequency feature map to obtain a pixel intensity variance of the high-frequency feature map; According to the pixel intensity variance and a transform level at which the high-frequency feature map is located, obtaining a dropout probability of the high-frequency feature map, wherein the dropout probability is negatively correlated with the pixel intensity variance and positively correlated with the transform level.

4. The defect sample generation method of claim 1, wherein, The step of training the to-be-trained convolutional neural network according to the preset defect training image and the target defect image until the to-be-trained convolutional neural network converges comprises: inputting the preset defect training image and the target defect image into the to-be-trained convolutional neural network to obtain a reference defect image, wherein the reference defect image comprises a first defect of the preset defect training image and a third defect generated based on a second defect of the target defect image; obtaining a loss value of a preset loss function according to first similarity information between the first defect and the second defect and second similarity information between the first defect and the third defect until the to-be-trained convolutional neural network converges.

5. The defective sample generation method of claim 4, wherein, The preset loss function is wherein d1 is a first defect, d2 is a second defect, d3 is a third defect, edge information of the first defect, edge information of the second defect, edge information of the third defect, NCC is a pixel correlation, SSIM is a structural similarity, ce is an edge similarity, and mse is a mean square error.

6. The defective sample generation method of claim 5, wherein, The step of obtaining a loss value of a preset loss function according to first similarity information between the first defect and the second defect and second similarity information between the first defect and the third defect until the to-be-trained convolutional neural network converges comprises: calculating the loss value of the preset loss function according to pixel correlation, structural similarity, and edge similarity between the first defect and the second defect, and pixel mean square error, structural similarity, and edge similarity between the first defect and the third defect; updating network parameters of the to-be-trained convolutional neural network according to the loss value; when the loss value is greater than a preset loss threshold, determining that the to-be-trained convolutional neural network converges.

7. A defect sample generation apparatus characterized by comprising: The defect sample generation device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program implements the steps of the method according to any one of claims 1 to 6 when executed by the processor.

8. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium and implements the steps of the defect sample generation method according to any one of claims 1 to 6 when executed by the processor.

Images