Defect detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a defect detection method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a first image; performing defect detection on the first image through a first model to obtain a first result; a first loss function is configured in the first model; the first loss function can instruct the first model to distinguish foreground feature information and background feature information of the first image. According to the method, the defect detection is performed on the first image through the first model, and the first loss function is configured in the first model, so that the foreground feature and the background feature of the first image can be aligned, and the feature consistency can be ensured, and therefore, the use of the first model can realize the automatic detection of the metal oxidation surface, and the detection efficiency is improved. And the defect identification precision of complex textures and edge areas on the metal oxidation surface can also be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a defect detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] Aluminum alloys possess excellent properties such as high specific strength and high specific stiffness, and are widely used in many fields such as 3C consumer electronics, aerospace, and rail transportation. Currently, after anodizing, the mid-frames or shells of aluminum-magnesium alloy consumer electronics require rigorous surface appearance inspection, including gloss / matte zoning, color difference detection, and scratch detection. Existing inspection methods mainly include manual sampling and traditional machine vision methods. Although these methods can detect defects, they suffer from insufficient accuracy in identifying complex defects, especially when the boundaries are blurred and the textures are similar, resulting in low detection accuracy. Furthermore, these methods cannot be linked with the production line in real time and lack automated rejection and traceability functions. Summary of the Invention

[0003] This invention provides a defect detection method, apparatus, electronic device, and storage medium to solve the problem of low defect detection accuracy.

[0004] According to one aspect of the present invention, a defect detection method is provided, comprising:

[0005] Determine the first image; the first image is a corrected image of the metal anodized surface.

[0006] The first image is subjected to defect detection by the first model to obtain a first result; the first model is configured with a first loss function; the first loss function can instruct the first model to distinguish between foreground feature information and background feature information of the first image; the first result is used to describe whether there are defects on the metal anodized surface.

[0007] According to another aspect of the present invention, a defect detection device is provided, comprising:

[0008] A first image determination module is used to determine a first image; the first image is a corrected image of a metal anodized surface.

[0009] The first result determination module is used to perform defect detection on the first image using a first model to obtain a first result; the first model is configured with a first loss function; the first loss function can instruct the first model to distinguish between foreground feature information and background feature information of the first image; the first result is used to describe whether there are defects on the metal anodized surface.

[0010] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0011] At least one processor; and

[0012] A memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the defect detection method according to any embodiment of the present invention.

[0014] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the defect detection method according to any embodiment of the present invention.

[0015] The technical solution of this invention involves determining a first image; performing defect detection on the first image using a first model to obtain a first result; the first model is configured with a first loss function; the first loss function instructs the first model to distinguish between foreground and background features of the first image, and the configuration of the first loss function ensures consistency of pixels of the same type in the feature space, thereby reducing boundary blurring and misclassification. This method performs defect detection on the first image using a first model. Because the first model is configured with a first loss function, it can align the foreground and background features of the first image, ensuring feature consistency. Therefore, the use of the first model can achieve automatic detection of metal oxide surfaces while also improving the accuracy of defect recognition for complex textures and edge regions on metal oxide surfaces.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of a defect detection method provided in an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of the structure of a ConvNeXt network provided in an embodiment of the present invention;

[0020] Figure 3 A flowchart of Fourier convolution processing provided for an embodiment of the present invention;

[0021] Figure 4 A flowchart for acquiring a time-frequency feature image provided in an embodiment of the present invention;

[0022] Figure 5 This is a schematic diagram of the structure of a defect detection device provided in an embodiment of the present invention;

[0023] Figure 6 A schematic diagram of the structure of an electronic device for implementing the defect detection method of this invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] Figure 1 This is a flowchart illustrating a defect detection method provided in an embodiment of the present invention. This embodiment is applicable to the detection of defects in metal workpieces. The method can be executed by a defect detection device, which can be implemented in hardware and / or software. This defect detection device can be configured in any electronic device with network communication capabilities. Figure 1 As shown, the method includes:

[0027] S110. Determine the first image; the first image is the corrected image of the metal anodized surface.

[0028] The corrections include: color correction, illumination normalization, and distortion correction.

[0029] Specifically, the second image is subjected to color correction, illumination normalization, and distortion correction to obtain the first image.

[0030] The second image is an image of the metal anodized surface captured by the imaging device.

[0031] S120. Defect detection is performed on the first image using the first model to obtain a first result; the first model is configured with a first loss function; the first loss function can instruct the first model to distinguish between foreground feature information and background feature information of the first image; the first result is used to describe whether there are defects on the metal anodized surface.

[0032] The first model includes a decoding layer, an encoding layer, and a first loss function.

[0033] The encoding layer uses the ConvNeXt network, which is a pure convolutional neural network that can capture richer and more robust visual features of metal anodized surface images while maintaining efficient computation and memory usage, making it suitable for high-precision and high-efficiency scenarios.

[0034] Furthermore, a network block of a ConvNeXt network, such as Figure 2 As shown in the figure, the input is a first image with 96 channels. This first image undergoes a depthwise separable convolution using a 7×7 kernel. The resulting feature image is then normalized, and a 1×1 convolution is used to convert the 96-dimensional image to 384 dimensions. Finally, an activation function restores the dimensions back to 96, and a residual concatenation is performed with the first image. Here, d represents the channel dimension.

[0035] The decoding layer incorporates Fourier convolutions with a skip fusion module.

[0036] Furthermore, the processing steps of Fourier convolution, such as Figure 3 As shown, the input feature X of the Fourier convolution has a shape of H×W×C. At the entrance of the Fourier convolution, feature X is split into local feature images X along the dimension of the feature channel. L and global feature image X G Local feature image X L shape Global feature image X G Shape For the local feature image X L Perform a convolution transformation and weight it with the feature image from the global feature transfer to obtain the local feature output feature, i.e., the fifth feature image Y. L For the global feature image X GFrequency domain transformation is performed, and features are extracted using a Transformer structure. These features are then weighted with the feature images derived from local feature transfer to obtain the global feature output, i.e., the sixth feature image Y. G The local features are output as feature Y. L and global feature output feature Y G Feature concatenation is performed to obtain the second feature image. Among them, H×W represents spatial resolution; C represents the number of channels.

[0037] Furthermore, the feature image transmitted from the global feature image and the feature image transmitted by the local feature image Generated by the jump fusion module.

[0038] The above steps introduce Fourier convolution to capture texture and edge information of the first image in the frequency domain and suppress high-frequency noise. In other words, Fourier convolution can transform temporal features to the frequency domain, allowing frequency domain information to feed back local features, improving the expressive power of local features, enhancing attention to texture and edge information, suppressing high-frequency noise and artifacts, and thus improving the ability to detect defects.

[0039] The first loss function consists of a second loss function and a third loss function. The second loss function characterizes the semantic consistency between foreground and background features in the metal anodized surface image. The third loss function uses standard binary cross-entropy to characterize the matching degree between the output image and the sixth image.

[0040] Further, the process of determining the second loss function is as follows: Feature extraction is performed on the sixth image using the second model to obtain at least one layer of feature images. The first layer of feature images is selected as the tenth feature image. The tenth feature image is multiplied element-wise with the foreground mask to obtain the foreground feature image. The tenth feature image is multiplied element-wise with the background mask to obtain the background feature image. The difference between the last layer of feature images in the foreground feature image and the remaining layer feature images is calculated and summed to obtain the foreground loss; the difference between the last layer of feature images in the background feature image and the remaining layer feature images is calculated and summed to obtain the background loss. The background loss and the foreground loss are summed to obtain the second loss function.

[0041] The sixth image is a corrected historical image of anodized metal surfaces. The second model is the first model, which was either untrained or trained to meet preset requirements.

[0042] Furthermore, the process of determining the first model is as follows: input the sixth image into the second model for defect detection, compare the detection result with the defect label corresponding to the sixth image, and adjust the weights of the second model according to the comparison result until the preset requirements are met. Then, the second model that meets the preset requirements is taken as the first model.

[0043] Specifically, the first model first performs depthwise separable convolution on the first image using a first preset convolution kernel. After convolution, channel expansion and compression are performed using a 1×1 convolution. The compressed feature image is then residually concatenated with the first image to obtain the first feature image. The first feature image is then split according to a preset number of channels. The split local feature image is used as the third feature image, and the split global feature image is used as the fourth feature image. The third feature image is then convolved using a third preset convolution kernel and weighted with the features passed from the fourth feature image to obtain the fifth feature image. The fourth feature image undergoes frequency domain feature extraction and is weighted with the features passed from the third feature image to obtain the sixth feature image. The fifth and sixth feature images are then concatenated bit-by-bit, and the concatenated image is used as the second feature image. The first layer feature image in the second feature image is selected and compared with the preset feature image. If the similarity between the first layer feature image and the preset feature image is greater than the preset similarity, the first result is that the first image has a defect. If the similarity between the first layer feature image and the preset feature image is less than or equal to the preset similarity, then the first result is that the first image has no defects.

[0044] Furthermore, in addition to including information on whether defects exist, the first result also includes at least: whether the metal oxide surface in the first image is glossy or matte, the color difference of the metal oxide surface in the first image, and the type of defect.

[0045] Furthermore, after obtaining the first result, the first image is annotated based on the first result, that is, the defects are marked with boxes and the defect type is noted.

[0046] Furthermore, after obtaining the first result, the defective metal is processed according to the first result. That is, if the first result indicates that the first image has a defect, the area of ​​the defect is calculated. The obtained area is compared with a preset area. If it exceeds the preset area, the metal belonging to the first image is considered unqualified and is removed; if it does not exceed the preset area, the metal belonging to the first image is considered qualified and is marked.

[0047] Furthermore, the labeling is achieved using an inkjet printer.

[0048] Furthermore, after obtaining the first result, the system also includes: real-time display of the first image, the first result, and the determination information of the metal corresponding to the first image. The detected information is saved to a database to achieve traceability and process optimization.

[0049] Optionally, determine the first image, including steps A1-A4:

[0050] Step A1: Determine the second image; the second image is an image obtained by taking a picture of the metal anodized surface using an imaging device.

[0051] Specifically, the metal to be tested is placed on a conveyor belt, and the oxide surface of the metal is photographed by an imaging device to obtain a second image.

[0052] The shooting device consists of an industrial shooting device and a multi-angle LED strip light source, and is fixed on the conveyor belt.

[0053] The above steps utilize multi-angle LEDs because different lighting angles can enhance surface texture differences and obtain high-quality images.

[0054] Step A2: Perform color correction on the second image to obtain the third image.

[0055] Specifically, the second image is subjected to color deviation judgment, and the corresponding correction method is matched according to the color deviation type determined. The second image corresponding to the problem is corrected by the obtained correction method to obtain the third image.

[0056] The types of color deviation are: color cast, too bright / too dark, insufficient contrast, and color gamut mismatch.

[0057] Furthermore, the correction method for color cast is white balance correction; the correction method for overbrightness / underbrightness is direct brightness adjustment; the correction method for insufficient contrast is histogram equalization; and the correction method for color gamut mismatch is color gamut mapping or color gamut compression.

[0058] The white balance correction process involves selecting a region with a neutral gray color from the second image, calculating the RGB average value of that region, determining the RGB channel gain based on the obtained RGB average value, and adjusting the corresponding color channels of the second image based on the obtained RGB channel gain.

[0059] Direct brightness adjustment involves adjusting the brightness of areas that are too bright or too low locally, or adjusting the brightness of the entire second image.

[0060] Histogram equalization converts the second image into a grayscale image, and then maps the grayscale distribution of the grayscale image to a uniform grayscale distribution through a transformation function.

[0061] Color gamut mapping converts an image from a source color gamut to a target color gamut, which is set according to requirements. Color gamut compression, for colors outside the target color gamut, either directly replaces them with the closest color within the target color gamut or proportionally reduces the color deviation.

[0062] Step A3: Perform illumination normalization on the third image to obtain the fourth image.

[0063] Specifically, the pixel values ​​of the third image are converted into a distribution with a mean of zero and a variance of one to obtain the fourth image.

[0064] Step A4: Perform distortion correction on the fourth image to obtain the first image.

[0065] Specifically, four vertices of the metal edge in the fourth image are selected, and four ideal corresponding points are defined. The transformation matrix is ​​solved based on the ideal corresponding points and vertices, and the fourth image is corrected using the transformation matrix to obtain the first image.

[0066] The ideal corresponding points can be set as (0,0), (O,0), (O,P), (0,P), where O and P are the ideal image sizes.

[0067] The correspondence between ideal corresponding points and vertices is as follows:

[0068] ;

[0069] in, Let (u, v) be the ideal corresponding point; (u, v) are the vertex coordinates. This is the normalization factor.

[0070] Optionally, defect detection is performed on the first image using the first model to obtain a first result, including steps B1-B3:

[0071] Step B1: The first model extracts features from the first image using a first preset convolution kernel to obtain a first feature image.

[0072] The first preset convolution kernel can be set to 7×7.

[0073] Specifically, the first model first performs depthwise separable convolution on the first image using a first preset convolution kernel. After convolution, the feature image obtained by the convolution is normalized through a normalization layer. After normalization, the normalized feature image is expanded to include more channels through a 1×1 convolution. After passing through an activation function, it is compressed back to the original number of channels through a 1×1 convolution. The compressed feature image is then residually concatenated with the first image to obtain the first feature image.

[0074] Step B2: Extract features from the first feature image in both the time domain and the frequency domain to obtain the second feature image.

[0075] Specifically, the first feature image is split into a local feature image and a global feature image according to a preset number of channels. The local feature image is used as the third feature image, and the global feature image is used as the fourth feature image. The third feature image is convolved with a third preset convolution kernel and weighted with the features passed from the fourth feature image to obtain the fifth feature image. Frequency domain features are extracted from the fourth feature image and weighted with the features passed from the third feature image to obtain the sixth feature image. The fifth and sixth feature images are then concatenated bit by bit, and the concatenated image is used as the second feature image.

[0076] Step B3: Determine the first result based on the second feature image and the preset feature image.

[0077] Specifically, the first layer feature image in the second feature image is selected, and compared with a preset feature image. If the similarity between the first layer feature image and the preset feature image is greater than the preset similarity, the first result is that the first image has a defect. If the similarity between the first layer feature image and the preset feature image is less than or equal to the preset similarity, the first result is that the first image does not have a defect.

[0078] Furthermore, similarity is confirmed using Euclidean distance. Specifically, the Euclidean distance between each pixel in the first-layer feature image and the preset feature image is calculated, and the average of all obtained Euclidean distances is used to obtain the similarity.

[0079] Optionally, the first feature image is subjected to feature extraction in both the time domain and the frequency domain to obtain the second feature image, including steps C1-C4:

[0080] Step C1: Split the first feature image according to the feature channels to obtain the third feature image and the fourth feature image.

[0081] Specifically, the first feature image is split into a local feature image and a global feature image according to a preset number of channels. The local feature image is used as the third feature image, and the global feature image is used as the fourth feature image.

[0082] Furthermore, the third feature image can be represented as: The fourth feature image can be represented as .

[0083] in, The preset number of channels, H×W represents spatial resolution; C represents the number of channels.

[0084] Step C2: Extract features from the third feature image using the second preset convolution kernel to obtain the fifth feature image.

[0085] The second preset convolution kernel can be set to 3×3.

[0086] Specifically, the third feature image is convolved with the second preset convolution kernel and weighted with the partial features transmitted from the fourth feature image to obtain the fifth feature image.

[0087] The fifth feature image can be represented as:

[0088] ;

[0089] Among them, X G The fourth feature image; X L The third feature image; The feature image passed to the fourth feature image is the feature image obtained after convolving the fourth feature image with the second preset convolution kernel; The feature image is obtained by convolving the third feature image with the second preset convolution kernel.

[0090] Step C3: Determine the sixth feature image based on the fourth and third feature images.

[0091] Specifically, the fourth feature image is first convolved using a third preset convolution kernel, and then the convolved feature image is transformed from the spatial domain to the frequency domain to obtain the seventh feature image. Each feature information in the seventh feature image is split according to its real and imaginary parts, and the obtained real features are concatenated, as are the imaginary features. The concatenated real and imaginary features are then concatenated again to obtain the eighth feature image. The eighth feature image is convolved using the third preset kernel to restore the number of channels, resulting in the ninth feature image. The ninth feature image is again split according to its real and imaginary parts, and the split real and imaginary features are weighted and then subjected to an inverse Fourier transform to convert the frequency domain features back to the time domain, obtaining the time-frequency feature image. The time-frequency feature image is weighted and combined with a portion of the feature image passed from the third feature image to obtain the sixth feature image.

[0092] The third preset convolution kernel can be set to 1×1.

[0093] Step C4: The fifth feature image and the sixth feature image are spliced ​​and fused to obtain the second feature image.

[0094] Specifically, the fifth and sixth feature images are spliced ​​bit by bit, and the spliced ​​image is used as the second feature image.

[0095] Optionally, a sixth feature image is determined based on the fourth and third feature images, including steps D1-D4:

[0096] Step D1: Transform the fourth feature image from the spatial domain to the frequency domain to obtain the seventh feature image.

[0097] Specifically, the fourth feature image is convolved using the third preset kernel to halve the number of channels. Then, normalization and activation functions are applied to obtain a feature image with half the channels. This feature image is then converted from the spatial domain to the frequency domain using a two-dimensional discrete Fourier transform, yielding the seventh feature image.

[0098] Furthermore, the channel-halved feature image can be represented as:

[0099] ;

[0100] Where ReLU is the activation function; BN is the normalization function.

[0101] Furthermore, the seventh feature image can be represented as:

[0102] ;

[0103] Where R is the width of the fourth feature image; Q is the height of the fourth feature image; and f(x,y) is the channel-halved feature image.

[0104] Step D2: The seventh feature image is split according to its real and virtual parts and then stitched together to obtain the eighth feature image.

[0105] Specifically, since the acquired seventh feature image contains complex features, each feature information of the seventh feature image is split according to its real and imaginary parts, and the acquired real features are spliced ​​together, the imaginary features are spliced ​​together, and the spliced ​​real features and spliced ​​imaginary features are spliced ​​together again to obtain the eighth feature image.

[0106] Furthermore, the eighth feature image can be represented as:

[0107] ;

[0108] in, The real features are those obtained after splicing. The imaginary part features after splicing; This is the eighth feature image.

[0109] Step D3: Convolve the eighth feature image using the third preset convolution kernel to obtain the ninth feature image.

[0110] Specifically, the eighth feature image is convolved with the third preset kernel to restore the number of channels, and then normalized and activated to obtain the ninth feature image.

[0111] Furthermore, the ninth feature image can be represented as:

[0112] .

[0113] Step D4: Decompose the ninth feature image according to its real and virtual parts, perform time-domain transformation, and weight it with the third feature image to obtain the sixth feature image.

[0114] Specifically, the ninth feature image is further split into real and virtual parts, and the split real and virtual part features are weighted and then subjected to inverse Fourier transform to convert the frequency domain features back to the time domain, resulting in a time-frequency feature image. The time-frequency feature image is then weighted with the partial feature image inherited from the third feature image to obtain the sixth feature image.

[0115] Furthermore, the process of determining the time-frequency feature image can be expressed as:

[0116] ;

[0117] ;

[0118] in, The feature image is obtained by weighting the real and virtual parts of the split feature.

[0119] Furthermore, the sixth feature image can be represented as:

[0120] ;

[0121] in, = This is a time-frequency feature image; X L The third feature image; The feature image passed to the third feature image is the feature image obtained after convolving the third feature image with the second preset convolution kernel.

[0122] For example, such as Figure 4 The figure shows the method for obtaining the time-frequency feature image. As can be seen from the figure, the fourth feature image is first convolved using a 1×1 kernel, followed by a two-dimensional discrete Fourier transform, and then the real and imaginary parts are separated. After separation, a 1×1 kernel is used for convolution, and the real and imaginary parts are separated again to obtain the imaginary and real feature images. The imaginary and real feature images are then weighted and subjected to an inverse Fourier transform to obtain the time-frequency feature image.

[0123] Optionally, at least one method for determining the first loss function includes steps E1-E5:

[0124] Step E1: Determine the tenth feature image; the tenth feature image is the feature image obtained after feature extraction from the sixth image; the sixth image is the corrected historical metal anodized surface image.

[0125] Specifically, features are extracted from the sixth image using the second model to obtain at least one layer of feature images. The first layer of feature images is selected from the at least one layer of feature images as the tenth feature image.

[0126] The second model is the first model that has not been trained or whose training does not meet the preset requirements.

[0127] Step E2: Extract features from the tenth feature image according to the foreground mask to obtain the foreground feature image.

[0128] The foreground mask is a binary matrix with the same size as the tenth feature image. Pixels with a value of 1 in the matrix represent defect areas in the tenth feature image, while pixels with a value of 0 in the matrix represent normal areas in the tenth feature image.

[0129] Specifically, the tenth feature image is multiplied element-wise with the foreground mask to obtain the foreground feature image.

[0130] Furthermore, the foreground feature image can be represented as:

[0131] ;

[0132] Where f is the tenth feature image; K is the foreground mask.

[0133] Step E3: Extract features from the tenth feature image according to the background mask to obtain the background feature image; the background mask and the foreground mask are complementary.

[0134] The background mask is a binary matrix with the same size as the tenth feature image. Pixels with a value of 1 in the matrix represent normal regions in the tenth feature image, while pixels with a value of 0 in the matrix represent defective regions in the tenth feature image.

[0135] Specifically, the tenth feature image is multiplied element-wise with the background mask to obtain the background feature image.

[0136] Furthermore, the background feature image can be represented as:

[0137] ;

[0138] Where f is the tenth feature image; 1-K is the background mask.

[0139] Step E4: Determine the second loss function based on the foreground feature image and the background feature image; the second loss function is used to characterize the semantic consistency between the foreground features and the background features.

[0140] Specifically, the foreground loss is obtained by subtracting the last layer feature image from the remaining layer feature images in the foreground feature image and then summing the results. Similarly, the background loss is obtained by subtracting the last layer feature image from the remaining layer feature images in the background feature image and then summing the results. The background loss and the foreground loss are then summed to obtain the second loss function.

[0141] Furthermore, the second loss function can be expressed as:

[0142] .

[0143] Step E5: Determine the first loss function based on the second and third loss functions; the third loss function is used to characterize the degree of matching between the output image and the sixth image.

[0144] The third loss function uses the standard binary cross-entropy.

[0145] Furthermore, the third loss function can be expressed as:

[0146] ;

[0147] in, This is the true label for the sixth image, which can be either 0 or 1. is the predicted probability of the sixth image, ranging from (0, 1); N is the number of sixth images.

[0148] Specifically, the third loss function is calculated based on the prediction results of the second model for the sixth image. The third loss function is then weighted with the second loss function to obtain the first loss function.

[0149] The first loss function is expressed as:

[0150] .

[0151] Where λ is the weight hyperparameter of the second loss function.

[0152] Furthermore, the second model is continuously modified using the sixth image, and λ is adjusted based on the prediction results of the training until the preset requirements are met. The second model that meets the preset requirements is then used as the first model.

[0153] The above steps, in determining λ, can control the impact of the second loss function on the training of the second model, so that the second model can not only learn good segmentation ability, but also enhance the distinction between foreground and background in the feature space.

[0154] Optionally, a second loss function is determined based on the foreground feature image and the background feature image, including steps F1-F3:

[0155] Step F1: Determine the foreground loss based on the foreground feature image.

[0156] Specifically, the foreground loss is obtained by subtracting the last layer feature image from the remaining layer feature images in the foreground feature image and summing the results.

[0157] Furthermore, assuming there are n layers of foreground feature images, the foreground loss can be expressed as:

[0158] ;

[0159] Where, N fg is the number of pixels in the foreground feature image; i is the pixel index of the pixel in the foreground feature image; L is the number of layers in the foreground feature image; The feature of the i-th pixel in the n-th foreground feature image; Let be the feature of the i-th pixel in the foreground feature image of layer L.

[0160] Step F2: Determine the background loss based on the background feature image.

[0161] Specifically, the background loss is obtained by subtracting the last layer feature image from the remaining layer feature images in the background feature image and then summing the results.

[0162] Furthermore, assuming there are n layers of background feature images, the background loss can be expressed as:

[0163] ;

[0164] Where, N bg is the number of pixels in the background feature image; i is the pixel index of the pixel in the background feature image; L is the number of layers in the background feature image; The feature of the i-th pixel in the n-th layer background feature image; Let be the feature of the i-th pixel in the background feature image of layer L.

[0165] Step F3: Weight the foreground loss and background loss to obtain the second loss function.

[0166] Specifically, the foreground loss and background loss are summed to obtain the second loss function.

[0167] Furthermore, the second loss function can be expressed as:

[0168] .

[0169] The steps described above, including the determination of the second loss function, enable the extracted features to focus more effectively on the most relevant features. Based on this, foreground and background features are aligned under supervision, balancing the second model's focus on the foreground and background. This ensures that pixels within the foreground and background have consistent semantic representations, reduces overlap in the feature space, minimizes boundary blurring and misclassification, and improves the segmentation accuracy of complex and edge regions.

[0170] The technical solution of this embodiment involves determining a first image; performing defect detection on the first image using a first model to obtain a first result; the first model is configured with a first loss function; the first loss function instructs the first model to distinguish between foreground and background features of the first image, and the configuration of the first loss function ensures consistency of pixels of the same type in the feature space, thereby reducing boundary blurring and misclassification. This method performs defect detection on the first image using the first model. Because the first model is configured with the first loss function, it can align the foreground and background features of the first image, ensuring feature consistency. Therefore, the use of the first model can achieve automatic detection of metal oxide surfaces while also improving the accuracy of defect recognition for complex textures and edge regions on metal oxide surfaces.

[0171] Figure 5 This is a schematic diagram of a defect detection device provided in an embodiment of the present invention. This embodiment is applicable to the detection of defects in metal workpieces. The defect detection device can be implemented in hardware and / or software, and can be configured in any electronic device with network communication capabilities. Figure 5 As shown, the device includes: a first image determination module 210 and a first result determination module 220, wherein:

[0172] First image determination module 210: used to determine a first image; the first image is a corrected image of the metal anodized surface;

[0173] First result determination module 220: used to perform defect detection on the first image through the first model to obtain a first result; the first model is configured with a first loss function; the first loss function can instruct the first model to distinguish the foreground feature information and background feature information of the first image; the first result is used to describe whether there are defects on the metal anodized surface.

[0174] Optionally, the first image determination module 210 includes:

[0175] Second image determination unit: used to determine a second image; the second image is an image obtained by taking a picture of the metal anodized surface using an imaging device;

[0176] The third image determination unit is used to perform color correction on the second image to obtain the third image.

[0177] Fourth image determination unit: used to perform illumination normalization processing on the third image to obtain the fourth image;

[0178] First image determination unit: used to perform distortion correction on the fourth image to obtain the first image.

[0179] Optionally, the first result determination module 220 includes:

[0180] First feature image determination unit: used by the first model to extract features from the first image using a first preset convolution kernel to obtain the first feature image;

[0181] Second feature image determination unit: used to extract features from the first feature image in the time domain and frequency domain respectively to obtain the second feature image;

[0182] First Result Determination Unit: Used to determine the first result based on the second feature image and the preset feature image.

[0183] Optionally, the second feature image determination unit includes:

[0184] Feature image determination subunit: used to split the first feature image according to the feature channels to obtain the third feature image and the fourth feature image;

[0185] The fifth feature image determination subunit is used to extract features from the third feature image using the second preset convolution kernel to obtain the fifth feature image.

[0186] The sixth feature image determination subunit is used to determine the sixth feature image based on the fourth and third feature images.

[0187] The second feature image determination subunit is used to stitch and fuse the fifth feature image and the sixth feature image to obtain the second feature image.

[0188] Optionally, the sixth feature image determines the subunit, specifically used for:

[0189] The fourth feature image is transformed from the spatial domain to the frequency domain to obtain the seventh feature image;

[0190] The seventh feature image is split according to its real and virtual parts and then stitched together to obtain the eighth feature image;

[0191] The eighth feature image is convolved with the third preset convolution kernel to obtain the ninth feature image;

[0192] The ninth feature image is split according to its real and virtual parts, and then transformed in the time domain. It is then weighted with the third feature image to obtain the sixth feature image.

[0193] Optionally, the first result determination module 220 includes:

[0194] The tenth feature image determination unit is used to determine the tenth feature image; the tenth feature image is the feature image obtained after feature extraction from the sixth image; the sixth image is a corrected historical metal anodized surface image.

[0195] Foreground Feature Image Determination Unit: Used to extract features from the tenth feature image according to the foreground mask to obtain the foreground feature image;

[0196] Background Feature Image Determination Unit: Used to extract features from the tenth feature image according to the background mask to obtain the background feature image; the background mask is complementary to the foreground mask.

[0197] The second loss function determination unit is used to determine the second loss function based on the foreground feature image and the background feature image; the second loss function is used to characterize the semantic consistency between the foreground features and the background features.

[0198] The first loss function determination unit is used to determine the first loss function based on the second and third loss functions; the third loss function is used to characterize the degree of matching between the output image and the sixth image.

[0199] Optionally, the second loss function determination unit includes:

[0200] Foreground loss determination subunit: used to determine the foreground loss based on the foreground feature image;

[0201] Background loss determination subunit: used to determine the background loss based on the background feature image;

[0202] The second loss function determines the sub-unit: it is used to weight the foreground loss and the background loss to obtain the second loss function.

[0203] The defect detection device provided in the embodiments of the present invention can execute the defect detection method provided in any of the embodiments of the present invention, and has the corresponding functions and beneficial effects of executing the defect detection method. For details, please refer to the relevant operations of the defect detection method in the foregoing embodiments.

[0204] Figure 6This is a schematic diagram of the structure of an electronic device for implementing the defect detection method of this invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0205] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0206] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0207] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as defect detection methods.

[0208] In some embodiments, the defect detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the defect detection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the defect detection method by any other suitable means (e.g., by means of firmware).

[0209] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0210] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0211] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0212] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0213] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0214] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0215] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0216] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A defect detection method, characterized in that, include: Determine the first image; The first image is a corrected image of the metal anodized surface; The first image is subjected to defect detection using the first model to obtain a first result; The first model is configured with a first loss function; the first loss function can instruct the first model to distinguish between foreground feature information and background feature information of the first image; The first result is used to describe whether there are defects on the metal anodized surface.

2. The method according to claim 1, characterized in that, Determining the first image includes: Determine the second image; the second image is an image obtained by taking a picture of the metal anodized surface using an imaging device; The second image is color-corrected to obtain the third image; The third image is subjected to illumination normalization processing to obtain the fourth image; Distortion correction is performed on the fourth image to obtain the first image.

3. The method according to claim 1, characterized in that, The step of performing defect detection on the first image using the first model to obtain a first result includes: The first model extracts features from the first image using a first preset convolutional kernel to obtain a first feature image; The first feature image is used to extract features from both the time domain and the frequency domain to obtain the second feature image. The first result is determined based on the second feature image and the preset feature image.

4. The method according to claim 3, characterized in that, The step of extracting features from the first feature image in both the time domain and the frequency domain to obtain the second feature image includes: The first feature image is split according to feature channels to obtain a third feature image and a fourth feature image; The third feature image is processed using a second preset convolution kernel to extract features, resulting in a fifth feature image. The sixth feature image is determined based on the fourth feature image and the third feature image; The fifth feature image and the sixth feature image are spliced ​​and fused to obtain the second feature image.

5. The method according to claim 4, characterized in that, Determining the sixth feature image based on the fourth feature image and the third feature image includes: The fourth feature image is transformed from the spatial domain to the frequency domain to obtain the seventh feature image; The seventh feature image is split according to its real and virtual parts and then stitched together to obtain the eighth feature image; The eighth feature image is convolved using a third preset convolution kernel to obtain the ninth feature image; The ninth feature image is split according to its real and virtual parts, and then transformed in the time domain. It is then weighted with the third feature image to obtain the sixth feature image.

6. The method according to claim 1, characterized in that, At least one method for determining the first loss function includes: The tenth feature image is determined; the tenth feature image is the feature image obtained after feature extraction from the sixth image; the sixth image is a corrected historical metal anodized surface image. The tenth feature image is subjected to feature extraction according to the foreground mask to obtain the foreground feature image; The tenth feature image is subjected to feature extraction according to the background mask to obtain a background feature image; the background mask is complementary to the foreground mask; A second loss function is determined based on the foreground feature image and the background feature image; the second loss function is used to characterize the semantic consistency between the foreground features and the background features. The first loss function is determined based on the second loss function and the third loss function; the third loss function is used to characterize the degree of matching between the output image and the sixth image.

7. The method according to claim 6, characterized in that, The step of determining the second loss function based on the foreground feature image and the background feature image includes: Determine the foreground loss based on the foreground feature image; Determine the background loss based on the background feature image; The foreground loss and the background loss are weighted together to obtain the second loss function.

8. A defect detection device, characterized in that, include: The first image determination module is used to determine the first image; The first image is a corrected image of the metal anodized surface; The first result determination module is used to perform defect detection on the first image using a first model to obtain a first result; The first model is configured with a first loss function; the first loss function can instruct the first model to distinguish between foreground feature information and background feature information of the first image; The first result is used to describe whether there are defects on the metal anodized surface.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the defect detection method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the defect detection method according to any one of claims 1-7.

Images