Line defect detection method and training method and device of defect detection model

Abstract

The invention provides a line defect detection method and a defect detection model training method and device, and the method comprises the steps: obtaining a grayscale image of a target product, and cutting the grayscale image into a plurality of image blocks; inputting each image block into a pre-trained defect detection model, respectively extracting horizontal path features and vertical path features of the image blocks by using the defect detection model, and carrying out feature fusion on the horizontal path features and the vertical path features to obtain a fusion feature map, outputting a defect prediction map corresponding to each image block based on the fused feature map; the horizontal path features are used for representing defect features of the image blocks in the row direction, and the vertical path features are used for representing defect features of the image blocks in the column direction; generating a defect distribution map corresponding to the target product according to the defect prediction map; and determining line defect information in the target product based on the defect distribution diagram. According to the method, the accuracy of detecting the line defect of the product can be improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method for detecting line defects, a method for training a defect detection model, and an apparatus. Background Technology

[0002] With the rapid development of computer vision and industrial precision inspection technologies, deep learning-based automatic surface defect detection technology has become a key means to improve production quality and efficiency. This technology automatically extracts image features through convolutional neural networks, effectively replacing traditional manual visual inspection and finding applications in many industrial scenarios.

[0003] Traditional technical solutions for detecting line defects in products mainly include two types: one is based on traditional image processing methods, such as using operators like Canny and Sobel for edge detection, combined with threshold segmentation to identify fine, elongated defects like scratches and bright lines; the other is to use general-purpose deep learning models, such as the YOLO (You Only Look Once) series of object detection networks or the U-Net series of semantic segmentation networks, to locate or classify defect areas at the pixel level. However, line defects, due to their unique morphological characteristics—extremely low pixel ratio, extremely high aspect ratio, diverse directional distribution, and weak contrast with the background—make it impossible for existing detection methods to accurately detect line defects in products.

[0004] Therefore, how to accurately detect line defects in products is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a method and apparatus for detecting line defects, a method and apparatus for training a defect detection model, a terminal device, a computer-readable storage medium, and a computer program product, with the aim of improving the accuracy of detecting line defects in products.

[0006] Firstly, this application provides a method for detecting line defects. The method includes: Obtain a grayscale image of the target product and cut the grayscale image into multiple image blocks; Each image block is input into a pre-trained defect detection model. The defect detection model extracts the horizontal and vertical path features of each image block. The horizontal and vertical path features are then fused to obtain a fused feature map. Based on the fused feature map, a defect prediction map corresponding to each image block is output. The horizontal path features are used to characterize the defect features of the image block in the row direction, and the vertical path features are used to characterize the defect features of the image block in the column direction. Generate a defect distribution map corresponding to the target product based on the defect prediction map; Based on the defect distribution map, determine the line defect information in the target product.

[0007] In one embodiment, the step of outputting a defect prediction map corresponding to each of the image blocks based on the fused feature map includes: Defect prediction is performed on the fused feature map using the first prediction branch to obtain a first-scale prediction map; Upsample the first-scale prediction map to obtain a preset-size prediction map; The second prediction branch is used to predict defects in the preset size prediction map to obtain a second-scale prediction map; the second detection scale of the second-scale prediction map is larger than the first detection scale of the first-scale prediction map. The first-scale prediction map and the second-scale prediction map are fused to obtain a defect prediction map corresponding to the image block.

[0008] In one embodiment, the step of extracting the horizontal and vertical path features of the image patch using the defect detection model includes: The defect detection model is used to determine a first intermediate feature map of the image block in the horizontal direction and a second intermediate feature map of the image block in the vertical direction. Channel attention and spatial attention calculations are performed on the first intermediate feature map to obtain a horizontally enhanced feature map, and the horizontal path features of the image block are determined based on the horizontally enhanced feature map; Channel attention and spatial attention calculations are performed on the second intermediate feature map to obtain a vertical enhancement feature map, and the vertical path features of the image block are determined based on the vertical enhancement feature map.

[0009] In one embodiment, generating a defect distribution map corresponding to the target product based on the defect prediction map includes: The defect prediction map is binarized and segmented to obtain a binarized defect identification map corresponding to the image block; The binarized defect identification images are stitched together according to the cut positions of the corresponding image blocks to obtain a defect distribution map corresponding to the target product.

[0010] In one embodiment, after binarizing the defect prediction map to obtain a binarized defect identification map corresponding to the image patch, the method further includes: A morphological closing operation is performed on the binarized defect marker image to obtain a processed binarized defect marker image.

[0011] Secondly, this application also provides a training method for a defect detection model, wherein the defect detection model is used to process an input grayscale image of a target product to generate a defect prediction map of the target product, and the defect prediction map is used to determine the line defect information of the target product; the training method includes: Obtain a training sample set; the training sample set includes product sample images and defect annotation images corresponding to the product sample images; The target network structure is iteratively trained based on the training sample set until it converges, thus obtaining the defect detection model. The target network structure includes a horizontal feature extraction path, a vertical feature extraction path, a feature fusion layer, and a prediction module. The horizontal and vertical feature extraction paths are used to extract the horizontal and vertical path features of the product sample image, respectively. The horizontal path features characterize the defect features of the product sample image in the row direction, and the vertical path features characterize the defect features of the product sample image in the column direction. The feature fusion layer fuses the horizontal and vertical path features to obtain a fused feature map. The prediction module outputs a corresponding defect prediction map based on the fused feature map.

[0012] In one embodiment, the target network structure further includes a first prediction branch and a second prediction branch; The first prediction branch is used to predict defects in the sample fusion feature map to obtain a first-scale prediction map of the sample. Upsample the first-scale prediction map of the sample to obtain a preset-size prediction map of the sample; The second prediction branch is used to predict defects in the sample preset size prediction map to obtain a sample second-scale prediction map; the second detection scale of the sample second-scale prediction map is greater than the first detection scale of the sample first-scale prediction map.

[0013] In one embodiment, the step of iteratively learning and training the target network structure based on the training sample set until the target network structure converges to obtain the defect detection model includes: The product sample image is input into the current target network structure to obtain the defect prediction probability map output by the target network structure; The first loss value and the second loss value of the current target network structure are calculated based on the defect prediction probability map and the corresponding defect annotation map, and a mixed loss value is determined based on the first loss value and the second loss value. The first loss value is used to reduce the training bias caused by the imbalance of the number of positive and negative samples in the training sample set, and to reduce the weight contribution of easily distinguishable samples in the loss calculation. The second loss value represents the degree of overlap between the defect prediction probability map and the defect annotation map. Determine whether the current target network structure has achieved the training objective based on the hybrid loss value or the number of training iterations. If the training objective is achieved, the current target network structure is determined to have converged, and the current target network structure is output as the defect detection model. If the training objective is not achieved, the model parameters of the current target network structure are adjusted using the mixed loss value, the current target network structure is updated, and the process returns to the step of inputting the product sample image into the target network structure to obtain the defect prediction probability map output by the target network structure, and subsequent steps.

[0014] Thirdly, this application also provides a device for detecting line defects, the device comprising: The acquisition module is used to acquire a grayscale image of the target product and cut the grayscale image into multiple image blocks; The detection module is used to input each of the image blocks into a pre-trained defect detection model, extract the horizontal path features and vertical path features of the image blocks using the defect detection model, fuse the horizontal path features and vertical path features to obtain a fused feature map, and output a defect prediction map corresponding to each of the image blocks based on the fused feature map; the horizontal path features are used to characterize the defect features of the image blocks in the row direction, and the vertical path features are used to characterize the defect features of the image blocks in the column direction; The execution module is used to generate a defect distribution map corresponding to the target product based on the defect prediction map; The output module is used to determine the line defect information in the target product based on the defect distribution map.

[0015] Fourthly, this application also provides a training apparatus for a defect detection model, characterized in that the defect detection model is used to process an input grayscale image of a target product to generate a defect prediction map of the target product, the defect prediction map being used to determine the line defect information of the target product; the apparatus includes: The sample acquisition module is used to acquire a training sample set; the training sample set includes product sample images and defect annotation images corresponding to the product sample images; The learning and training module is used to iteratively learn and train the target network structure based on the training sample set until the target network structure converges to obtain the defect detection model. The target network structure includes a horizontal feature extraction path, a vertical feature extraction path, a feature fusion layer, and a prediction module. The horizontal feature extraction path and the vertical feature extraction path are used to extract the sample horizontal path features and sample vertical path features of the product sample image, respectively. The sample horizontal path features are used to characterize the defect features of the product sample image in the row direction, and the sample vertical path features are used to characterize the defect features of the product sample image in the column direction. The feature fusion layer is used to fuse the sample horizontal path features and the sample vertical features to obtain a sample fusion feature map. The prediction module is used to output the corresponding sample defect prediction map based on the sample fusion feature map.

[0016] Fifthly, this application also provides a terminal device. The terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0017] Sixthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described above.

[0018] In a seventh aspect, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described above.

[0019] This application provides a method for detecting line defects. The method utilizes a defect detection model to extract horizontal and vertical path features from image blocks. Horizontal path features characterize defects in the row direction, while vertical path features characterize defects in the column direction. The horizontal and vertical path features are fused to obtain a fused feature map, and a defect prediction map corresponding to each image block is output based on the fused feature map. A defect distribution map of the target product is generated based on the defect prediction map. Based on the defect distribution map, line defect information in the target product is determined. Therefore, this method, by extracting defect features from both horizontal and vertical directions and fusing the horizontal and vertical path features, can more comprehensively detect line defects in the target product. Furthermore, by fusing the horizontal and vertical path features, diagonal line defects can be effectively identified. Thus, this method can improve the accuracy of detecting line defects in products.

[0020] It is understood that the line defect detection device, the defect detection model training method and device, the terminal device, the computer-readable storage medium and the computer program product provided in the embodiments of this application have the same beneficial effects as the above-described line defect detection, and will not be repeated here. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a method for detecting line defects provided in an embodiment of this application; Figure 2 An architecture diagram of a defect detection model provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a CBMA attention module provided in an embodiment of this application; Figure 4 A flowchart illustrating a training method for a defect detection model provided in an embodiment of this application; Figure 5 The diagram shown is a structural schematic of a line defect detection device provided in an embodiment of this application; Figure 6 The diagram shown is a structural schematic of a training device for a defect detection model provided in an embodiment of this application. Figure 7 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0023] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0024] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0025] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0026] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0027] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0028] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. "A plurality" means "two or more."

[0029] The embodiment of this application provides a method for detecting line defects, which can be executed by the processor of a terminal device when running a corresponding computer program.

[0030] Figure 1 The flowchart illustrates a method for detecting line defects according to an embodiment of this application. For ease of explanation, only the parts relevant to this embodiment are shown. The method provided in this embodiment includes the following steps: S100: Acquire a grayscale image of the target product and cut the grayscale image into multiple image blocks.

[0031] It should be noted that the target products in this embodiment include industrial products such as Micro OLED screens, flexible display panels, metal films, and glass substrates. Line defects are a key factor affecting the surface quality of the target products, including micron-level bright lines, dark lines, scratches, and other elongated defects. These line defects have significant characteristics, such as extremely low pixel ratio (often a single pixel width), extremely high aspect ratio (generally 1:100 to 1:10000), diverse directional distribution (0°-180° omnidirectional coverage), and weak contrast with the background (easily affected by lighting, texture, and reflections).

[0032] In this embodiment, grayscale images are typically represented as single-channel matrices with pixel values ​​ranging from 0 to 255. The resolution of grayscale images can be 2K, 4K, or 8K, and this embodiment does not impose any limitations on this. Specifically, the target product can be directly photographed using a monochrome industrial camera to obtain a grayscale image; alternatively, an RGB image of the target product can be acquired first using a color camera, and then the RGB image can be converted to a grayscale image.

[0033] Specifically, a grayscale image can be cut into multiple image blocks according to a preset fixed cutting size, such as 512×512, in 1×H×W format; or the cutting size can be determined based on the size of the target product, and the grayscale image can be cut into multiple image blocks according to that cutting size.

[0034] In practical applications, after the image blocks are determined, each image block can be further preprocessed. The preprocessing operations include histogram equalization (CLAHE), clip limit (which can be set to 0.01), and suppression of local overly bright areas (such as substrate reflection) to enhance image features.

[0035] S200: Input each image block into the pre-trained defect detection model, use the defect detection model to extract the horizontal path features and vertical path features of the image blocks respectively, fuse the horizontal path features and vertical path features to obtain a fused feature map, and output the defect prediction map corresponding to each image block based on the fused feature map; the horizontal path features are used to characterize the defect features of the image block in the row direction, and the vertical path features are used to characterize the defect features of the image block in the column direction.

[0036] The defect detection model can be a DualPathNet model. In this step, the preprocessed image patches are input into the pre-trained defect detection model.

[0037] The horizontal path feature refers to the feature map obtained by detecting horizontal line defect features in an image patch through the horizontal feature extraction path (H-Path) in the defect detection model. Its dimensions are [C, H, W], where C is the number of channels (e.g., 64), and H and W are the height and width, respectively. The vertical path feature refers to the feature map obtained by detecting vertical line defect features in an image patch through the vertical feature extraction path (V-Path), also with dimensions [C, H, W]. The horizontal and vertical feature extraction paths (H-Path) are structurally symmetrical, but their core parameters are designed differently.

[0038] Figure 2 This is an architecture diagram of a defect detection model provided in an embodiment of this application. Figure 2 As shown, the horizontal feature extraction path (H-Path) includes: Convolutional layer (Cov): A 3×15 two-dimensional convolutional kernel is used to scan the image patch to obtain the corresponding feature map; the "wide" convolutional kernel is more sensitive to the pixel association and continuity changes in the horizontal direction (i.e., the row direction) of the image patch. Its parameters are set as follows: stride (1,2), padding=(1,7), and output channels are 64. Normalization layer (BatchNorm2d): Performs batch normalization on the 64-channel feature map output by the convolutional layer, that is, standardizes the data distribution of each channel within a small batch, stabilizes the training process, accelerates model convergence, and suppresses model overfitting. Hardswish activation layer: Applying the Hardswish activation function to the normalized feature map can balance computational efficiency and feature representation ability; Pooling layer (MaxPool): Employs a (2,1) size max pooling operation. This operation performs a 2x downsampling only in the height direction (vertical direction) while keeping the width direction (horizontal direction) unchanged. The output feature map size is 64×256×256.

[0039] The vertical feature extraction path (H-Path) includes: Convolutional layer (Cov): A 15×3 two-dimensional convolutional kernel scans the input image to obtain the corresponding feature map; "high" convolutional kernels are more sensitive to pixel associations and continuity changes in the vertical direction (i.e., column direction) of image patches. Its parameters are set as follows: stride (2,1), padding=(7,1), and output channels are 64. Normalization layer (BatchNorm2d): Performs batch normalization on the 64-channel feature map output by the convolutional layer, that is, standardizes the data distribution of each channel within a small batch, stabilizes the training process, accelerates model convergence, and suppresses model overfitting. Hardswish activation layer: Applying the Hardswish activation function to the normalized feature map can balance computational efficiency and feature representation ability; Pooling layer (MaxPool): Employs a max pooling operation of size (1,2), performing a 2x downsampling in the width direction (horizontal direction) and keeping it unchanged in the height direction (vertical direction), resulting in an output feature map of size 64×256×256, which is used for feature fusion with the output of the horizontal path (H-Path).

[0040] In practical applications, when an image patch is input into the defect detection model, the image patch flows in parallel into the horizontal feature extraction path (H-Path) and the vertical feature extraction path (V-Path), respectively extracting the horizontal path features and the vertical path features of the image patch; then the horizontal path features and the vertical path features are fused, specifically by concatenating the horizontal path features and the vertical path features in the channel dimension to obtain a fused feature map.

[0041] The defect prediction map refers to the output map of the defect prediction model, which represents the line defects in a single image patch. In the defect prediction map, each pixel value represents the confidence (probability) that the pixel belongs to a line defect. For example, assuming a pixel value of 0.92, it means that there is a 92% probability that the pixel can be identified as having a line defect. Specifically, for the feature fusion map, high-dimensional features are mapped back to the original pixel space, and a defect probability value is calculated for each pixel to obtain the defect prediction map.

[0042] S300: Generates a defect distribution map corresponding to the target product based on the defect prediction map.

[0043] The defect distribution map refers to a binarized image with the same size as the grayscale image of the target product. Specifically, each pixel value in the defect prediction map is binarized and segmented, with each pixel having only two possible discrete values. For example, 1 (or 255) indicates that the pixel has a line defect, while 0 indicates that the pixel is either background or a defect-free area.

[0044] Specifically, after determining the defect prediction map, the binarized defect identification map obtained after processing all image blocks is reassembled according to the accurate positions and order cut from the grayscale image of the target product. The set of all pixels with a value of 1 (or 255) in the image represents the location, shape, and distribution of all line defects detected on the surface of the target product.

[0045] S400: Determine line defect information in the target product based on the defect distribution map.

[0046] Among them, line defect information refers to data used to describe the characteristics of detected line defects, including whether a line defect exists, i.e., the judgment result of whether a line defect exists or not in the target product; the location of the line defect, i.e., recording the center coordinates or circumscribed rectangle coordinates of the defect area corresponding to the line defect; the length and width of the line defect; the direction of the line defect, i.e., the direction of the connected domain of the line defect, such as the horizontal or vertical direction; the total number of pixels with line defects and the severity of the line defects, etc. This embodiment does not limit the specific content of the line defect information.

[0047] Specifically, based on the defect distribution map, connected component analysis is performed to determine the set of pixels with interconnected line defects (values ​​of 1 or 255), resulting in one or more independent line defect connected components, that is, one or more line defects are identified, and then the corresponding line defect information is determined based on the identified line defects.

[0048] This application provides a method for detecting line defects. The method utilizes a defect detection model to extract horizontal and vertical path features from image blocks. Horizontal path features characterize defects in the row direction, while vertical path features characterize defects in the column direction. The horizontal and vertical path features are fused to obtain a fused feature map, and a defect prediction map corresponding to each image block is output based on the fused feature map. A defect distribution map of the target product is generated based on the defect prediction map. Based on the defect distribution map, line defect information in the target product is determined. Therefore, this method, by extracting defect features from both horizontal and vertical directions and fusing the horizontal and vertical path features, can more comprehensively detect line defects in the target product. Furthermore, by fusing the horizontal and vertical path features, diagonal line defects can be effectively identified. Thus, this method can improve the accuracy of detecting line defects in products.

[0049] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, the defect prediction map corresponding to each image block is output based on the fused feature map, including: Defect prediction is performed on the fused feature map using the first prediction branch to obtain the first-scale prediction map; Upsample the first-scale prediction map to obtain a prediction map of the preset size; The second prediction branch is used to predict defects in the preset size prediction map to obtain the second scale prediction map; the second detection scale of the second scale prediction map is larger than the first detection scale of the first scale prediction map. The first-scale prediction map and the second-scale prediction map are fused to obtain the defect prediction map corresponding to the image patch.

[0050] The first prediction branch (output head 1) is a network branch that uses the first detection scale for defect prediction, and the first-scale prediction graph refers to the prediction graph corresponding to the first prediction branch (output head 1). The second prediction branch (output head 2) is a network branch that uses the second detection scale for defect prediction; the second-scale prediction graph refers to the prediction graph corresponding to the second prediction branch (output head 2).

[0051] In this embodiment, after using the first prediction branch to predict defects in the fused feature map and obtaining a first-scale prediction map, the first-scale prediction map is directly used for feature fusion, and simultaneously upsampled to obtain a preset-size prediction map. Then, the second prediction branch is used to predict defects in the preset-size prediction map to obtain a second-scale prediction map. Therefore, in this embodiment, the second detection scale of the second-scale prediction map is larger than the first detection scale of the first-scale prediction map.

[0052] In a specific instance, combined Figure 2 As shown, the first detection scale of the first-scale prediction map is 1×512×512, and the second detection scale of the second-scale prediction map is 1×1024×1024.

[0053] It should be noted that the first prediction branch (output head1) includes: Convolutional layer (Cov): Employs a 3×3 convolutional kernel with padding=1, increasing the number of input channels from 128 to 256 to enhance the expressive power of fused features; Up-Sample: The bilinear interpolation method is used, and the scaling factor is set to 2 to restore the feature map size to the default 256×512×512 (i.e., H2=W2=512). Convolutional layer (Cov): Using a 1×1 convolutional kernel, the number of input channels is reduced from 256 to 1, thereby outputting a single-channel prediction map for the first detection scale, with a size of 1×512×512.

[0054] The second prediction branch (output head2) includes: Up-Sample: Employs bilinear interpolation with a scaling factor of 1, and is used to receive the first-scale prediction map output from the first prediction branch. Convolutional layer (Cov): Uses a 3×3 convolutional kernel, sets padding=1, and has 1 input and 1 output channel. It smooths the first-scale prediction map of the input to integrate contextual information. Dynamic upsampling layer: During the training phase, it is dynamically adjusted to adjust the output size of the feature map to 1×1024×1024 through interpolation operations, thereby obtaining the second-scale prediction map.

[0055] Specifically, the first-scale prediction map output by the first prediction branch and the second-scale prediction map output by the second prediction branch are fused using methods such as weighted averaging and feature addition to obtain the defect prediction map corresponding to the image patch.

[0056] In this embodiment, the design of the multi-scale prediction branch can take into account line defects of different scales in the image patch, such as short line defects, medium line defects, and long line defects. In practical applications, the detection rate for line defects with a length ≥ 5 pixels is ≥ 95%. Therefore, this method can avoid missing short line defects and preventing long line defects from breaking, thus improving the accuracy of line defect detection.

[0057] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, the horizontal path features and vertical path features of the image blocks are extracted using a defect detection model, including: The defect detection model is used to determine the first intermediate feature map of the image patch in the horizontal direction and the second intermediate feature map of the image patch in the vertical direction. Channel attention and spatial attention are calculated on the first intermediate feature map to obtain a horizontal enhancement feature map, and the horizontal path features of the image block are determined based on the horizontal enhancement feature map. Channel attention and spatial attention are calculated on the second intermediate feature map to obtain a vertical enhancement feature map, and the vertical path features of the image block are determined based on the vertical enhancement feature map.

[0058] Specifically, the image patch is processed sequentially using the convolutional layer, normalization layer, and activation layer in the horizontal feature extraction path (H-Path) of the defect detection model to obtain the first intermediate feature map of the image patch in the horizontal direction; the image patch is processed sequentially using the convolutional layer, normalization layer, and activation layer in the vertical feature extraction path (V-Path) of the defect detection model to obtain the second intermediate feature map of the image patch in the vertical direction.

[0059] Determine the channel weight vector and spatial weight vector corresponding to the horizontal and vertical feature extraction paths, respectively. The channel weight vector is a C×1×1 vector, where C is the number of feature channels, such as 64. Each value in the vector represents the global importance of the corresponding feature channel.

[0060] The spatial weight vector refers to a 1×H×W feature map, where H and W are the height and width of the feature map, respectively. Each value in the vector represents the importance of the corresponding spatial location. Regions that may have line defects have high weights, while background regions have low weights.

[0061] Figure 3 This is a schematic diagram of the structure of a CBMA attention module provided in an embodiment of this application, combined with... Figure 2 and Figure 3 As shown, in this embodiment, after the activation layer (Hardswish) of the horizontal feature extraction path, a CBMA attention module is set to enhance the horizontal defect features from both the channel weight vector and spatial weight vector dimensions, suppressing background noise. Similarly, after the activation layer (Hardswish) of the vertical feature extraction path, a CBMA attention module is set to enhance the vertical defect features from both the channel weight vector and spatial weight vector dimensions, suppressing background noise. Therefore, the horizontal feature extraction path outputs horizontal path features enhanced in both dimensions, and the vertical feature extraction path outputs vertical path features enhanced in both dimensions.

[0062] For the horizontal feature extraction path, channel attention is calculated on the first intermediate feature map to obtain a C×1×1 horizontal channel weight vector; each value in the horizontal channel weight vector represents the global importance of the corresponding feature channel to the lateral defect. The horizontal channel weight vector is then multiplied channel by channel with the first intermediate feature map to obtain the horizontal channel enhanced feature map.

[0063] Spatial attention is calculated on the horizontal channel enhanced feature map to generate a 1×H×W horizontal spatial weight vector. The horizontal spatial weight vector has high weights in continuous spatial regions where horizontal line defects may occur, and low weights in background regions. The horizontal spatial weight vector is multiplied spatially with the horizontal channel enhanced feature map to obtain the horizontal enhanced feature map. The horizontal enhanced feature map is then processed through a targeted pooling layer (e.g., (2,1) pooling for H-Path) to obtain the horizontal path features.

[0064] For the vertical feature extraction path, channel attention is calculated on the second intermediate feature map to obtain a C×1×1 vertical channel weight vector; each value in the vertical channel weight vector represents the global importance of the corresponding feature channel to the vertical defect. The vertical channel weight vector is then multiplied channel by channel with the second intermediate feature map to obtain the vertical channel enhanced feature map.

[0065] Spatial attention is calculated on the vertical channel enhancement feature map to generate a 1×H×W vertical spatial weight vector. The vertical spatial weight vector has high weights in continuous spatial regions where vertical line defects may occur, and low weights in background regions. The vertical spatial weight vector is multiplied spatially with the vertical channel enhancement feature map to obtain the vertical enhancement feature map. The vertical enhancement feature map is then processed through a targeted pooling layer (e.g., (2,1) pooling for H-Path) to obtain the vertical path features.

[0066] In a specific example, taking the horizontal feature extraction path as an example, the process of determining the horizontal channel weight vector is as follows: Global average pooling (GAP) is used to calculate the average value of all pixels in each channel of the first intermediate feature map, resulting in a 64×1×1 first feature vector. Global max pooling (GMP) is used to extract the maximum value of each channel, resulting in a 64×1×1 second feature vector. The first and second feature vectors are concatenated along the channel dimension to obtain a 128×1×1 joint feature. The joint feature is then dimensionality-reduced using a 1×1 convolutional layer (compression factor r=8). ReLU activation is applied, and the dimensionality is increased to 64 using a 1×1 convolutional layer. Finally, a 64×1×1 horizontal channel weight vector is generated using the Sigmoid activation function.

[0067] In a specific example, taking the horizontal feature extraction path as an example, the process of determining the horizontal spatial weight vector is as follows: Based on the horizontal channel enhanced feature map (64×256×256), channel average pooling (AVG) and channel max pooling (MAX) are performed in the channel dimension to obtain two 1×256×256 feature maps (first feature map and second feature map); these two feature maps are concatenated in the channel dimension to obtain a 2×256×256 joint feature map; spatial correlation features are extracted through a 7×7 large kernel convolutional layer (padding=3) to output a 1×256×256 feature map; then a 1×256×256 horizontal spatial weight vector is generated using the Sigmoid activation function.

[0068] The method described in this embodiment can enhance low-contrast features and improve the accuracy of detecting line defects. Experiments have shown that, under complex backgrounds with a signal-to-noise ratio of less than 3dB, the line defect detection rate is greater than or equal to 98%, and the false detection rate is less than or equal to 0.5%.

[0069] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, generating a defect distribution map corresponding to the target product based on the defect prediction map includes: The defect prediction map is binarized and segmented to obtain a binarized defect identification map corresponding to the image block; The binarized defect marker images are stitched together according to the cut positions of the corresponding image blocks to obtain a defect distribution map corresponding to the target product.

[0070] Binarization segmentation refers to converting the confidence (probability) of each pixel in the defect prediction map into two possible discrete values, such as using 1 (or 255) to indicate that the pixel has a line defect, and using 0 to indicate that the pixel is a background or a defect-free area.

[0071] Specifically, when cutting a grayscale image into N image blocks, the cutting positions (x_i, y_i) corresponding to each image block are recorded. For each image block Block_i, when the defect prediction model outputs the defect prediction map P_i corresponding to the image block, the local mean (mean_val) of the confidence level of each pixel in the defect prediction map P_i is calculated, and the segmentation threshold T_i is determined based on the local mean (mean_val). Each pixel (h, w) in the defect prediction map P_i is traversed. If the confidence level of the pixel is greater than the segmentation threshold T_i, the value of the pixel is set to 1 (or 255); otherwise, the value of the pixel is set to 0. After traversing the defect prediction map P_i, a binary defect identification map B_i corresponding to the image block Block_i is obtained. For each binary defect identification map B_i, the cutting position (x_i, y_i) corresponding to its image block Block_i is obtained. Each binary defect identification map B_i is then processed according to its corresponding cutting position (x_i, y_i). The image blocks (y_i) are stitched together; after stitching together the N image blocks, a defect distribution map is obtained. Following the method of this embodiment, the defect distribution map can be determined efficiently and conveniently, thereby improving the efficiency of line defect detection.

[0072] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, after performing binarization segmentation on the defect prediction map to obtain a binarized defect identification map corresponding to the image block, the method further includes: A morphological closing operation is performed on the binarized defect marker image to obtain the processed binarized defect marker image.

[0073] Among them, morphological closing operation refers to a composite operation in image processing technology that first dilates and then erodes.

[0074] In practical applications, a dilation operation is first performed on the binarized defect marker image: A structuring element S is determined, and the center of structuring element S is sequentially passed over each pixel in the binarized defect marker image B. If at least one pixel within the area covered by structuring element S has a value of "1" (indicating the presence of a line defect), then the pixel corresponding to the center of the structuring element in the output image is set to "1," resulting in the final image. Then, the same structuring element S is used to perform an erosion operation on the dilated final image: only when all pixels within the area covered by structuring element S have values ​​of "1" is the pixel corresponding to the center of the output image set to "1"; otherwise, it is set to "0," resulting in the processed binarized defect marker image.

[0075] According to the method of this embodiment, morphological closing operations are performed on the binarized defect marker image to fill in the predicted break points of line defects caused by noise, uneven lighting, or low contrast, so that slender scratches, cracks, and other line defects appear as continuous lines in the binary image, thereby improving the accuracy of detecting line defects.

[0076] Figure 4 This is a flowchart illustrating a training method for a defect detection model provided in an embodiment of this application. The defect detection model processes an input grayscale image of a target product to generate a defect prediction map of the target product. The defect prediction map is used to determine the line defect information of the target product. Figure 4 As shown, a training method for a defect detection model includes: S410: Obtain the training sample set; the training sample set includes product sample images and defect annotation images corresponding to the product sample images; S420: The target network structure is iteratively trained based on the training sample set until the target network structure converges to obtain the defect detection model. The target network structure includes a horizontal feature extraction path and a vertical feature extraction path, a feature fusion layer, and a prediction module. The horizontal and vertical feature extraction paths are used to extract the horizontal path features and vertical path features of the product sample image, respectively. The horizontal path features are used to characterize the defect features of the product sample image in the row direction, and the vertical path features are used to characterize the defect features of the product sample image in the column direction. The feature fusion layer is used to fuse the sample horizontal path features and sample vertical features to obtain a sample fusion feature map. The prediction module is used to output the corresponding sample defect prediction map based on the sample fusion feature map.

[0077] The training sample set includes a large number of product sample images and corresponding label information for each product sample image; the label information is the defect annotation map. In practical applications, the training sample set can be divided into a training set and a test set according to a preset ratio.

[0078] In practical applications, product sample images can also be preprocessed. Preprocessing operations include normalization and image enhancement operations, such as rotation, flipping, and adding noise, to increase sample diversity and improve model robustness.

[0079] The target network structure is determined, and initial values ​​are set for the model parameters (learnable parameters) within the target network structure. The target network structure includes horizontal and vertical feature extraction paths, a feature fusion layer, and a prediction module. The horizontal and vertical feature extraction paths are used to extract line defect features in the horizontal and vertical directions of the product sample image, respectively, resulting in sample horizontal path features and sample vertical path features. The feature fusion layer fuses the sample horizontal and vertical path features to obtain a sample fused feature map. The prediction module outputs the corresponding sample defect prediction map based on the sample fused feature map. In this embodiment, the target network structure can be a DualPathNet network; this embodiment is not limited to this.

[0080] Specifically, the learning and training process in one iteration is as follows: A batch of product sample images is input into the target network structure. The product sample images pass through the dual-path feature extraction module of the target network structure in sequence, namely the horizontal feature extraction path and the vertical feature extraction path, the feature fusion layer and the prediction module, and output the corresponding sample defect prediction map.

[0081] Then, the sample defect prediction map is compared with the preset defect annotation map, and the loss value is calculated using the loss function. The gradient value of the loss value with respect to the model parameters in the target network structure is calculated using the backpropagation algorithm. To improve training stability, a gradient accumulation strategy (accumulation steps of 2) is adopted, that is, two consecutive forward-backward propagations are performed, the gradient is accumulated, and then a parameter update is performed to simulate the effect of training a larger batch.

[0082] The optimizer updates the model parameters based on the gradient values, which is equivalent to updating the target network architecture. Specifically, the AdamW optimizer (initial learning rate 1e-4, weight decay 1e-4) can be used to update the model parameters based on the accumulated gradient values, thereby completing an adjustment to the target network structure. The weight decay in the AdamW optimizer effectively suppresses overfitting.

[0083] At the end of each iteration of learning and training, it is determined whether the target network structure has converged; if it has converged, the current target network structure is output as a defect detection model; if it has not converged, the process continues to the next iteration of learning and training.

[0084] Throughout the iterative learning and training process, the ReduceLROnPlateau learning rate scheduling strategy is employed: during the validation phase, the loss value on the validation set is monitored, and if this loss value does not decrease for five consecutive training epochs, the system automatically reduces the learning rate to fine-tune the model parameters. Training is performed on a GPU, and cuDNN automatic optimization is enabled to accelerate convolution operations. During the validation phase, gradient computation is disabled using the torch.inference_mode() context manager to reduce memory usage and improve validation efficiency.

[0085] Repeat this iterative process until the target network structure converges, and finally obtain the trained defect detection model.

[0086] The defect detection model is trained according to the method of this embodiment. The model then extracts horizontal and vertical path features from the product sample image. The horizontal path features characterize defects in the row direction, while the vertical path features characterize defects in the column direction. These features are then fused to obtain a fused feature map, which is used to output a corresponding defect prediction map. As can be seen, by extracting defect features from both horizontal and vertical directions and fusing the horizontal and vertical path features, the defect detection model can more comprehensively learn the characteristics of line defects in the product sample image. Furthermore, by fusing the horizontal and vertical path features, it can effectively identify diagonal line defects. Therefore, this method can improve the accuracy of detecting line defects in products.

[0087] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, the target network structure further includes a first prediction branch and a second prediction branch. The first prediction branch is used to predict defects in the sample fusion feature map to obtain the first-scale prediction map of the sample. Upsample the first-scale prediction map of the sample to obtain the prediction map of the sample at the preset size; The second prediction branch is used to predict defects in the sample's preset size prediction map to obtain the sample's second-scale prediction map; the second detection scale of the second-scale prediction map is greater than the first detection scale of the first-scale prediction map.

[0088] The first prediction branch (output head 1) is a network branch that uses the first detection scale for defect prediction, and the first-scale prediction map refers to the sample prediction map corresponding to the first prediction branch (output head 1). The second prediction branch (output head 2) is a network branch that uses the second detection scale for defect prediction; the second-scale prediction map refers to the sample prediction map corresponding to the second prediction branch (output head 2).

[0089] In this embodiment, after using the first prediction branch to predict defects in the fused feature map and obtaining a first-scale prediction map of the sample, the first-scale prediction map is directly used for feature fusion, and simultaneously upsampled to obtain a preset-size prediction map of the sample. Then, the second prediction branch is used to predict defects in the preset-size prediction map of the sample to obtain a second-scale prediction map of the sample. Therefore, in this embodiment, the second detection scale of the second-scale prediction map of the sample is larger than the first detection scale of the first-scale prediction map of the sample. In a specific example, the first detection scale of the first-scale prediction map of the sample is 1×512×512, and the second detection scale of the second-scale prediction map of the sample is 1×1024×1024.

[0090] It should be noted that the first prediction branch (output head1) includes: The first convolutional layer (Cov) uses a 3×3 convolutional kernel with padding=1, increasing the number of input channels from 128 to 256 to enhance the expressive power of the fused features. Up-Sample: The bilinear interpolation method is used, and the scaling factor is set to 2 to restore the feature map size to the default 256×512×512 (i.e., H2=W2=512). The second convolutional layer (Cov) uses a 1×1 convolutional kernel to reduce the number of input channels from 256 to 1, thereby outputting a single-channel prediction map for the first detection scale with a size of 1×512×512.

[0091] The second prediction branch (output head2) includes: Up-Sample: Employs bilinear interpolation with a scaling factor of 1, and is used to receive the first-scale prediction map output from the first prediction branch. Convolutional layer (Cov): Uses a 3×3 convolutional kernel, sets padding=1, and has 1 input and 1 output channel. It smooths the first-scale prediction map of the input to integrate contextual information. Dynamic upsampling layer: During the training phase, it is dynamically adjusted to adjust the output size of the feature map to 1×1024×1024 through interpolation operations, thereby obtaining the second-scale prediction map.

[0092] In this embodiment, the design of the multi-scale prediction branch can take into account line defects of different scales in the image patch, such as short line defects, medium line defects, and long line defects. In practical applications, the detection rate for line defects with a length ≥ 5 pixels is ≥ 95%. Therefore, this method can avoid missing short line defects and preventing long line defects from breaking, thus improving the accuracy of line defect detection.

[0093] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, the target network structure is iteratively trained based on the training sample set until the target network structure converges to obtain the defect detection model, including: Input the product sample image into the current target network structure to obtain the defect prediction probability map output by the target network structure; The first loss value and the second loss value of the current target network structure are calculated based on the defect prediction probability map and the corresponding defect annotation map, and the mixed loss value is determined based on the first loss value and the second loss value. The first loss value is used to reduce the training bias caused by the imbalance of the number of positive and negative samples in the training sample set, and to reduce the weight contribution of easily distinguishable samples in the loss calculation. The second loss value represents the degree of overlap between the prediction probability map and the defect annotation map. Determine whether the current target network structure has achieved the training objective based on the mixed loss value or the number of training iterations; If the training objective is achieved, the current target network structure is determined to have converged, and the current target network structure is output as a defect detection model. If the training objective is not achieved, the model parameters of the current target network structure are adjusted using the mixed loss value, the current target network structure is updated, and the process returns to the step of inputting the product sample image into the target network structure to obtain the defect prediction probability map output by the target network structure, as well as subsequent steps.

[0094] Specifically, after inputting the product sample image into the current target network structure, the current target network structure outputs a defect prediction probability map corresponding to the product sample image, and calculates the first loss value and the second loss value respectively.

[0095] It should be noted that the first loss value is used to reduce the training bias caused by the imbalance of positive and negative samples in the training sample set, and to reduce the weight contribution of easily distinguishable samples in the loss calculation.

[0096] Specifically, in defect detection, defect pixels (positive samples) are typically far fewer than background pixels (negative samples). This imbalance causes the conventional loss function to be dominated by a massive number of negative samples, making model training prone to bias—that is, a tendency to predict all pixels as background to reduce the loss. The first loss value is adjusted by introducing a weight parameter α, which assigns higher weight to the scarce positive samples when calculating the loss, thereby balancing their influence in gradient updates and forcing the model to also learn to recognize defects.

[0097] Furthermore, high-contrast, clear defect edges are easily classified samples, and the model can efficiently and quickly learn the features of easily classified samples; low-contrast, blurry, and small defects are difficult to classify samples, and the model has a relatively greater difficulty in learning the features of difficult-to-classify samples. In actual training, the model will tend to train on easily classified samples. The first loss value, through the focusing parameter γ and the modulation factor (1-pt)^γ, dynamically reduces the loss value of pixels that have been correctly classified by the model with high confidence (easily classified samples), thereby forcing the model to focus its optimization on the difficult samples that it has not yet learned.

[0098] In a specific instance, the formula for calculating the first loss (Focal Loss) is: Focal Loss= α*(1 pt)^γ*log(pt); Here, α is the weighting factor for balancing positive and negative samples. Usually, a larger weight (e.g., 0.8) is set for positive samples to alleviate the imbalance between the number of positive and negative samples. pt represents the predicted probability; for a defective pixel (positive sample), pt is the probability p that the model predicts it to be a defect; for a background pixel (negative sample), pt is the probability 1-p that the model predicts it to be background.

[0099] γ is a focusing parameter used to adjust the degree of suppression of easily separated samples. The larger γ is, the stronger the suppression of easily separated samples and the stronger the focus on difficult-to-separate samples. γ is generally set to 2.

[0100] (1 pt)^γ is the modulation factor. When the product sample image is an easily separable sample, the prediction probability pt will approach 1, (1 pt)^γ will approach 0, thus significantly suppressing the loss contribution of the product sample image; when the product sample image is a difficult sample to distinguish, the prediction probability pt is low, (1 pt)^γ Jiaotong University, thereby preserving or amplifying the loss contribution of the product sample image.

[0101] The second loss value represents the overlap between the predicted probability map and the defect annotation map, and is used to improve the segmentation accuracy of the defect region. Specifically, the defect prediction map and the defect annotation map are treated as two sets, and the proportion of the overlapping area of ​​the defect prediction map and the defect annotation map to the total area is calculated. The lower the proportion (the worse the overlap), the larger the loss value, and the second loss value is determined.

[0102] In a specific example, the formula for calculating the second loss (Dice Loss) based on the intersection-union ratio (IUU) of the defect prediction map and the defect annotation map is as follows: Dice Loss=1 (2* |pred |target |+) ) / (∣pred∣+∣target∣+ ); Wherein, pred represents the sample defect prediction map, and |pred| represents the sum of all pixel values ​​in the defect prediction map; target represents the defect annotation map, and |target| represents the sum of all pixel values ​​in the defect annotation map; |pred∩target| represents the intersection between the defect prediction map and the defect annotation map; |pred| + |target| represents the sum of all pixel values ​​in the defect prediction map and the sum of all pixel values ​​in the defect annotation map. =1e 6 (Avoid denominators of 0).

[0103] After determining the first loss value and the second loss value, the first loss value and the second loss value are added together or weighted summed to obtain the total loss value.

[0104] In a specific instance, Total Loss = Focal Loss + Dice Loss.

[0105] Then, determine whether the mixed loss value has stopped decreasing and has stabilized on the validation set, or whether the number of training iterations has reached the preset number; if so, it means that the current target network model has reached the training objective, the current target network structure has converged, and the current target network structure is output as a defect detection model; otherwise, it means that the current target network model has not reached the training objective, then the model parameters of the current target network structure are adjusted using the mixed loss value, the current target network structure is updated, and the process returns to the step of inputting the product sample image into the target network structure to obtain the defect prediction probability map output by the target network structure and subsequent steps.

[0106] According to the method of this embodiment, a mixed loss value is calculated using a first loss value and a second loss value, and the mixed loss value is used to solve the category imbalance problem of low percentage of defective pixels.

[0107] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0108] It should be noted that the information collection process (such as the facial image collection process, fingerprint information collection process, etc.) / feature extraction process involved in this application is carried out with the user's knowledge and permission. That is, the information collection process / feature extraction process complies with the requirements of laws and regulations and does not constitute an act that harms the public interest.

[0109] Figure 5 The diagram shown is a structural schematic of a line defect detection device provided in an embodiment of this application. Figure 5 As shown, a line defect detection device according to this embodiment includes an acquisition module 510, a detection module 520, an execution module 530, and an output module 540; wherein, The acquisition module 510 is used to acquire a grayscale image of the target product and cut the grayscale image into multiple image blocks; The detection module 520 is used to input each image block into a pre-trained defect detection model, extract the horizontal path features and vertical path features of the image blocks respectively using the defect detection model, fuse the horizontal path features and vertical path features to obtain a fused feature map, and output a defect prediction map corresponding to each image block based on the fused feature map; the horizontal path features are used to characterize the defect features of the image block in the row direction, and the vertical path features are used to characterize the defect features of the image block in the column direction. Execution module 530 is used to generate a defect distribution map corresponding to the target product based on the defect prediction map; Output module 540 is used to determine line defect information in the target product based on the defect distribution map.

[0110] The line defect detection device provided in this application embodiment has the same beneficial effects as the line defect detection method described above.

[0111] In one embodiment, the detection module 520 includes: The first prediction submodule is used to perform defect prediction on the fused feature map using the first prediction branch to obtain the first scale prediction map. The upsampling submodule is used to upsample the first-scale prediction map to obtain a prediction map of a preset size. The second prediction submodule is used to perform defect prediction on the preset size prediction map using the second prediction branch to obtain the second scale prediction map; the second detection scale of the second scale prediction map is greater than the first detection scale of the first scale prediction map. The feature fusion submodule is used to fuse the first-scale prediction map and the second-scale prediction map to obtain the defect prediction map corresponding to the image patch.

[0112] In one embodiment, the detection module 520 includes: The detection submodule is used to determine the first intermediate feature map of the image block in the horizontal direction and the second intermediate feature map of the image block in the vertical direction using the defect detection model. The first calculation submodule is used to perform channel attention and spatial attention calculations on the first intermediate feature map to obtain a horizontal enhancement feature map, and to determine the horizontal path features of the image block based on the horizontal enhancement feature map; The second calculation submodule is used to perform channel attention and spatial attention calculations on the second intermediate feature map to obtain a vertical enhancement feature map, and to determine the vertical path features of the image block based on the vertical enhancement feature map.

[0113] In one embodiment, the execution module 530 includes: The processing submodule is used to perform binarization segmentation on the defect prediction map to obtain a binarized defect identification map corresponding to the image block; The stitching submodule is used to stitch together the binarized defect identification images according to the cut positions of the corresponding image blocks to obtain a defect distribution map corresponding to the target product.

[0114] In one embodiment, a line defect detection device further includes: The operation submodule is used to perform morphological closing operations on the binarized defect marker image to obtain the processed binarized defect marker image.

[0115] Figure 6 The diagram shown is a structural schematic of a training device for a defect detection model provided in an embodiment of this application. Figure 6 As shown, the training device for a defect detection model in this embodiment is used to process the grayscale image of the input target product to generate a defect prediction map of the target product. The defect prediction map is used to determine the line defect information of the target product. The device includes a sample acquisition module 610 and a learning and training module 620. The sample acquisition module 610 is used to acquire a training sample set; the training sample set includes product sample images and defect annotation images corresponding to the product sample images. The learning and training module 620 is used to iteratively learn and train the target network structure based on the training sample set until the target network structure converges to obtain the defect detection model. The target network structure includes a horizontal feature extraction path, a vertical feature extraction path, a feature fusion layer, and a prediction module. The horizontal and vertical feature extraction paths are used to extract the sample horizontal path features and sample vertical path features of the product sample image, respectively. The sample horizontal path features are used to characterize the defect features of the product sample image in the row direction, and the sample vertical path features are used to characterize the defect features of the product sample image in the column direction. The feature fusion layer is used to fuse the sample horizontal path features and sample vertical path features to obtain a sample fusion feature map. The prediction module is used to output the corresponding sample defect prediction map based on the sample fusion feature map.

[0116] In one embodiment, the target network structure further includes a first prediction branch and a second prediction branch; The first prediction branch is used to predict defects in the sample fusion feature map to obtain the first-scale prediction map of the sample. Upsample the first-scale prediction map of the sample to obtain the prediction map of the sample at the preset size; The second prediction branch is used to predict defects in the sample's preset size prediction map to obtain the sample's second-scale prediction map; the second detection scale of the sample's second-scale prediction map is greater than the first detection scale of the sample's first-scale prediction map.

[0117] In one embodiment, the learning and training module 620 is further configured to: Input the product sample image into the current target network structure to obtain the defect prediction probability map output by the target network structure; The first loss value and the second loss value of the current target network structure are calculated based on the defect prediction probability map and the corresponding defect annotation map, and the mixed loss value is determined based on the first loss value and the second loss value. The first loss value is used to reduce the training bias caused by the imbalance of the number of positive and negative samples in the training sample set, and to reduce the weight contribution of easily distinguishable samples in the loss calculation. The second loss value represents the degree of overlap between the defect prediction probability map and the defect annotation map. Determine whether the current target network structure has achieved the training objective based on the mixed loss value or the number of training iterations; If the training objective is achieved, the current target network structure is determined to have converged, and the current target network structure is output as a defect detection model. If the training objective is not achieved, the model parameters of the current target network structure are adjusted using the mixed loss value, the current target network structure is updated, and the process returns to the step of inputting the product sample image into the target network structure to obtain the defect prediction probability map output by the target network structure, as well as subsequent steps.

[0118] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0119] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0120] Figure 7 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Figure 7 As shown, the terminal device 700 of this embodiment includes a memory 710, a processor 720, and a computer program 730 stored in the memory 710 and executable on the processor 720; when the processor 720 executes the computer program 730, it implements the steps in the embodiments of the above-mentioned methods for detecting line defects or training methods for defect detection models; or when the processor 720 executes the computer program 730, it implements the functions of each module / unit in the above-mentioned device embodiments.

[0121] For example, computer program 730 can be divided into one or more modules / units, one or more of which are stored in memory 710 and executed by processor 720 to implement the method of the embodiments of this application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of computer program 730 in terminal device 700. For example, computer program 730 can be divided into an acquisition module, a detection module, an execution module, and an output module, with the specific functions of each module as follows: The acquisition module is used to acquire a grayscale image of the target product and cut the grayscale image into multiple image blocks; The detection module is used to input each of the image blocks into a pre-trained defect detection model, extract the horizontal path features and vertical path features of the image blocks using the defect detection model, fuse the horizontal path features and vertical path features to obtain a fused feature map, and output a defect prediction map corresponding to each of the image blocks based on the fused feature map; the horizontal path features are used to characterize the defect features of the image blocks in the row direction, and the vertical path features are used to characterize the defect features of the image blocks in the column direction; The execution module is used to generate a defect distribution map corresponding to the target product based on the defect prediction map; The output module is used to determine the line defect information in the target product based on the defect distribution map.

[0122] The line defect detection device provided in this application embodiment has the same beneficial effects as the line defect detection method described above.

[0123] In applications, terminal device 700 can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. Terminal device 700 may include, but is not limited to, memory 710 and processor 720. Those skilled in the art will understand that... Figure 7 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown, or combine certain components, or different components. For example, a terminal device may also include input / output devices, network access devices, buses, etc.; among which, input / output devices may include cameras, audio acquisition / playback devices, displays, etc.; network access devices may include communication modules for wireless communication with external devices.

[0124] In applications, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0125] In applications, memory can be an internal storage unit of a terminal device, such as its hard drive or RAM; it can also be an external storage device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card; or it can include both internal and external storage units. Memory is used to store operating systems, applications, boot loaders, data, and other programs, such as computer program code. Memory can also be used to temporarily store data that has been output or will be output.

[0126] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described method embodiments.

[0127] This application implements all or part of the processes in the methods of the above embodiments, which can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, such as a USB flash drive, a portable hard drive, a magnetic disk, or an optical disk.

[0128] The computer-readable storage medium provided in this application embodiment has the same beneficial effects as the above-described method for detecting line defects or training a defect detection model.

[0129] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps in the various method embodiments described above.

[0130] The computer program product provided in this application embodiment has the same beneficial effects as the above-described method for detecting line defects or training a defect detection model.

[0131] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0132] Those skilled in the art will recognize that the device and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0133] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interface, or the device may be indirectly coupled or communicated, and may be electrical, mechanical, or other forms.

[0134] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for detecting line defects, characterized in that, The method includes: Obtain a grayscale image of the target product and cut the grayscale image into multiple image blocks; Each image block is input into a pre-trained defect detection model. The defect detection model extracts the horizontal and vertical path features of each image block. The horizontal and vertical path features are then fused to obtain a fused feature map. Based on the fused feature map, a defect prediction map corresponding to each image block is output. The horizontal path features are used to characterize the defect features of the image block in the row direction, and the vertical path features are used to characterize the defect features of the image block in the column direction. Generate a defect distribution map corresponding to the target product based on the defect prediction map; Based on the defect distribution map, the line defect information in the target product is determined.

2. The method according to claim 1, characterized in that, The step of outputting defect prediction maps corresponding to each of the image blocks based on the fused feature maps includes: The first prediction branch is used to predict defects in the fused feature map to obtain a first-scale prediction map. Upsample the first-scale prediction map to obtain a preset-size prediction map; The second prediction branch is used to predict defects in the preset size prediction map to obtain a second-scale prediction map; the second detection scale of the second-scale prediction map is larger than the first detection scale of the first-scale prediction map. The first-scale prediction map and the second-scale prediction map are fused to obtain a defect prediction map corresponding to the image block.

3. The method according to claim 1, characterized in that, The step of extracting the horizontal and vertical path features of the image patch using the defect detection model includes: The defect detection model is used to determine a first intermediate feature map of the image block in the horizontal direction and a second intermediate feature map of the image block in the vertical direction. Channel attention and spatial attention calculations are performed on the first intermediate feature map to obtain a horizontally enhanced feature map, and the horizontal path features of the image block are determined based on the horizontally enhanced feature map. Channel attention and spatial attention calculations are performed on the second intermediate feature map to obtain a vertical enhancement feature map, and the vertical path features of the image block are determined based on the vertical enhancement feature map.

4. The method according to claim 1, characterized in that, The step of generating a defect distribution map corresponding to the target product based on the defect prediction map includes: The defect prediction map is binarized and segmented to obtain a binarized defect identification map corresponding to the image block; The binarized defect identification images are stitched together according to the cut positions of the corresponding image blocks to obtain a defect distribution map corresponding to the target product.

5. The method according to claim 4, characterized in that, After performing binarization segmentation on the defect prediction map to obtain a binarized defect identification map corresponding to the image patch, the method further includes: A morphological closing operation is performed on the binarized defect marker image to obtain a processed binarized defect marker image.

6. A training method for a defect detection model, characterized in that, The defect detection model is used to process the grayscale image of the input target product to generate a defect prediction map of the target product, and the defect prediction map is used to determine the line defect information of the target product. The training method includes: Obtain a training sample set; the training sample set includes product sample images and defect annotation images corresponding to the product sample images; The target network structure is iteratively trained based on the training sample set until it converges, thus obtaining the defect detection model. The target network structure includes a horizontal feature extraction path, a vertical feature extraction path, a feature fusion layer, and a prediction module. The horizontal and vertical feature extraction paths are used to extract the horizontal and vertical path features of the product sample image, respectively. The horizontal path features characterize the defect features of the product sample image in the row direction, and the vertical path features characterize the defect features of the product sample image in the column direction. The feature fusion layer fuses the horizontal and vertical path features to obtain a fused feature map. The prediction module outputs a corresponding defect prediction map based on the fused feature map.

7. The method according to claim 6, characterized in that, The target network structure also includes a first prediction branch and a second prediction branch; The first prediction branch is used to predict defects in the sample fusion feature map to obtain a first-scale prediction map of the sample. Upsample the first-scale prediction map of the sample to obtain a preset-size prediction map of the sample; The second prediction branch is used to predict defects in the sample preset size prediction map to obtain a sample second-scale prediction map; the second detection scale of the sample second-scale prediction map is greater than the first detection scale of the sample first-scale prediction map.

8. The method according to claim 6, characterized in that, The step of iteratively learning and training the target network structure based on the training sample set until the target network structure converges to obtain the defect detection model includes: The product sample image is input into the current target network structure to obtain the defect prediction probability map output by the target network structure; The first loss value and the second loss value of the current target network structure are calculated based on the defect prediction probability map and the corresponding defect annotation map, and a mixed loss value is determined based on the first loss value and the second loss value. The first loss value is used to reduce the training bias caused by the imbalance of the number of positive and negative samples in the training sample set, and to reduce the weight contribution of easily distinguishable samples in the loss calculation. The second loss value represents the degree of overlap between the defect prediction probability map and the defect annotation map. Determine whether the current target network structure has achieved the training objective based on the hybrid loss value or the number of training iterations. If the training objective is achieved, the current target network structure is determined to have converged, and the current target network structure is output as the defect detection model. If the training objective is not achieved, the model parameters of the current target network structure are adjusted using the mixed loss value, the current target network structure is updated, and the process returns to the step of inputting the product sample image into the target network structure to obtain the defect prediction probability map output by the target network structure, and subsequent steps.

9. A device for detecting line defects, characterized in that, The device includes: The acquisition module is used to acquire a grayscale image of the target product and cut the grayscale image into multiple image blocks; The detection module is used to input each of the image blocks into a pre-trained defect detection model, extract the horizontal path features and vertical path features of the image blocks using the defect detection model, fuse the horizontal path features and vertical path features to obtain a fused feature map, and output a defect prediction map corresponding to each of the image blocks based on the fused feature map; the horizontal path features are used to characterize the defect features of the image blocks in the row direction, and the vertical path features are used to characterize the defect features of the image blocks in the column direction; The execution module is used to generate a defect distribution map corresponding to the target product based on the defect prediction map; The output module is used to determine the line defect information in the target product based on the defect distribution map.

10. A training device for a defect detection model, characterized in that, The defect detection model is used to process the input grayscale image of the target product to generate a defect prediction map of the target product, and the defect prediction map is used to determine the line defect information of the target product; the device includes: The sample acquisition module is used to acquire a training sample set; the training sample set includes product sample images and defect annotation images corresponding to the product sample images; The learning and training module is used to iteratively learn and train the target network structure based on the training sample set until the target network structure converges to obtain the defect detection model. The target network structure includes a horizontal feature extraction path, a vertical feature extraction path, a feature fusion layer, and a prediction module. The horizontal feature extraction path and the vertical feature extraction path are used to extract the sample horizontal path features and sample vertical path features of the product sample image, respectively. The sample horizontal path features are used to characterize the defect features of the product sample image in the row direction, and the sample vertical path features are used to characterize the defect features of the product sample image in the column direction. The feature fusion layer is used to fuse the sample horizontal path features and the sample vertical features to obtain a sample fusion feature map. The prediction module is used to output the corresponding sample defect prediction map based on the sample fusion feature map.

11. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5 or 6 to 8.

12. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5 or 6 to 8.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5 or 6 to 8.

Images