Metal structure surface defect image semantic segmentation method based on deep learning

Abstract

The application relates to the technical field of semantic segmentation, and discloses a metal structural part surface defect image semantic segmentation method based on deep learning, which comprises the following steps: pre-processing an original gray image to obtain an enhanced surface image; inputting a multi-scale encoder network to extract shallow and deep feature maps; extracting local strike direction information based on the first shallow feature map, analyzing and generating an anisotropic convolution kernel matched with a defect strike, performing direction adaptive convolution enhancement to obtain a direction enhanced feature map and a direction response map; using the direction response map to perform semantic correction on the deep feature map, and performing channel weighted fusion on the direction enhanced feature map to obtain a fusion feature map; and inputting the segmentation decoder to output a binary segmentation mask. Through the analytical expression embedding of the defect strike anisotropy physical prior, high-precision semantic segmentation of linear defects is realized.

Description

Technical Field

[0001] This invention relates to the field of semantic segmentation technology, and in particular to a semantic segmentation method for surface defect images of metal structural parts based on deep learning. Background Technology

[0002] Surface defect detection of metal structural components is a key link in industrial manufacturing quality control. Linear defects such as cracks and scratches seriously affect the fatigue life and safety of structural components. Existing semantic segmentation methods based on deep learning mainly adopt an encoder-decoder structure, which trains the network through a large amount of labeled data to achieve pixel-level localization of defect areas. However, existing methods have the following drawbacks: standard convolutional kernels have isotropic square receptive fields and cannot adaptively adjust the shape and orientation of the convolutional kernel according to the local orientation of linear defects. This results in discontinuities and poor continuity in the segmentation results of linear defects such as cracks and scratches, and defects in areas with drastic changes in orientation are easily missed.

[0003] Existing cross-layer feature fusion methods employ element-wise addition or channel concatenation followed by convolution, failing to demonstrate semantic consistency in the defect orientation across feature maps at different scales. Furthermore, severe loss of high-frequency detail occurs during upsampling, leading to blurred defect edge segmentation. Therefore, improving the continuity and edge accuracy of semantic segmentation for linear defects has become a pressing issue. Summary of the Invention

[0004] This invention provides a semantic segmentation method for surface defect images of metal structural components based on deep learning, in order to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides a deep learning-based semantic segmentation method for surface defect images of metal structural components, comprising:

[0006] S1, acquire the original grayscale image of the surface area of ​​the metal structural component, preprocess the original grayscale image to obtain the enhanced surface image;

[0007] S2, The enhanced surface image is input into a pre-trained multi-scale encoder network to extract a shallow feature map set and a deep feature map. The shallow feature map set consists of shallow feature maps that preserve spatial location details and orientation details.

[0008] S3. Based on the first shallow feature map in the shallow feature map set, extract the local orientation information of each spatial location, parse and generate an anisotropic convolution kernel that matches the defect orientation according to the orientation information of the local orientation, and use the anisotropic convolution kernel to perform spatially adaptive orientation convolution enhancement on the first shallow feature map to obtain the orientation enhancement feature map, and retain the orientation response information to generate the orientation response map.

[0009] S4, use the directional response map to perform directional-guided semantic correction on the deep feature map to obtain a semantically corrected deep feature map, and perform channel-weighted fusion on the directional enhancement feature map and the semantically corrected deep feature map to obtain a fused feature map;

[0010] S5, the fused feature map is input into the segmentation decoder, and a binary segmentation mask with the same size as the original grayscale image is output to complete the semantic segmentation of surface defects of metal structural parts.

[0011] Compared with the prior art, the present invention has the following beneficial effects:

[0012] 1. This invention extracts the local orientation information of each spatial location in the shallow feature map, analyzes and generates anisotropic convolution kernels that match the defect orientation, and performs spatially adaptive directional convolution enhancement. Unlike existing data-driven methods that rely on large amounts of data to allow the network to implicitly learn defect features, this invention uses the anisotropic geometric and physical prior of the defect orientation to analytically model and embed it into the feature extraction process of the network. This allows the network to actively perceive the geometric features of the defect orientation during inference, fundamentally solving the problems of breakpoints and missed detections caused by the mismatch between standard square convolution and the geometric shape of linear defects. This significantly improves the pixel-level segmentation continuity of linear defects such as cracks and scratches, without requiring additional orientation information annotation.

[0013] 2. This invention utilizes directional response maps to perform directional-guided semantic correction on deep feature maps, and introduces a directional consistency loss function to constrain the consistency of spatial distribution between deep semantic responses and shallow directional responses, before performing channel-weighted fusion. Unlike existing methods that indiscriminately fuse features across all scales, this invention solves the problem of blurred defect edges caused by the loss of directional information in multi-scale fusion by explicitly transferring and constraining directional prior information across cross-scale features, thereby improving the boundary integrity of defect segmentation masks and the segmentation robustness in scenarios where defects coexist at multiple scales. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a deep learning-based semantic segmentation method for surface defect images of metal structural components, provided in an embodiment of the present invention.

[0015] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0016] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0017] This application provides a deep learning-based semantic segmentation method for surface defect images of metal structural components. The execution entity of this deep learning-based semantic segmentation method for surface defect images of metal structural components includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the deep learning-based semantic segmentation method for surface defect images of metal structural components can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0018] Reference Figure 1 The diagram shown is a flowchart illustrating a deep learning-based semantic segmentation method for surface defect images of metal structural components according to an embodiment of the present invention. In this embodiment, the deep learning-based semantic segmentation method for surface defect images of metal structural components includes:

[0019] S1, acquire the original grayscale image of the surface area of ​​the metal structural component, preprocess the original grayscale image to obtain the enhanced surface image;

[0020] In this embodiment of the invention, the step of acquiring the original grayscale image of the surface area of ​​the metal structural component and preprocessing the original grayscale image to obtain the enhanced surface image includes:

[0021] Acquire the original grayscale image of the surface area of ​​the metal structural component;

[0022] The original grayscale image is subjected to median filtering for denoising to obtain a denoised grayscale image.

[0023] The denoised grayscale image is subjected to contrast-limited adaptive histogram equalization. The denoised grayscale image is divided into a preset number of local sub-blocks. Histogram equalization is performed on each local sub-block, and the local contrast enhancement is limited to obtain an enhanced surface image.

[0024] It should be noted that during the process of acquiring the original grayscale image of the surface area of ​​the metal structural component, the surface of the metal structural component is easily affected by defects such as scratches, indentations, cracks, and pits during industrial manufacturing. However, there are also interference factors such as uneven lighting, oxidation spots, and processing textures. Therefore, it is necessary to preprocess the original grayscale image to eliminate noise interference and enhance the contrast of the defect area.

[0025] It should be noted that the original grayscale image is subjected to median filtering for noise reduction. Median filtering is a non-linear filtering method. Its principle is to replace the grayscale value of the center pixel with the median of the grayscale values ​​of all pixels in the neighborhood. The filtering window size of the median filter is set to 3×3. This size can effectively eliminate the salt-and-pepper noise generated during the acquisition process while maximizing the preservation of the sharpness of the defect edges and directional detail information.

[0026] It should be noted that the contrast-limited adaptive histogram equalization process performed on the denoised grayscale image first divides the denoised grayscale image into 64 local sub-blocks in 8 rows and 8 columns. The size of each local sub-block is determined by dividing the total image size by 8. Histogram equalization is then performed on each local sub-block, that is, the cumulative distribution function of gray levels within the sub-block is used as a mapping function to remap the pixel grayscale values, making the grayscale distribution within the sub-block more uniform, thereby enhancing local contrast. At the same time, to avoid the problem of over-enhancement and amplified noise in uniform areas, the local contrast enhancement is limited, with a limiting threshold set to 3. When the frequency of a certain grayscale level in the histogram exceeds the limiting threshold, the excess part is cropped and evenly distributed among all grayscale levels. After processing, the grayscale difference between the defective area and the normal surface area is significantly enhanced, while the noise in the background area is effectively suppressed, resulting in an enhanced surface image.

[0027] Furthermore, the core advantage of contrast-limited adaptive histogram equalization lies in its local adaptability. The lighting conditions of different areas on the surface of a metal structure may differ, such as darker lighting in the edge area and brighter lighting in the center area. Global histogram equalization will cause noise in dark areas to be excessively amplified and details in bright areas to be compressed.

[0028] By employing a block-based local processing approach, each sub-block is independently equalized, ensuring that the contrast of each local region in the image is reasonably enhanced. Furthermore, the amplitude limiting mechanism effectively avoids excessive noise enhancement in uniform regions. The combination of contrast-limited adaptive histogram equalization and median filtering enhances the grayscale contrast between defect and background regions while eliminating noise, providing high-quality input data for feature extraction in the multi-scale encoder network. This fundamentally guarantees the segmentation network's ability to capture small defects with weak contrast. This preprocessing strategy embodies the analytical design concept of physical priors, selectively choosing filtering methods and enhancement strategies based on the type of noise on the metal surface and the contrast characteristics of the defects, rather than using a general, indiscriminate data augmentation method.

[0029] S2, The enhanced surface image is input into a pre-trained multi-scale encoder network to extract a shallow feature map set and a deep feature map. The shallow feature map set consists of shallow feature maps that preserve spatial location details and orientation details.

[0030] In this embodiment of the invention, the step of inputting the enhanced surface image into a pre-trained multi-scale encoder network to extract shallow feature map sets and deep feature maps includes:

[0031] The enhanced surface image is input into a pre-trained multi-scale encoder network. The feature map output from the first stage is selected as the first shallow feature map, and the feature map output from the second stage is selected as the second shallow feature map. The first shallow feature map and the second shallow feature map together constitute a shallow feature map set, which is used to preserve fine spatial location information and orientation details.

[0032] The feature map output from the fourth stage is selected as the deep feature map, which is used to preserve high-level semantic information.

[0033] It should be noted that the spatial size of the first-stage output feature map is half that of the original image. This feature map has the smallest receptive field, and the image region corresponding to each pixel is extremely small. Therefore, it retains the finest spatial location information and directional details, including the pixel-level direction of cracks and the edge gradient direction of scratches. The spatial size of the second-stage output feature map is one-quarter that of the original image. While retaining a certain spatial resolution, it begins to aggregate local region information. The third and fourth stages output feature maps of one-eighth and one-sixteenth the size of the original image, respectively. The spatial resolution gradually decreases, while the semantic information gradually increases.

[0034] It should be noted that the selective extraction of the first shallow feature map, the second shallow feature map, and the deep feature map output from the fourth stage, excluding the one-eighth-sized feature map output from the third stage, is based on the analytical analysis of the geometric and physical properties of metal surface defects. The receptive field of the feature map output from the third stage is already relatively large, with the image area corresponding to a single pixel being approximately 40 pixels × 40 pixels or larger than the original image. At this point, directional details have been largely lost due to pooling, but the semantic discrimination ability is not as good as that of the deep feature map from the fourth stage. It is in an intermediate state where directional information is ambiguous and semantic information is not strong enough. Retaining this feature map not only increases the computational cost of subsequent steps, but more importantly, it introduces noise features with blurred orientation information, interfering with the orientation extraction accuracy of subsequent anisotropic convolution kernels. Therefore, the shallow feature map set is explicitly limited to the first shallow feature map and the second shallow feature map. This selective design reflects the core idea of ​​the present invention—based on the physical prior of defect orientation, it analytically determines which scales of features are most effective for orientation extraction, rather than relying on a large amount of data stacking to allow the network to filter through implicit learning. This fundamentally avoids the common problems of invalid feature redundancy and orientation noise pollution in data-driven methods.

[0035] In this embodiment of the invention, the downsampling stage of the multi-scale encoder network includes:

[0036] The multi-scale encoder network contains four sequentially connected downsampling stages. Each downsampling stage consists of a convolutional layer, a batch normalization layer, a ReLU activation function, and a max pooling layer connected in sequence, which respectively output feature maps with spatial sizes of one-half, one-quarter, one-eighth, and one-sixteenth of the enhanced surface image.

[0037] It should be noted that the convolutional layer in each downsampling stage is used to extract local features. It performs sliding convolution operation on the input feature map through learnable convolutional kernels to capture texture and edge information in the local neighborhood. The first stage uses 64 3×3 convolutional kernels with a stride of 1 and 64 output channels.

[0038] The second stage uses 128 3×3 convolutional kernels with a stride of 1 and 128 output channels.

[0039] The third stage uses 256 3×3 convolutional kernels with a stride of 1 and 256 output channels.

[0040] The fourth stage uses 512 3×3 convolution kernels with a stride of 1 and 512 output channels.

[0041] Each convolutional layer is followed by a batch normalization layer. The batch normalization layer standardizes the output of the convolutional layer, adjusting it to a distribution with a mean of 0 and a variance of 1. Then, it restores the expressive power of the features through learnable scaling and translation parameters. The batch normalization layer can accelerate network training convergence, alleviate the gradient vanishing problem, and improve the robustness of the model to the initial learning rate.

[0042] After the batch normalization layer, a ReLU activation function is connected. The ReLU activation function sets negative values ​​in the input to 0 and keeps positive values ​​unchanged, introducing non-linear expressive power into the network, enabling it to fit complex non-linear mapping relationships. Each downsampling stage is finally connected to a max pooling layer. The max pooling layer uses a 2×2 pooling window with a stride of 2, taking the maximum value in a 2×2 local region as the output. Through the max pooling operation, the spatial size of the feature map is compressed to half of the input, achieving downsampling. The max pooling layer has translation invariance, which can retain the most significant feature responses in the local region, while expanding the receptive field of subsequent layers.

[0043] Furthermore, the four downsampling stages are connected sequentially. The spatial size of the feature map output by each stage is halved and the number of channels is doubled step by step. The spatial size of the feature map output by the first stage is half that of the original image and the number of channels is 64; the spatial size of the feature map output by the second stage is one-quarter that of the original image and the number of channels is 128; the spatial size of the feature map output by the third stage is one-eighth that of the original image and the number of channels is 256; and the spatial size of the feature map output by the fourth stage is one-sixteenth that of the original image and the number of channels is 512. This step-by-step downsampling structure design follows the classic paradigm of encoder networks in the field of computer vision. Shallow feature maps have high spatial resolution and fewer channels, and each pixel corresponds to a small image region, thus preserving fine spatial location information and orientation details.

[0044] Deep feature maps have lower spatial resolution and more channels, with each pixel corresponding to a larger image region. Therefore, they capture high-level semantic category information and can determine whether the region belongs to a defect category and the type of defect. This complementary characteristic of "shallow layers preserving details and deep layers capturing semantics" provides a reasonable feature foundation for subsequent extraction of directional information based on shallow features and fusion of directional information with deep semantic features.

[0045] S3. Based on the first shallow feature map in the shallow feature map set, extract the local orientation information of each spatial location, parse and generate an anisotropic convolution kernel that matches the defect orientation according to the orientation information of the local orientation, and use the anisotropic convolution kernel to perform spatially adaptive orientation convolution enhancement on the first shallow feature map to obtain the orientation enhancement feature map, and retain the orientation response information to generate the orientation response map.

[0046] In this embodiment of the invention, the step of extracting local orientation information for each spatial location based on the first shallow feature map in the shallow feature map set, and parsing and generating anisotropic convolutional kernels that match the defect orientation based on the orientation information of the local orientation, includes:

[0047] The first shallow feature map is subjected to directional gradient filtering using a preset number of anisotropic Gabor filters covering the directional range from 0 degrees to 180 degrees to obtain a preset number of directional response maps.

[0048] Non-maximum suppression processing is performed on the directional response map. For each spatial location, the direction with the largest response value is retained in the neighborhood along the corresponding direction, and the response in other directions is suppressed to obtain the suppressed directional response map.

[0049] The suppressed directional response map is subjected to directional histogram statistics. The cumulative sum of the directional response values ​​is calculated within a local window of a preset size with each spatial location as the center. The direction index with the largest cumulative sum is taken as the local main direction of that spatial location, and a directional index map with the same spatial size as the first shallow feature map is obtained.

[0050] Based on the orientation index of each spatial location in the orientation index map, the corresponding principal orientation angle, standard deviation along the principal orientation, and standard deviation perpendicular to the principal orientation are queried from the preset mapping table to generate an anisotropic Gaussian convolution kernel weight matrix for each spatial location. The anisotropic Gaussian convolution kernel weight matrices generated for all spatial locations are normalized to obtain an anisotropic convolution kernel map.

[0051] It should be noted that the Gabor filter is a sinusoidal plane wave filter modulated by a Gaussian envelope, exhibiting good directional and frequency selectivity. The preset value M ranges from 8 to 16. When M is 12, the directional angle interval between adjacent Gabor filters is 180 degrees divided by 12, which equals 15 degrees. That is, the filter covers 12 directions: 0 degrees, 15 degrees, 30 degrees, 45 degrees, 60 degrees, 75 degrees, 90 degrees, 105 degrees, 120 degrees, 135 degrees, 150 degrees, and 165 degrees.

[0052] Each Gabor filter has the maximum response to edges and textures in a specific direction. Therefore, in the directional response map obtained after filtering, the spatial location with the larger response value indicates that the local texture direction at that location is more consistent with the directional angle of the Gabor filter. For linear defects such as cracks and scratches on the surface of metal structural parts, their local direction is represented as an edge response with a clear direction in the shallow feature map. The Gabor filter bank can capture the response intensity distribution of each pixel location in different directions.

[0053] It should be noted that the purpose of nonmaximum suppression is to eliminate ambiguity and redundancy in the directional response. For each spatial location, the response values ​​of that location in 12 directions are checked, and only the response value of the direction with the largest response value is retained, while the response values ​​of the remaining 11 directions are set to 0. At the same time, for the two adjacent locations along the direction of the largest response, it is checked whether the response value of that location is a local maximum. If the response value of that location is less than the response values ​​of its neighboring locations along the direction, then the response value of that location is also set to 0. This process ensures that each spatial location retains only sparse responses with clear directional dominance, making the directional response spatially sparse and clear, and avoiding the directional response from being too smooth and ambiguous due to noise or texture interference.

[0054] It should be noted that the local window size for the orientation histogram statistics is set to 3×3. For each spatial location in the suppressed orientation response map, nine spatial locations within a 3×3 neighborhood centered on that location are taken. The cumulative sum of the response values ​​of these nine locations in each direction is calculated to obtain a 12-dimensional orientation cumulative sum vector. The orientation index corresponding to the maximum value in this vector is taken as the local principal orientation of that spatial location. This local statistical method makes the orientation index map spatially consistent and robust. Even if a pixel location has an abnormal orientation response due to noise interference, most pixels in its neighborhood can still provide correct orientation information. Through local statistics, isolated abnormal orientation responses can be corrected, so that the generated orientation index map can accurately reflect the continuous direction of linear defects in the local area.

[0055] Furthermore, the parameters of the anisotropic convolution kernel are directly generated analytically based on the geometric and physical priors of the defects. Traditional data-driven methods allow the network to learn how to focus on defect regions by stacking a large amount of labeled data. What the network learns is often a statistical correlation rather than causality—for example, the network may learn a statistical association such as "a textured area may be a defect," but it cannot understand the geometric essence that "the defect should extend in a certain direction." This invention actively embeds physical priors into the network through the following analytical process: Gabor filters extract directional responses, non-maximum suppression eliminates ambiguity, directional histograms statistically determine local principal directions, directional indices are mapped to anisotropic Gaussian kernel parameters, and discrete sampling generates convolution kernel weights. Each step in this process has a clear physical meaning and geometric interpretability. The network no longer needs to "guess" the direction of defects through massive amounts of data, but directly uses known physical priors to explicitly guide the feature extraction process, realizing the essential methodological difference between analytical embedding of physical priors and data-driven implicit learning.

[0056] In this embodiment of the invention, the step of generating anisotropic Gaussian convolution kernel weight matrix corresponding to each spatial location, and normalizing the anisotropic Gaussian convolution kernel weight matrices generated for all spatial locations to obtain an anisotropic convolution kernel map, includes:

[0057] First, an anisotropic Gaussian kernel is calculated. Based on the anisotropic Gaussian kernel, an anisotropic Gaussian convolution kernel weight matrix is ​​generated. The anisotropic Gaussian convolution kernel weight matrices generated at all spatial locations are normalized to obtain the anisotropic convolution kernel map.

[0058] It should be noted that the mathematical expression for calculating the anisotropic Gaussian kernel is as follows:

[0059] In the formula, Represents an anisotropic Gaussian kernel. Let e ​​represent an exponential function with the natural constant e as its base. This represents a continuous coordinate offset relative to the center point of the convolution kernel. and It is a continuous real number variable, with a value range from negative infinity to positive infinity. This indicates the principal direction angle of the anisotropic Gaussian kernel. Indicates the standard deviation along the principal direction. This represents the standard deviation perpendicular to the principal direction;

[0060] Indicates the coordinates of the point The coordinate values ​​after rotation and projection along the main direction, i.e., the coordinate points. The projection component in the principal direction; the larger this value, the farther the point is from the principal direction axis in the principal direction.

[0061] Indicates the coordinates of the point The coordinates of the points after rotation and projection along a direction perpendicular to the principal direction. The projection component perpendicular to the principal direction; the larger this value, the farther the point is from the principal direction axis in a perpendicular direction.

[0062] This represents the square of the projection component along the principal direction divided by the square of the standard deviation along the principal direction. It is used to measure the degree to which the point deviates from the center point along the principal direction and is normalized by the standard deviation.

[0063] This represents the square of the projection component perpendicular to the principal direction divided by the square of the standard deviation perpendicular to the principal direction. It is used to measure the degree to which the point deviates from the center point perpendicular to the principal direction, and is normalized by the standard deviation.

[0064] This represents the exponential part of the exponential function, and its value is always negative. The larger its absolute value, the smaller the Gaussian function value at the corresponding location. When the coordinate point is located at the center of the convolution kernel... When the value of the exponential part is 0, the Gaussian function reaches its maximum value.

[0065] The normalization coefficient of the Gaussian kernel is used to ensure that the integral of the continuous Gaussian function is 1 over the entire two-dimensional plane. After discrete sampling, since only a limited range of discrete sampling points are taken, this normalization coefficient only guarantees the normalization of the integral in the continuous domain. Discrete normalization needs to be performed separately after discrete sampling.

[0066] The value ranges from 0 degrees to 180 degrees, and is used to control the extension direction of the Gaussian kernel. At a certain degree, the Gaussian kernel extends horizontally; when At a certain degree, the Gaussian kernel extends in the vertical direction.

[0067] The standard deviation along the principal direction is obtained by querying the preset mapping table using the direction index. Controlling the extension scale of the Gaussian kernel in the principal direction, The larger the Gaussian kernel, the wider its coverage in the principal direction, according to the preset mapping table. The value range is 3.0 to 5.0, with a recommended value of 4.0.

[0068] The standard deviation perpendicular to the principal direction is obtained by querying a pre-defined mapping table using the direction index. Controlling the extension scale of the Gaussian kernel perpendicular to the principal direction, The smaller the value, the narrower the coverage area of ​​the Gaussian kernel perpendicular to the principal direction. (Preset mapping table) The value range is 1.0 to 1.5, with a recommended value of 1.2.

[0069] It should be noted that the specific process for generating the anisotropic Gaussian convolution kernel weight matrix is ​​as follows:

[0070] First, based on the standard deviation along the principal direction Determine the size K of the convolution kernel, where K takes the value of... Round up to ensure the convolution kernel covers the main energy region of the Gaussian function; when When K=12, it is usually rounded up to 13, because odd-numbered dimensions are easier to align with the center point.

[0071] Secondly, on a K×K discrete coordinate grid, with the grid center as the origin. Take the integer coordinates of each grid point in sequence. (in and The range of values ​​is arrive ),Will Substituting into the above mathematical expression for the continuity of the anisotropic Gaussian kernel, the Gaussian function value of the grid point is calculated.

[0072] Then, for all K×K grid points, the Gaussian function values ​​are calculated and then discretely normalized: the Gaussian function value of each grid point is divided by the sum of the Gaussian function values ​​of all grid points, so that the sum of the weights of all grid points after normalization is 1. The normalization process ensures that the convolution operation does not change the overall energy level of the feature map and avoids feature response shift caused by the sum of the weights of the convolution kernel not being 1.

[0073] It should be noted that the above process is repeated for all spatial locations. Each spatial location generates an independent anisotropic Gaussian convolution kernel weight matrix based on its orientation index. The convolution kernel weight matrices of all spatial locations are combined into an anisotropic convolution kernel map with the same spatial size as the first shallow feature map.

[0074] In this embodiment of the invention, the step of performing spatially adaptive directional convolution enhancement on the first shallow feature map using the anisotropic convolution kernel to obtain a directionally enhanced feature map, and retaining directional response information to generate a directional response map, includes:

[0075] The anisotropic convolution kernel map is used to perform position-wise adaptive convolution with the first shallow feature map in a sliding window manner. For each spatial position in the first shallow feature map, the local region on the feature map is weighted and summed using the anisotropic Gaussian convolution kernel weight matrix corresponding to each spatial position to obtain the enhanced response value of the corresponding spatial position. After performing the same operation on all spatial positions, the first shallow feature map with enhanced orientation is obtained.

[0076] The first shallow feature map after directional enhancement is concatenated with the second shallow feature map in the channel dimension, and the number of channels after concatenation is mapped to a preset number of channels through a 1×1 convolution to obtain the shallow directional enhancement feature map.

[0077] A preset number of directional response maps are concatenated along the channel dimension, and a 1×1 convolution is used to map the number of channels after concatenation to the preset number of channels to obtain the directional response map.

[0078] It should be noted that the specific execution process of the orientation-adaptive convolution operation is as follows: Assume that the spatial size of the first shallow feature map is H×W, the number of channels is C1, and the size of the anisotropic Gaussian convolution kernel weight matrix corresponding to each spatial position is K×K. For the spatial position of the i-th row and j-th column in the output feature map, a K×K neighborhood region centered at position (i,j) is extracted from the first shallow feature map. The spatial size of this neighborhood region is K×K and the number of channels is C1.

[0079] The anisotropic Gaussian convolution kernel weight matrix corresponding to position (i,j) in the anisotropic convolution kernel map is broadcast along the channel dimension to align it with the number of channels C1 in the neighborhood region. The aligned convolution kernel weight matrix is ​​then multiplied element-wise with the neighborhood region, and the multiplications are accumulated to obtain a scalar value, which is the enhanced response value at position (i,j). The above operation is repeated for all spatial positions (i,j) to obtain the first shallow feature map after directional enhancement of H×W×C1.

[0080] Furthermore, since the anisotropic Gaussian convolution kernel extends along the main direction and contracts perpendicular to the main direction, when the convolution kernel acts on a linear defect region extending along this direction, the large coverage of the convolution kernel along the main direction can accumulate and enhance the continuous pixel features of the defect, while the small coverage perpendicular to the main direction can suppress background texture noise perpendicular to the defect direction, so that the response value of the defect region in the output feature map is significantly higher than that of the background region.

[0081] Conversely, in the background region, since there is no linear structure matching the direction of the anisotropic convolution kernel, the weighted summation of the convolution kernels is close to zero or a low value, thereby achieving directional enhancement of the defect signal and directional suppression of background noise.

[0082] It should be noted that the spatial size of the first shallow feature map after orientation enhancement is half that of the original image, and the spatial size of the second shallow feature map is one-quarter that of the original image. Since the spatial sizes of the two are inconsistent, before performing channel-dimensional concatenation, the first shallow feature map after orientation enhancement is first subjected to max pooling with a 2x downsampling to make its spatial size consistent with one-quarter of the size of the second shallow feature map. Then, the downsampled orientation-enhanced feature map and the second shallow feature map are concatenated in the channel dimension.

[0083] Assuming the number of channels in the first shallow feature map after orientation enhancement is C1 and the number of channels in the second shallow feature map is C2, the number of channels after concatenation is the sum of C1 and C2. A 1×1 convolution maps the number of channels after concatenation to a preset number of channels, usually set to 256. The purpose of this 1×1 convolution is to perform linear fusion and dimensionality reduction of the two features from different sources, so that the output shallow orientation-enhanced feature map contains both the orientation enhancement information of the first shallow feature map and the local region information of the second shallow feature map, thus achieving the initial integration of shallow multi-scale information.

[0084] It should be noted that, in the preset number of directional response maps, the spatial size of each directional response map is related to the first shallow layer features. Figure 1 Given M orientation response maps with a channel count of 1, concatenate them along the channel dimension to obtain an orientation response tensor with M channels. Then, use a 1×1 convolution to map the channel count of this orientation response tensor from M to a preset number of channels (default is 256) to obtain the final orientation response map.

[0085] The directional response map retains the response intensity distribution information of each spatial location in different directions. Its spatial size is one-quarter of the original image, which is consistent with the spatial size of the shallow and medium-depth directional enhancement feature map, ensuring that it can be spatially aligned and fused across layers with the deep feature map in subsequent steps.

[0086] Furthermore, the generation of the orientation response map and the generation of the orientation enhancement feature map are two independent but complementary processes: the orientation enhancement feature map is used to retain the original feature content after orientation adaptive convolution enhancement, while the orientation response map is used to retain the orientation response intensity distribution information as the orientation guidance signal for subsequent cross-layer fusion. As explicit orientation prior information, the orientation response map is used to guide the deep feature map for semantic correction in subsequent steps, so that the orientation information not only plays a role in shallow feature extraction, but can also be passed across layers to deep semantic features, realizing the pervasive guidance of orientation prior information throughout the entire network.

[0087] S4, use the directional response map to perform directional-guided semantic correction on the deep feature map to obtain a semantically corrected deep feature map, and perform channel-weighted fusion on the directional enhancement feature map and the semantically corrected deep feature map to obtain a fused feature map;

[0088] In this embodiment of the invention, the step of using the directional response map to perform directional-guided semantic correction on the deep feature map to obtain a semantically corrected deep feature map includes:

[0089] The deep feature map is upsampled using bilinear interpolation so that its spatial size is consistent with the spatial size of the shallow direction-enhanced feature map, thus obtaining the upsampled deep feature map.

[0090] The orientation response map is downsampled by average pooling so that its spatial size is consistent with the spatial size of the upsampled deep feature map, thus obtaining an aligned orientation response map.

[0091] The upsampled deep feature map and the aligned orientation response map are concatenated along the channel dimension. The concatenated tensor is then input into the orientation-aware semantic correction surface, which includes a first two-dimensional convolutional layer, a ReLU activation function, and a second two-dimensional convolutional layer connected in sequence, and outputs a semantically corrected deep feature map.

[0092] It should be noted that the spatial size of the deep feature map is one-sixteenth of the original image size, while the spatial size of the shallow directional enhancement feature map is one-quarter of the original image size; therefore, the deep feature map needs to be upsampled by 4 times. The basic principle of bilinear interpolation upsampling is: for each pixel position in the target image, its corresponding floating-point coordinates in the source image are calculated through reverse mapping, and the values ​​of the four nearest integer coordinate pixels around the floating-point coordinates are taken. A bilinear weighted average is then performed according to distance weights to obtain the value of the target pixel. Bilinear interpolation can maintain the smoothness of the feature map during the upsampling process and avoid the generation of checkerboard artifacts. After 4 times upsampling, the spatial size of the deep feature map is consistent with the spatial size of the shallow directional enhancement feature map, providing a prerequisite for spatial alignment for subsequent channel stitching and semantic correction.

[0093] It should be noted that the spatial size of the orientation response map is one-quarter of the original image size, which is consistent with the spatial size of the shallow orientation enhancement feature map. The spatial size of the upsampled deep feature map has also been adjusted to one-quarter of the original image size. Therefore, the spatial size of the orientation response map and the upsampled deep feature map are already consistent, and no additional downsampling is required.

[0094] Here, average pooling downsampling is used as a general description. When the spatial dimensions of the directional response map are inconsistent with those of the upsampled deep feature map, average pooling can be used to downsample the directional response map to a size consistent with the deep feature map. Figure 1 To achieve consistent spatial dimensions, average pooling uses a 2×2 pooling window with a stride of 2. The average value of the four pixels within the pooling window is taken as the output, which can preserve the overall distribution information of directional response intensity while reducing spatial resolution.

[0095] It should be noted that the number of channels in the upsampled deep feature map is 512, and the number of channels in the aligned orientation response map is the preset number of 256. The two are concatenated in the channel dimension to obtain a concatenated tensor with 768 channels. In this concatenated tensor, the first 512 channels contain deep semantic features, and the last 256 channels contain the orientation response intensity distribution information of each spatial location. The concatenated tensor is then input into the orientation-aware semantic correction surface.

[0096] The orientation-aware semantic correction surface comprises a first two-dimensional convolutional layer, a ReLU activation function, and a second two-dimensional convolutional layer connected in sequence. The first two-dimensional convolutional layer uses a 3×3 convolutional kernel, mapping the input channel count of 768 to the output channel count of 512. This convolutional layer's function is to initially fuse orientation response information with deep semantic features, enabling the convolutional kernel to learn the mapping relationship of "enhancing semantic response in regions of strong orientation response and suppressing semantic response in regions of weak orientation response." The ReLU activation function sets negative values ​​in the output of the first two-dimensional convolutional layer to 0 and keeps positive values ​​unchanged, introducing non-linear expressive power, allowing the semantic correction surface to learn a non-linear orientation-semantic mapping relationship. The second two-dimensional convolutional layer uses a 1×1 convolutional kernel, mapping the 512 channels back to the original 512 channels of the deep feature map, outputting a semantically corrected deep feature map. The mathematical expression for its core calculation process is as follows:

[0097]

[0098] in, This represents the deep feature map after upsampling. This represents the aligned directional response map. This indicates a concatenation operation along the channel dimension. This represents the first two-dimensional convolutional layer (3×3 convolutional kernel). Represents the ReLU activation function. This represents the second two-dimensional convolutional layer (1×1 convolutional kernel). This represents the deep feature map after semantic correction.

[0099] Furthermore, high response areas in the directional response map indicate the presence of a linear texture structure matching a certain direction at that spatial location, and this area is likely to be a linear defect such as a crack or scratch; low response areas in the directional response map indicate that the spatial location does not have a clear directional structure, and this area belongs to a normal surface or an isotropic pit-type defect area.

[0100] The orientation-aware semantic correction surface learns to enhance the response values ​​of deep semantic features at spatial locations with strong orientation responses, making the network more inclined to classify these locations as defective. At spatial locations with weak orientation responses, it retains or suppresses the response values ​​of deep semantic features, allowing the discriminative ability of the deep semantic features themselves to determine whether the location belongs to the defective category. This correction process enables orientation prior information to be transmitted from shallow feature maps to deep feature maps, realizing the transfer and fusion of orientation information across scale features.

[0101] To further ensure the effectiveness of direction-guided semantic correction, a direction consistency loss function is introduced during the training phase to supervise the spatial distribution consistency between the semantically corrected deep feature map and the direction response map. The formula for calculating the direction consistency loss function is as follows:

[0102]

[0103] in, This represents the direction consistency loss function. Coordinate index representing spatial location, and These represent the height and width of the feature map, respectively. This indicates the location of the semantically corrected deep feature map. The channel dimension vector at that location, This represents a learnable linear projection matrix used to project the 512-dimensional semantic vectors of the deep feature map onto the directional response. Figure 1 The 256-dimensional space of the universe This indicates the aligned directional response map at position. The channel dimension vector at that location (dimension 256). It represents the square of the L2 norm, which is the sum of the squares of the differences between the dimensions of the vectors;

[0104] Spatial location coordinates in the feature map This is the row index, with a value range of 1 to... , For column indexes, the value range is 1 to... The loss function calculates the loss value for each spatial location in the feature map.

[0105] The height and width of the feature map, in the semantically corrected deep feature map and the aligned orientation response map, and The values ​​are the same, all being one-quarter of the original image size. This means averaging the loss values ​​over all spatial locations, making the loss function value independent of the feature map size;

[0106] The spatial location of the semantically corrected deep feature map The channel dimension vector at a given location contains 512 values, each representing the feature response value of that spatial location in the corresponding channel, comprehensively reflecting the semantic category information of that location;

[0107] A learnable linear projection matrix of size 512×256 is used to project the 512-dimensional deep semantic feature vectors into a 256-dimensional space identical to the directional response map, allowing for comparison between the two in the same dimensional space. The parameters in the training process are optimized end-to-end through backpropagation.

[0108] Indicates spatial location 512-dimensional deep semantic feature vector and projection matrix at the location Perform matrix multiplication to obtain a 256-dimensional projected semantic vector;

[0109] This indicates the spatial location of the aligned directional response map. The channel dimension vector at that location. This vector contains 256 values, each representing the directional response intensity of that spatial location in the corresponding directional channel;

[0110] This represents the square of the L2 distance between the projected semantic vector and the orientation response vector. The smaller this value, the more consistent the spatial distribution of the projected semantic feature vector and the orientation response vector, i.e., the more consistent the spatial location of the semantically corrected deep feature map. The semantic response distribution at a given location matches the distribution of shallow directional response information at that location.

[0111] The total loss of the orientation consistency loss function is the mean of the squared L2 distances across all spatial locations. During network training, this loss function, together with the cross-entropy loss function used in the segmentation task, constitutes the total loss function, jointly optimizing the network parameters. The formula for calculating the total loss function is as follows:

[0112]

[0113] in, Represents the total loss function. This represents the splitting cross-entropy loss function. This represents the direction consistency loss function. This represents the weighting coefficient of the directional consistency loss function, used to balance the contribution ratio of the two loss functions, and its value ranges from 0.1 to 0.5.

[0114] Furthermore, by using explicit loss function constraints, the semantic response distribution of deep semantic features is forced to align with the directional response distribution of shallow directional prior information. Without directional consistency loss function constraints, although the direction-aware semantic correction surface can inject directional response information into the deep feature map, the network may choose to ignore the guidance of directional response information, degenerating into ordinary feature concatenation and convolution operations. After introducing the directional consistency loss function, the network is forced to learn how to "align" the directional response with the semantic response in spatial location—in regions with strong directional responses, the semantic response must also be enhanced accordingly; in regions with weak directional responses, the semantic response should be suppressed or remain unchanged. This explicit constraint is the essential difference between physical prior analytical embedding and data-driven implicit learning: the former actively constructs constraints based on known physical laws to guide the network's learning direction; the latter relies solely on statistical correlations in a large amount of labeled data, allowing the network to explore potentially correct mapping relationships on its own. The directional consistency loss function injects the prior knowledge that "the direction of defects and the spatial distribution of semantic responses should be consistent" into the network training process in the form of mathematical constraints, so that the network does not passively accept data-driven learning but actively follows physical laws.

[0115] In this embodiment of the invention, the step of performing channel-weighted fusion of the directional enhancement feature map and the semantically corrected deep feature map to obtain a fused feature map includes:

[0116] Global average pooling is performed on the semantically corrected deep feature map to obtain a deep channel description vector; global average pooling is performed on the shallow directional enhancement feature map to obtain a shallow channel description vector; the deep channel description vector and the shallow channel description vector are concatenated along the channel dimension to obtain a fused description vector.

[0117] The fused description vector is input into a fully connected network containing one hidden layer, and the deep channel weight vector and shallow channel weight vector are output by the Sigmoid function respectively.

[0118] The deep channel weight vector and the semantically corrected deep feature map are multiplied channel by channel in the channel dimension to obtain a deep weighted feature map. The shallow channel weight vector and the shallow directional enhancement feature map are multiplied channel by channel in the channel dimension to obtain a shallow weighted feature map. The deep weighted feature map and the shallow weighted feature map are added element by element to obtain a fused feature map.

[0119] It should be noted that global average pooling averages the feature values ​​of all spatial locations of each channel in the feature map to obtain a scalar value. Each channel corresponds to a scalar value. Therefore, after global average pooling, each feature map yields a one-dimensional vector with a length equal to the number of channels. The semantically corrected deep feature map has 512 channels. After global average pooling, it yields a deep channel description vector of length 512. Each value in the deep channel description vector represents the average response intensity of the corresponding channel in the global space, reflecting the contribution of the channel to the semantic discrimination of defects at the entire image level. The shallow directional enhancement feature map has a preset number of channels, 256. After global average pooling, it yields a shallow channel description vector of length 256. Each value in the shallow channel description vector represents the average response intensity of the corresponding channel in the global space, reflecting the channel's ability to represent directional information. The deep channel description vector and the shallow channel description vector are concatenated along the channel dimension to obtain a fused description vector of length 768. The fused description vector contains global channel importance information of both deep semantic features and shallow directional enhancement features, providing a global statistical basis for the subsequent generation of channel weight vectors.

[0120] It should be noted that the fully connected network comprises an input layer, a hidden layer, and an output layer connected sequentially. The input layer has 768 neurons, corresponding to the length of the fused description vector. The hidden layer has half the number of neurons as the input layer, i.e., 384. The hidden layer uses the ReLU activation function to introduce non-linear mapping capabilities, enabling the fully connected network to learn complex non-linear relationships between channels. The output layer contains two branches, outputting deep channel weight vectors and shallow channel weight vectors respectively. The number of output neurons in each branch is consistent with the number of channels in the corresponding feature map. The deep branch outputs a deep channel weight vector of length 512, and the shallow branch outputs a shallow channel weight vector of length 256.

[0121] The Sigmoid function operates on each neuron in the output layer, mapping the real values ​​output by the neuron to an open interval between 0 and 1. The mathematical expression for the Sigmoid function is as follows:

[0122]

[0123] in, This represents the raw output value of a neuron in the output layer of a fully connected network. Represents the natural constant. Represented by natural constant base Power of 1 This represents the weight value after mapping by the Sigmoid function, and its value ranges from 0 to 1;

[0124] when The larger the value, The closer the value is to 0, The closer the value is to 1, the better; The smaller the value, The larger the value, The closer the value is to 0, the more the Sigmoid function makes each element in the output channel weight vector between 0 and 1, representing the importance weight of that channel in the fusion process. Channels with weight values ​​closer to 1 have a larger proportion in the fusion, while channels with weight values ​​closer to 0 have their contribution to the fusion suppressed.

[0125] It should be noted that, assuming the deep channel weight vector is a one-dimensional vector of length 512, and the number of channels in the semantically corrected deep feature map is 512, the first channel weight vector in the deep channel weight vector will be... The weight value is related to the weight of the semantically corrected deep feature map. Perform scalar multiplication on the feature values ​​of all spatial locations on the i-th channel, i.e., the i-th The feature value of each spatial location on each channel is multiplied by the weight value. After performing the same operation on all 512 channels, a deep weighted feature map is obtained. The generation process of the shallow weighted feature map is similar. The shallow channel weight vector of length 256 is multiplied by the 256 channels of the shallow directional enhancement feature map respectively to obtain the shallow weighted feature map.

[0126] The deep weighted feature map and the shallow weighted feature map are summed element-wise based on their corresponding spatial locations and channel feature values. The shallow weighted feature map has a spatial size of one-quarter of the original image and 256 channels. The spatial size of the deep weighted feature map is equal to that of the shallow weighted feature map. Figure 1 Since the number of channels is inconsistent between the two, the deep weighted feature map is first compressed from 512 to 256 through a 1×1 convolution to make the number of channels consistent. Then, the element-wise addition operation is performed. The result of the addition is the fused feature map. The spatial size of the fused feature map is one-quarter of the original image, and the number of channels is 256.

[0127] The complete calculation formula for channel-weighted fusion is as follows:

[0128]

[0129] in, Represents the deep channel weight vector. This represents the deep feature map after semantic correction. This represents the shallow channel weight vector. This represents a shallow-area directional enhancement feature map. This represents channel-by-channel multiplication along the channel dimension. This indicates element-wise addition after channel alignment. This represents the fused feature map.

[0130] Furthermore, channel-weighted fusion differs from traditional simple feature fusion methods. Traditional methods, such as element-wise addition or convolution after channel concatenation, treat all channels equally and cannot distinguish the differences in contribution of different channels to the fusion result. In contrast, this step learns the importance weight of each channel through a fully connected network and a sigmoid function, enabling the network to adaptively adjust the weight of different channels in the fusion based on the content of the input image. For defect regions, direction-sensitive channels in the orientation enhancement feature map receive higher weights, and semantic discriminative channels in the semantically corrected deep feature map also receive higher weights. For background regions, the weights of channels with weak orientation responses are suppressed to avoid misclassifying background textures as defect information and incorporating them into the fused feature map. This adaptive channel weighting mechanism, combined with the constraint of the orientation consistency loss function, ensures that the spatial distribution of the fused feature map is consistent with the physical prior of the defect orientation, further demonstrating the essential difference between physical prior analytical embedding and data-driven implicit learning.

[0131] S5, the fused feature map is input into the segmentation decoder, and a binary segmentation mask with the same size as the original grayscale image is output to complete the semantic segmentation of surface defects of metal structural parts.

[0132] In this embodiment of the invention, the step of inputting the fused feature map into the segmentation decoder and outputting a binary segmentation mask with the same size as the original grayscale image to complete the semantic segmentation of surface defects of the metal structural component includes:

[0133] The fused feature map is input into the segmentation decoder, which includes an upsampling layer, a two-dimensional convolutional layer and a softmax output layer connected in sequence. The upsampling layer uses bilinear interpolation to restore the spatial size of the fused feature map to be consistent with the original grayscale image. The two-dimensional convolutional layer maps the number of channels to the number of categories, which includes defect categories and normal background categories.

[0134] The category score of each pixel is normalized by the Softmax output layer to obtain the probability value of each pixel belonging to the defect category;

[0135] Pixels with a probability value greater than a preset threshold are identified as defective pixels, and pixels with a probability value less than or equal to the preset threshold are identified as normal pixels. A binary segmentation mask with the same size as the original grayscale image is output.

[0136] It should be noted that the spatial size of the fused feature map is one-quarter of the original image size, and the number of channels is 256. The first layer of the segmentation decoder is an upsampling layer, which uses bilinear interpolation to enlarge the spatial size of the fused feature map by 4 times, restoring it to the same size as the original grayscale image. Bilinear interpolation is a classic upsampling method. By performing distance-weighted linear interpolation on the four nearest neighbor pixels of the target pixel in the source feature map, the feature value of the target pixel is calculated. This method can maintain the smoothness and continuity of the feature map during the upsampling process and avoid the generation of checkerboard artifacts or blocky effects.

[0137] The upsampled feature map is input into a two-dimensional convolutional layer. This two-dimensional convolutional layer uses a 1×1 convolutional kernel to map the 256 input channels to 2 output channels. The two channels correspond to the score maps of the defect category and the normal background category, respectively. The score value of each spatial location on these two channels represents the confidence that the pixel belongs to the defect category and the normal background category, respectively. The role of the 1×1 convolutional kernel is to perform a linear transformation of the channel dimension of the feature map without changing the spatial resolution, compressing the 256-dimensional fused features into 2-dimensional category scores, realizing the mapping from the high-dimensional feature space to the category space. It is important to emphasize that the 1×1 convolution is chosen instead of a large-size convolutional kernel to avoid introducing additional spatial information mixing during the classification process, ensuring that the classification decision of each pixel is based only on the fused feature vector corresponding to that location.

[0138] It should be noted that the Softmax function converts the two class scores output by the two-dimensional convolutional layer into probability values, such that the sum of the two probability values ​​is 1. The mathematical expression of the Softmax function is as follows:

[0139]

[0140] in, Represents the spatial coordinates of a pixel. Indicates position The score that belongs to the defect category. Indicates position The score value belongs to the normal background category. Represented by natural constant An exponential function with base 0. Indicates position The probability value that the location belongs to the defect category, ranging from 0 to 1.

[0141] The spatial coordinates of a pixel in an image. For row index, The column index, with a value range determined by the size of the original grayscale image, represents the defect category score map in the output of the 2D convolutional layer at the specified position. The value at this location. A larger value indicates a higher confidence level in the network's assessment that the pixel belongs to the defect category; a smaller value indicates a lower confidence level. The score map of the normal background category in the output of the 2D convolutional layer is located at... The value at this location indicates that the network has a higher confidence level in considering the pixel to be a normal background; the smaller the value, the lower the confidence level in considering the pixel to be a normal background.

[0142] Represented by natural constant as the base This is the value of the exponential function of the exponent. The exponential function converts the score values ​​to positive numbers while preserving the relative magnitudes of the scores, so that categories with higher scores receive larger probability values ​​after normalization.

[0143] This represents the sum of the exponential function values ​​of the two categories, used as the normalized denominator to ensure that the sum of the probability values ​​of the two categories is 1.

[0144] This represents the probability value of the defect category in the final output, and the value is between 0 and 1. The closer the value is to 1, the greater the likelihood that the pixel belongs to the defect category; The closer the value is to 0, the greater the likelihood that the pixel belongs to the normal background category.

[0145] The Softmax function maps unbounded scores to bounded probability values, making the category scores comparable and interpretable in terms of probability, and providing a unified probability threshold standard for subsequent binarization.

[0146] It should be noted that the preset threshold is usually set to 0.5. That is, when the probability of a pixel belonging to the defect category is greater than 0.5, the pixel is determined to be a defective pixel, and the pixel value at the corresponding position in the binary segmentation mask is set to 1; when the probability is not greater than 0.5, the pixel is determined to be a normal pixel, and the pixel value at the corresponding position in the binary segmentation mask is set to 0.

[0147] The preset threshold can be dynamically adjusted according to the requirements of detection accuracy and recall in the actual application scenario. When the tolerance for missed defect detection is low, the preset threshold can be lowered to 0.3 to 0.4 to improve the recall rate, but may introduce false detections. When the tolerance for false detection is low, the preset threshold can be raised to 0.6 to 0.7 to improve the precision rate, but may reduce the recall rate. The output binary segmentation mask has the same size as the original grayscale image, and the value of each pixel is 0 or 1. The connected region formed by the pixels with a value of 1 is the detected defect region, realizing pixel-level precise positioning of surface defects of metal structural parts.

[0148] In the several embodiments provided by this invention, it should be understood that the disclosed method can be implemented in other ways.

[0149] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0150] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, and technology that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A semantic segmentation method for surface defect images of metal structural components based on deep learning, characterized in that, The method includes: S1, acquire the original grayscale image of the surface area of ​​the metal structural component, preprocess the original grayscale image to obtain the enhanced surface image; S2, the enhanced surface image is input into a pre-trained multi-scale encoder network to extract a shallow feature map set and a deep feature map. The shallow feature map set consists of shallow feature maps that preserve spatial location details and orientation details. S3. Based on the first shallow feature map in the shallow feature map set, extract the local orientation information of each spatial location, parse and generate an anisotropic convolution kernel that matches the defect orientation according to the orientation information of the local orientation, and use the anisotropic convolution kernel to perform spatially adaptive orientation convolution enhancement on the first shallow feature map to obtain the orientation enhancement feature map, and retain the orientation response information to generate the orientation response map. S4, use the directional response map to perform directional-guided semantic correction on the deep feature map to obtain a semantically corrected deep feature map, and perform channel-weighted fusion on the directional enhancement feature map and the semantically corrected deep feature map to obtain a fused feature map; S5, the fused feature map is input into the segmentation decoder, and a binary segmentation mask with the same size as the original grayscale image is output to complete the semantic segmentation of surface defects of metal structural parts.

2. The semantic segmentation method for surface defect images of metal structural components based on deep learning as described in claim 1, characterized in that, The original grayscale image of the surface area of ​​the metal structural component is acquired, and the original grayscale image is preprocessed to obtain the enhanced surface image, including: Acquire the original grayscale image of the surface area of ​​the metal structural component; The original grayscale image is subjected to median filtering for denoising to obtain a denoised grayscale image. The denoised grayscale image is subjected to contrast-limited adaptive histogram equalization. The denoised grayscale image is divided into a preset number of local sub-blocks. Histogram equalization is performed on each local sub-block, and the local contrast enhancement is limited to obtain an enhanced surface image.

3. The semantic segmentation method for surface defect images of metal structural components based on deep learning as described in claim 1, characterized in that, The enhanced surface image is input into a pre-trained multi-scale encoder network to extract shallow feature map sets and deep feature maps, including: The enhanced surface image is input into a pre-trained multi-scale encoder network. The feature map output from the first stage is selected as the first shallow feature map, and the feature map output from the second stage is selected as the second shallow feature map. The first shallow feature map and the second shallow feature map together constitute a shallow feature map set, which is used to preserve fine spatial location information and orientation details. The feature map output from the fourth stage is selected as the deep feature map, which is used to preserve high-level semantic information.

4. The semantic segmentation method for surface defect images of metal structural components based on deep learning as described in claim 3, characterized in that, The downsampling stage of a multi-scale encoder network includes: The multi-scale encoder network contains four sequentially connected downsampling stages. Each downsampling stage consists of a convolutional layer, a batch normalization layer, a ReLU activation function, and a max pooling layer connected in sequence, which respectively output feature maps with spatial sizes of one-half, one-quarter, one-eighth, and one-sixteenth of the enhanced surface image.

5. The semantic segmentation method for surface defect images of metal structural components based on deep learning as described in claim 1, characterized in that, Based on the first shallow feature map in the shallow feature map set, local orientation information for each spatial location is extracted. Anisotropic convolutional kernels matching the defect orientation are then parsed and generated based on this local orientation information, including: The first shallow feature map is subjected to directional gradient filtering using a preset number of anisotropic Gabor filters covering the directional range from 0 degrees to 180 degrees to obtain a preset number of directional response maps. Non-maximum suppression processing is performed on the directional response map. For each spatial location, the direction with the largest response value is retained in the neighborhood along the corresponding direction, and the response in other directions is suppressed to obtain the suppressed directional response map. The suppressed directional response map is subjected to directional histogram statistics. The cumulative sum of the directional response values ​​is calculated within a local window of a preset size with each spatial location as the center. The direction index with the largest cumulative sum is taken as the local main direction of that spatial location, and a directional index map with the same spatial size as the first shallow feature map is obtained. Based on the orientation index of each spatial location in the orientation index map, the corresponding principal orientation angle, standard deviation along the principal orientation, and standard deviation perpendicular to the principal orientation are queried from the preset mapping table to generate an anisotropic Gaussian convolution kernel weight matrix for each spatial location. The anisotropic Gaussian convolution kernel weight matrices generated for all spatial locations are normalized to obtain an anisotropic convolution kernel map.

6. The semantic segmentation method for surface defect images of metal structural components based on deep learning as described in claim 5, characterized in that, Generate an anisotropic Gaussian convolution kernel weight matrix for each spatial location. Normalize the anisotropic Gaussian convolution kernel weight matrices generated for all spatial locations to obtain an anisotropic convolution kernel map, including: First, an anisotropic Gaussian kernel is calculated. Based on the anisotropic Gaussian kernel, an anisotropic Gaussian convolution kernel weight matrix is ​​generated. The anisotropic Gaussian convolution kernel weight matrices generated at all spatial locations are normalized to obtain the anisotropic convolution kernel map.

7. The semantic segmentation method for surface defect images of metal structural components based on deep learning as described in claim 1, characterized in that, The anisotropic convolution kernel is used to perform spatially adaptive directional convolution enhancement on the first shallow feature map to obtain a directionally enhanced feature map, and directional response information is retained to generate a directional response map, including: The anisotropic convolution kernel map is used to perform position-wise adaptive convolution with the first shallow feature map in a sliding window manner. For each spatial position in the first shallow feature map, the local region on the feature map is weighted and summed using the anisotropic Gaussian convolution kernel weight matrix corresponding to each spatial position to obtain the enhanced response value of the corresponding spatial position. After performing the same operation on all spatial positions, the first shallow feature map with enhanced orientation is obtained. The first shallow feature map after directional enhancement is concatenated with the second shallow feature map in the channel dimension, and the number of channels after concatenation is mapped to a preset number of channels through a 1×1 convolution to obtain the shallow directional enhancement feature map. A preset number of directional response maps are concatenated along the channel dimension, and a 1×1 convolution is used to map the number of channels after concatenation to the preset number of channels to obtain the directional response map.

8. The semantic segmentation method for surface defect images of metal structural components based on deep learning as described in claim 7, characterized in that, Using the directional response map, semantic correction guided by direction is performed on the deep feature map to obtain a semantically corrected deep feature map, including: The deep feature map is upsampled using bilinear interpolation so that its spatial size is consistent with the spatial size of the shallow direction-enhanced feature map, thus obtaining the upsampled deep feature map. The orientation response map is downsampled by average pooling so that its spatial size is consistent with the spatial size of the upsampled deep feature map, thus obtaining an aligned orientation response map. The upsampled deep feature map and the aligned orientation response map are concatenated along the channel dimension. The concatenated tensor is then input into the orientation-aware semantic correction surface, which includes a first two-dimensional convolutional layer, a ReLU activation function, and a second two-dimensional convolutional layer connected in sequence, and outputs a semantically corrected deep feature map.

9. The semantic segmentation method for surface defect images of metal structural components based on deep learning as described in claim 7, characterized in that, The directional enhancement feature map and the semantically corrected deep feature map are fused using channel weighting to obtain a fused feature map, including: Global average pooling is performed on the semantically corrected deep feature map to obtain a deep channel description vector; global average pooling is performed on the shallow directional enhancement feature map to obtain a shallow channel description vector; the deep channel description vector and the shallow channel description vector are concatenated along the channel dimension to obtain a fused description vector. The fused description vector is input into a fully connected network containing one hidden layer, and the deep channel weight vector and shallow channel weight vector are output by the Sigmoid function respectively. The deep channel weight vector and the semantically corrected deep feature map are multiplied channel by channel in the channel dimension to obtain a deep weighted feature map. The shallow channel weight vector and the shallow directional enhancement feature map are multiplied channel by channel in the channel dimension to obtain a shallow weighted feature map. The deep weighted feature map and the shallow weighted feature map are added element by element to obtain a fused feature map.

10. The semantic segmentation method for surface defect images of metal structural components based on deep learning as described in claim 1, characterized in that, The fused feature map is input into the segmentation decoder, which outputs a binary segmentation mask with the same size as the original grayscale image, thus completing the semantic segmentation of surface defects on the metal structure, including: The fused feature map is input into the segmentation decoder, which includes an upsampling layer, a two-dimensional convolutional layer and a softmax output layer connected in sequence. The upsampling layer uses bilinear interpolation to restore the spatial size of the fused feature map to be consistent with the original grayscale image. The two-dimensional convolutional layer maps the number of channels to the number of categories, which includes defect categories and normal background categories. The category score of each pixel is normalized by the Softmax output layer to obtain the probability value of each pixel belonging to the defect category; Pixels with a probability value greater than a preset threshold are identified as defective pixels, and pixels with a probability value less than or equal to the preset threshold are identified as normal pixels. A binary segmentation mask with the same size as the original grayscale image is output.

Images