An industrial part defect detection method based on multi-scale feature fusion

Abstract

The application relates to the technical field of image detection, and discloses an industrial part defect detection method based on multi-scale feature fusion, which comprises the following steps: taking the geometric center of a part as a reference, and dividing a surface optical image into a plurality of mutually non-overlapping image blocks; performing feature up-sampling on a deep feature map of each image block to obtain a semantic enhancement feature map; performing channel recalibration on the semantic enhancement feature map based on the channel attention weight of a middle feature map to generate a semantic guide feature map; performing multi-scale fusion on the deep feature map, the semantic guide feature map and the middle feature map; analyzing the rotation direction between a fusion feature map of each image block and a defect prior feature vector, and performing a multi-angle rotation alignment operation with the center of the industrial part to be detected as the center of a circle on each image block; and fusing the defect detection results of all the image blocks to obtain a pixel-level defect classification and positioning map of the part. The application can improve the defect detection accuracy of industrial parts.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image detection, and in particular to an industrial part defect detection method based on multi-scale feature fusion. BACKGROUND

[0002] In the existing industrial part automatic visual detection technology, the semantic segmentation and target detection method based on deep learning has become the mainstream. However, when dealing with precision parts with complex three-dimensional structure and possible posture deviation in imaging, this kind of method faces inherent limitations.

[0003] The traditional method usually performs global processing on the whole part image. When the part is rotated or locally warped during installation or imaging, the visual representation of the defect (especially linear cracks and directional scratches) will change in direction, resulting in decreased generalization ability of the model trained based on fixed direction features and increased missed detection rate. Secondly, although the existing network architecture based on multi-scale feature fusion (such as FPN and U-Net variants) can integrate global semantics and local details, the fusion process lacks explicit modeling of the geometric topology and physical characteristics of the part itself, resulting in incomplete recognition and inaccurate positioning of large-size defects across image blocks or small defects at the edge of the block. Finally, the output defect map has problems such as fragmentation, ghosting, and inconsistent boundaries, making it difficult to meet the needs of high-precision industrial quality inspection. SUMMARY

[0004] The present application provides an industrial part defect detection method based on multi-scale feature fusion, which mainly aims to solve the problem of low accuracy in industrial part defect detection.

[0005] To achieve the above purpose, the present application provides an industrial part defect detection method based on multi-scale feature fusion, which comprises:

[0006] Obtaining the surface optical image of the industrial part to be detected, taking the geometric center of the industrial part to be detected as the reference, dividing the surface optical image into a plurality of non-overlapping image blocks, and constructing a local coordinate system corresponding to each image block;

[0007] Extracting the deep feature map and the middle feature map of each image block, performing feature upsampling on the deep feature map to obtain a semantic enhanced feature map, and performing channel recalibration on the semantic enhanced feature map based on the channel attention weight of the middle feature map to generate a semantic guided feature map;

[0008] Multi-scale fusion of the deep feature map, the semantic guided feature map and the middle feature map to generate a fusion feature map;

[0009] analyze the rotation direction between the fusion feature map of each image block and the pre-acquired defect prior feature vector, under the local coordinate system, according to the rotation direction, perform a multi-angle rotation alignment operation with the center of the industrial part to be detected as the center for each image block, and obtain a defect detection result of each image block;

[0010] fuse the defect detection results of all image blocks to obtain a pixel-level defect classification and positioning map of the industrial part to be detected.

[0011] The embodiment of the present application divides the image into image blocks in the local coordinate system according to the center of the part, so that the analysis of each local area is strictly associated with the overall geometric topology of the part; by analyzing the rotation direction between the fusion feature map and the defect prior feature vector, and performing a rotation alignment operation with the center of the part as the center for each image block, the detection direction sensitivity introduced by the change of the part posture or the local deformation is effectively eliminated, so that the recognition of the directional defect is more stable and reliable. The fusion feature map generated after the calibration and fusion of the multi-scale features under the guidance of the semantics has rich semantic discrimination and detail resolution. Finally, the detection results of all image blocks are fused in the unified global coordinate system based on the confidence and geometric consistency, effectively repairing the defect cracking and edge misplacement caused by blocking, and generating a complete, continuous and high-precision pixel-level defect classification and positioning map. Therefore, the industrial part defect detection method based on multi-scale feature fusion provided by the present application can solve the problem of low accuracy in industrial part defect detection. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 A flowchart of an industrial part defect detection method based on multi-scale feature fusion provided by an embodiment of the present application.

[0013] The implementation, functional characteristics and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0014] It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0015] The embodiment of the present application provides a kind of industrial parts defect detection method based on multi-scale feature fusion.The execution subject of the industrial parts defect detection method based on multi-scale feature fusion includes but is not limited to at least one of the electronic devices that can be configured to execute the method provided by the embodiment of the present application, such as server, terminal and the like.In other words, the industrial parts defect detection method based on multi-scale feature fusion can be executed by software or hardware installed in terminal device or server device, and the software can be a blockchain platform.The server includes but is not limited to: single server, server cluster, cloud server or cloud server cluster and the like.The server can be a stand-alone server, or a cloud server providing cloud services, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content distribution network (Content Delivery Network, CDN), and big data and artificial intelligence platform, etc.Basic cloud computing services.

[0016] Referring to Figure 1 As shown in the flowchart of the industrial parts defect detection method based on multi-scale feature fusion provided by an embodiment of the present application.In this embodiment, the industrial parts defect detection method based on multi-scale feature fusion includes:

[0017] S1, obtain the surface optical image of the industrial part to be detected, take the geometric center of the industrial part to be detected as the reference, divide the surface optical image into a plurality of non-overlapping image blocks, and construct the local coordinate system corresponding to each image block.

[0018] In the embodiment of the present application, the industrial part to be detected refers to various structural parts that need to be detected for surface defects in industrial production, including but not limited to machined gears, automobile engine blocks, aerospace connecting parts, etc.;The surface optical image refers to the gray or color digital image of the part collected by the industrial linear array or area array CCD camera under uniform illumination conditions.

[0019] In detail, the surface optical image of the industrial part to be detected can be obtained from the pre-stored storage area by computer statements (such as Java statements, Python statements, etc.) with data grabbing function, wherein the storage area includes but is not limited to database, blockchain.

[0020] In the embodiment of the present application, the image block refers to a rectangular sub-region segmented from the whole image;The local coordinate system is a two-dimensional rectangular coordinate system, which is defined independently for each image block, and is used to describe the position relationship of the pixels in the block.

[0021] In the embodiment of the present application, the geometric center of the industrial part to be detected is taken as a reference, the surface optical image is divided into a plurality of non-overlapping image blocks, and a local coordinate system corresponding to each image block is constructed, comprising:

[0022] Based on the three-dimensional model topology of the industrial part to be detected, the surface optical image is mapped to a two-dimensional development diagram;

[0023] According to the distribution of the functional area boundary and the stress concentration area of the surface of the industrial part to be detected, the two-dimensional development diagram is non-uniformly meshed to obtain a plurality of non-overlapping image blocks;

[0024] A local coordinate system with its own centroid as the origin is established for each divided image block, and the axis direction of the local coordinate system is aligned and calibrated with the principal stress direction or typical texture direction of the corresponding area of the industrial part to be detected, to obtain a local coordinate system corresponding to each image block.

[0025] In detail, the geometric center refers to the center point coordinate calculated based on the three-dimensional model design parameters of the part, which can be solved by the mean value of the vertex coordinates of the three-dimensional model. The three-dimensional model topology is obtained by analyzing the three-dimensional design file of the part in STEP format, which refers to the connection relationship and geometric composition of the vertices, edges and faces in the three-dimensional model of the part. The computer-aided design three-dimensional model of the part to be detected is loaded, the parametric surface information of the three-dimensional model is used, and the UV development algorithm is used to map the three-dimensional surface of the part to a two-dimensional parameter plane to generate a two-dimensional development diagram. That is, through the texture mapping algorithm, the collected surface optical image is accurately mapped to the corresponding position of the two-dimensional development diagram according to the corresponding relationship between the three-dimensional model surface elements and the two-dimensional development diagram, thereby establishing a one-to-one correspondence between the three-dimensional part surface points and the two-dimensional image pixels. For example, the axial development method is used for cylindrical surface, the sector development method is used for conical surface, and the triangular slicing approximation development method is used for complex free-form surface. For a cylindrical bearing ring, its outer cylindrical surface can be developed into a rectangular two-dimensional diagram, so that the surrounding curved surface image is converted into a planar image for processing, eliminating the influence of curved surface deformation on subsequent analysis.

[0026] Specifically, on the obtained two-dimensional unfolding diagram, instead of uniform meshing, different functional area boundaries are identified according to the design knowledge of the part, such as sealing surfaces, mounting hole periphery, raceway surfaces, etc. Meanwhile, combined with finite element analysis simulation results or empirical data, the stress concentration areas of the part under typical working conditions are calibrated, such as the corner transition and the groove bottom. When meshing, denser meshing is used in the functional key areas and stress concentration areas to generate smaller image blocks, so as to realize higher spatial resolution analysis in these defect-prone areas; sparser meshing is used in non-key or flat areas. For example, in the bearing raceway area, which is prone to fatigue spalling, it is divided into dense small image blocks; while in the non-mating area of the bearing end face, larger image blocks are divided, so as to realize the optimal allocation of computing resources and focus on the key areas.

[0027] Further, for each image block obtained by division, the average value of all pixel coordinates of the image block is calculated to obtain the centroid of the image block, and the centroid is taken as the origin of the local coordinate system thereof. The principal stress direction of the three-dimensional area corresponding to the image block under the main stress state is obtained, or the typical texture direction (such as the forging streamline direction) of the area in the image of the sample without defects is analyzed. The X-axis or Y-axis of the local coordinate system is adjusted to be parallel to the principal stress direction or the texture direction. For example, for an image block located on a stress rib of a part, the X-axis of the local coordinate system thereof is set to be consistent with the principal stress direction of the rib (usually the length direction of the rib), so that the subsequent feature extraction and rotation analysis have a clear physical meaning reference, and the sensitivity to defects with directional features (such as cracks along the principal stress direction) is enhanced.

[0028] Further, the image block set closely combined with the physical structure of the part is obtained, and each image block has a local coordinate system with a specific physical direction, thereby laying a foundation for subsequent fine feature analysis and defect matching in each independent and physically meaningful sub-area.

[0029] S2, extract a deep feature map and a middle feature map of each image block, perform feature up-sampling on the deep feature map to obtain a semantic enhanced feature map, and perform channel recalibration on the semantic enhanced feature map based on a channel attention weight of the middle feature map to generate a semantic guided feature map.

[0030] In the embodiment of the application, the deep feature map refers to a feature tensor extracted by a deep convolutional layer of a neural network, which has rich semantic information but low spatial resolution, and can understand the global structure and high-level defect category of the image block. The middle feature map refers to a feature tensor extracted by a middle convolutional layer of a neural network, which retains more spatial detail information but has lower semantic abstraction, and is sensitive to local texture, edges and other details.

[0031] In the embodiment of the present application, the deep feature map and the middle feature map of each image block are extracted, comprising:

[0032] A feature extraction network representing a parallel double branch is constructed, and a lateral connection is established between the double branches, representing that the second branch in the feature extraction network can selectively receive a semantic gating signal from the first branch or related to the current spatial position during the extraction process;

[0033] According to the type of the region to which each image block belongs on the industrial part to be detected and the semantic gating signal, the weight of the convolution channel number of the feature extraction network of the double branch is dynamically adjusted;

[0034] The first convolution kernel in the first branch of the adjusted feature extraction network and the deep layer stack capture the global structure context of each image block, and the deep feature map of each image block is generated according to the global structure context;

[0035] The second convolution kernel in the second branch of the adjusted feature extraction network and the dense connection focus on the local micro-texture and edge details of each image block, and the middle feature map is obtained.

[0036] In detail, the parallel double branch feature extraction network refers to a neural network containing two structurally independent and parallel running feature extraction branches. An improved parallel double branch encoder architecture is adopted. The first branch (deep branch) focuses on global structure feature extraction, which is stacked by several convolution layers and pooling layers using a first convolution kernel (such as 5x5, 7x7), aiming to quickly expand the receptive field and capture the global structure of the image block. The second branch (middle branch) focuses on local detail feature extraction, which is composed of multiple dense connection blocks using a second convolution kernel (such as 3x3), aiming to preserve the fine spatial details of the first branch. The two branches are not completely independent, and a lateral connection is provided at a specific network level. The connection contains a semantic gating mechanism: first, the feature map output by the first branch at this layer is subjected to global average pooling and a lightweight fully connected layer operation to generate a semantic context vector; then, the vector passes through a sigmoid activation function to generate a gating signal; finally, the gating signal is multiplied with the feature map of the corresponding layer of the second branch channel by channel, so that the second branch can adaptively enhance or suppress the detail response of certain channels when extracting details according to the global semantic context provided by the first branch, for example, suppressing texture detail noise in areas suspected to be smooth background, and enhancing detail response in areas suspected to be defect edges.

[0037] Specifically, according to the region type identified when the image block is divided (such as the raceway surface and the mounting hole edge), a region type coding vector is provided for the network. At the same time, combined with the current image block content complexity reflected by the generated semantic gating signal, through a small adjustment network, two groups of weight coefficients are dynamically generated, which act on the convolution channel number of the first branch and the second branch respectively. For example, for the raceway surface which is sensitive to small spalling defects, the channel weight of the second branch (detail branch) will be increased, and the channel weight of the first branch can be fine-tuned according to the semantic gating signal (if the signal indicates the content complexity), such as the maximum weight = the basic weight x (the semantic gating signal value + 0.5), so that the feature extraction network can adaptively allocate feature expression capacity according to the task demand, wherein the semantic gating signal refers to a signal for controlling information transmission, which can selectively allow the semantic information of the first branch to pass into the second branch according to the current feature extraction demand, and its essence is a weight coefficient vector between 0 and 1.

[0038] Further, the first convolution kernel refers to a large-size convolution kernel used to expand the receptive field and capture global-range feature information; the deep stack refers to stacking multiple convolution layers and pooling layers, and realizing deep abstraction of features through progressive calculation of multiple layers of networks. The global structural context refers to feature information reflecting the overall structural relationship in the image block, such as hole distribution and contour morphology on the surface of the part. The preprocessed image block is input into the dynamically adjusted feature extraction network, the first branch gradually abstracts the image content through the first convolution kernel and the deep structure, and finally outputs a tensor with a smaller spatial size but a larger number of channels at the end of the branch, that is, a deep feature map, which can effectively represent whether the image block contains defects and the approximate category of the defects. For example, in the adjusted first branch network, first, a 7x7 first convolution kernel is used for convolution operation on the image block, and the step size is set to 2, so that the receptive field covers 1 / 4 of the image block, and the large-range structural features are preliminarily captured; then, the features are refined through 2 layers of 3x3 convolution layers, and then reduced in spatial resolution and reserved key features through a maximum pooling layer (pooling kernel 2x2); then, the features are deeply abstracted through 3 layers of convolution layers stacked in the deep layer (each layer is a 3x3 convolution kernel, and the number of channels is 64, 128, and 256 in turn), and the global structural context information is gradually extracted; finally, the feature channel number is compressed to 64 dimensions through a 1x1 convolution layer, and a deep feature map with a size of 32x32x64 is generated. At the same time, the second branch works in parallel, and the second convolution kernel and the dense connection are used, wherein the second convolution kernel refers to a small-size convolution kernel used to accurately capture the detail features of the local area, and the dense connection refers to the full connection relationship between the network layers, and the input of each network layer contains the output of all previous layers, which is used to fully utilize the shallow detail features. The detail features are extracted and fused at multiple scales, and finally a tensor with higher spatial resolution and rich details is output, that is, a middle-layer feature map. Since it is regulated by the semantic gating signal from the first branch, the details retained in the middle-layer feature map are more discriminative details associated with global semantics. The local microtexture refers to the fine lines on the surface of the part, such as machining marks and scratches; and the edge detail refers to the boundary contour of different regions on the surface of the part, such as the edge of a hole and the edge of a groove.

[0039] Further, after the deep feature map and the middle-layer feature map are extracted, since the spatial resolutions of the two are different, they cannot be directly and effectively interacted and fused. The deep feature map has strong semantics but rough spatial information, and the middle-layer feature map is rich in details but lacks high-level semantic guidance. Therefore, the deep feature map needs to be upsampled to match the resolution and guided and enhanced by the information of the middle-layer feature map.

[0040] In the embodiment of the present application, the semantic enhanced feature map refers to a feature map obtained by restoring the resolution of the deep feature map and incorporating detail information.

[0041] In the embodiment of the present application, the feature up-sampling on the deep feature map is performed to obtain a semantic enhancement feature map, including:

[0042] Point-by-point convolution is performed on the deep feature map to generate spatially adaptive up-sampling kernel parameters;

[0043] The spatially adaptive up-sampling kernel parameters are used to perform irregular interpolation operation on each local region of the deep feature map to obtain an initial up-sampling feature map;

[0044] The high-frequency detail guide signal in the middle layer feature map is extracted, and the initial up-sampling feature map is element-by-element fused with the high-frequency detail guide signal to generate a detail enhancement feature map;

[0045] Feature pyramid pooling is performed on the detail enhancement feature map to obtain a plurality of context features of different scales, and the plurality of context features of different scales are fused to generate a semantic enhancement feature map.

[0046] In detail, point-by-point convolution refers to using a 1x1 size convolution kernel to perform convolution operation on the deep feature map, which is used for feature channel fusion and dimension adjustment without changing the spatial size of the feature map. Unlike traditional fixed interpolation kernel (such as bilinear) up-sampling, a learnable and spatially adaptive up-sampling method is used. A point-by-point convolution layer (1x1 convolution) is applied to the deep feature map, and the number of output channels of the convolution layer is determined according to the required up-sampling multiple and kernel size. For example, to perform 2 times up-sampling and use a 3x3 interpolation kernel, the number of output channels is 2x2x3x3=36. The 36 parameters are rearranged to generate an independent 3x3 up-sampling kernel parameter for each spatial position of the deep feature map, that is, different semantic content regions will use different kernels for up-sampling, for example, using a smooth kernel in a smooth region and using a kernel that can sharpen the edge in an edge region.

[0047] Specifically, the local region refers to a 3x3 neighborhood region centered on each pixel in the deep feature map. Based on the dedicated up-sampling kernel generated at each position, convolution operation is performed on the corresponding local region of the deep feature map, such as calculating the sum of the products of 9 pixel feature values and corresponding up-sampling kernel weights to obtain the coordinates of the up-sampled feature map pixel feature values of the deep layer feature map in this way, so as to enlarge the deep layer feature map to the same spatial size as the middle layer feature map, and obtain an initial up-sampling feature map, so that the structure information related to semantics can be better maintained and recovered in the up-sampling process. From the middle layer feature map, a high-pass filter (such as Laplacian operator convolution) or a shallow convolutional network is used to extract high-frequency components thereof as a detail guide signal. The high-frequency detail guide signal refers to a feature signal reflecting high-frequency information such as edges and textures extracted from the middle layer feature map. The initial up-sampling feature map and the detail guide signal are added element by element or spliced and then convolved to realize fusion, so that the up-sampled feature map not only recovers the structure but also embeds real detail textures from the middle layer feature, generating a detail-enhanced feature map. The detail-enhanced feature map is subjected to pyramid pooling operation, that is, a plurality of pooling windows of different sizes (for example, 1x1, 2x2, 4x4) are used to pool the detail-enhanced feature map, obtaining a plurality of feature vectors of different scales. The feature vectors are restored to the original size by up-sampling or interpolation, and then spliced with the original detail-enhanced feature map, and finally fused by a convolution layer, which aggregates multi-scale context information, so that the generated semantic-enhanced feature map has rich details, clear structure and multi-scale context perception ability.

[0048] Further, after obtaining the semantic-enhanced feature map, although the details are fused, the importance of each channel feature is equal. In order to further optimize the feature expression, the channel importance information contained in the middle layer feature map needs to be used to recalibrate it, so as to generate a more accurate semantic guide feature map.

[0049] In the embodiment of the present application, the channel attention weight is a vector, and each element value of the vector corresponds to the importance score of a feature Figure 1 The semantic guide feature map refers to a feature map in which the semantic information and the detail information are optimally fused after channel attention recalibration.

[0050] In the embodiment of the present application, the channel attention weight based on the middle layer feature map recalibrates the channel of the semantic-enhanced feature map to generate a semantic guide feature map, comprising:

[0051] The global average pooling and the global maximum pooling of the middle layer feature map are aggregated to obtain an aggregated feature vector, and the aggregated feature vector is converted into an initial channel attention weight by using a multi-layer perceptron with a bottleneck structure;

[0052] The global semantic importance vector of the semantic-enhanced feature map is extracted, the initial channel attention weight is multiplied by the global semantic importance vector by gating to generate an enhanced channel attention weight;

[0053] The enhanced channel attention weight is used as a modulation coefficient to reweight each channel of the semantic enhancement feature map;

[0054] The reweighted feature map is added to the edge structure feature in the middle layer feature map through a preset residual connection, and the added features are activated through a preset activation function to generate a semantic guidance feature map.

[0055] In detail, global average pooling and global maximum pooling are respectively performed on the middle layer feature map in the spatial dimension to obtain two vectors summarizing global information. After adding or concatenating the two vectors, they are input into a multi-layer perceptron with a bottleneck structure. The perceptron compresses the channel number to 1 / r (r is the compression ratio, for example, 16) through a fully connected layer, and then recovers to the original channel number after ReLU activation. Finally, a vector with a value range of 0 to 1 is generated through a Sigmoid activation function, which is the initial channel attention weight. This weight reflects the importance of each feature channel to the current image block content from the perspective of the middle layer details. At the same time, the semantic enhancement feature map is globally averaged pooled, and a separate lightweight fully connected layer is used to generate a vector reflecting the global semantic distribution of the semantic enhancement feature map, which is the global semantic importance vector. The initial channel attention weight and the global semantic importance vector are multiplied element by element (gating operation), so that the global semantic distribution of the semantic enhancement feature map is used to verify and modulate the channel importance evaluation from the middle layer feature, avoiding the influence of local noise in the middle layer feature on the accuracy of weight distribution, thereby generating more robust and accurate enhanced channel attention weight.

[0056] Specifically, the modulation coefficient refers to a coefficient for adjusting the intensity of the feature channel, i.e., the enhanced channel attention weight. The enhanced channel attention weight is regarded as a modulation coefficient vector, which is multiplied by each channel of the semantic enhancement feature map. The channel features with high importance are enhanced, and the channel features with low importance are suppressed. This realizes the refinement of the semantic enhancement feature map in the channel dimension, making its feature expression more focused on the information most relevant to the current image block content. In order to further strengthen the spatial details, significant edge structure features are extracted from the middle layer feature map through an edge detection convolution kernel. Through a residual connection (i.e., a jump connection), the feature map after channel reweighting is added to the extracted edge structure features element by element. The addition result is output as the final semantic guidance feature map through a nonlinear activation function (such as ReLU). The feature map fuses the optimized and calibrated deep semantic information and the strong edge structure information from the middle layer, where the edge structure feature refers to the structure features such as part surface edges and contours extracted from the middle layer feature map, providing a feature representation with strong discrimination for subsequent defect matching.

[0057] Further, three different feature maps with different emphases are generated for each image block: deep feature map (global semantics), middle feature map (local details), and semantic guidance feature map (optimized fusion). In order to comprehensively utilize the multi-scale and multi-semantic level information, different feature maps are deeply fused.

[0058] S3, multi-scale fusion of the deep feature map, the semantic guidance feature map and the middle feature map is performed to generate a fused feature map.

[0059] In the embodiment of the application, the fused feature map refers to a feature map obtained after multi-scale feature fusion, which can comprehensively reflect the part surface defect information and has global semantics, local details and edge structure characteristics.

[0060] In the embodiment of the application, the multi-scale fusion of the deep feature map, the semantic guidance feature map and the middle feature map to generate a fused feature map comprises:

[0061] The deep feature map, the semantic guidance feature map and the middle feature map are respectively input into a spatial attention module to generate their respective spatial importance masks;

[0062] Based on the spatial importance masks, the deep feature map, the semantic guidance feature map and the middle feature map are weighted and summed to obtain a preliminary fused feature;

[0063] The preliminary fused feature is channel fused by using a pre-constructed fusion decision network to obtain a three-channel pixel-level fusion weight map;

[0064] According to the pixel-level fusion weight map, the deep feature map, the semantic guidance feature map and the middle feature map are pixel-level soft selected and mixed to generate a fused feature map.

[0065] In detail, the spatial importance mask is a matrix with the same spatial size as the feature map, and the value of each position of the matrix represents the importance of the spatial position in the corresponding feature map. A same spatial attention module is configured for each input feature map (deep layer, semantic guidance, middle layer). The module usually consists of a channel dimension reduction convolution, a convolution for calculating spatial attention (or using a self-attention mechanism), and a Sigmoid activation function. The spatial attention module takes a feature map as input and outputs a single-channel mask, each pixel value of which is between 0 and 1, identifying the feature response intensity or importance of the input feature map at the position. For example, for the semantic guidance feature map, a high response is generated in the edge region of the potential defect, and the value of the corresponding spatial importance mask in the region is close to 1. For example, in the spatial importance mask corresponding to the deep layer feature map, the value of the center region of the part structure is relatively high (such as 0.9), and the value of the edge region is relatively low (such as 0.3); in the mask of the semantic guidance feature map, the value of the region where the defect may exist is relatively high; in the mask of the middle layer feature map, the value of the region with rich edge details is relatively high.

[0066] Specifically, the preliminary fusion feature is an intermediate feature obtained by weighted summation. The three generated spatial importance masks are respectively multiplied with the original feature maps corresponding to the masks, to realize re-weighting of the spatial dimensions of the respective feature maps, and then the three spatially weighted feature maps are added element by element to obtain the preliminary fusion feature, so as to realize primary fusion based on the spatial activation degree of each feature map. The fusion decision network is a small neural network for learning to generate pixel-level fusion weights. The pixel-level fusion weight map is a three-channel weight tensor, and each channel corresponds to a fusion weight of each input feature map at each pixel point. The fusion decision network can be composed of several consecutive convolution layers and a final Sigmoid activation layer. The obtained preliminary fusion feature is input into the fusion decision network, and the learning goal of the fusion decision network is to determine whether the final feature expression of each pixel point should depend more on the deep layer feature, the semantic guidance feature or the middle layer feature according to the content of the preliminary fusion feature at each pixel point. The fusion decision network outputs a three-channel feature map, i.e. a pixel-level fusion weight map, each pixel position of which has three weight values corresponding to the three input feature maps, and the sum of the three weights is 1 (guaranteed by Sigmoid and normalization). For example, in the flat background region of the image block, the network can assign a high weight to the deep layer feature map; in the fine edge of the suspected defect, a high weight is assigned to the semantic guidance feature map; and in the complex texture region, a certain weight is assigned to the middle layer feature map.

[0067] Exemplarily, the preliminary fusion feature (64x64x64) is input into the fusion decision network: high-level semantic information of the fusion feature is extracted through a depth separable convolution layer, and the number of channels is adjusted to 128, 64 and 32 in turn; the feature map is converted into a 32-dimensional vector through global average pooling, and input into a full connection layer; the first layer full connection layer (32->64) expands the feature, and the second layer full connection layer (64->3x64x64) outputs a pixel-level fusion weight map; the three channels of the weight map correspond to the weights of the three types of original feature maps respectively, the pixel value in each channel ranges from 0 to 1, and the sum of the weights of the three channels at each pixel position is 1; for example, the weight map value of a certain pixel position is (0.1, 0.7, 0.2), which indicates that the contribution degree of the semantic guidance feature at this position is 70%, the deep feature is 10%, and the middle layer feature is 20%.

[0068] In addition, soft selection and mixing refer to weighted combination of multiple features according to weights. The deep feature map, the semantic guidance feature map and the middle layer feature map are respectively multiplied with the corresponding weight channels in the pixel-level fusion weight map to obtain three weighted feature maps, and the three weighted feature maps are added element by element to finally generate a fusion feature map, so as to realize adaptive feature selection and mixing at the pixel level, so that each position is dominated by the most relevant feature source, thereby generating a fusion feature map with maximum discrimination and minimum information redundancy, which is directly used as a basis for subsequent defect matching.

[0069] Further, the fusion feature map is a highly abstract and robust representation of the content of the current image block. In order to detect defects, it needs to be compared with known defect patterns. However, since the part may have a posture rotation, direct comparison may fail. Therefore, the rotation is analyzed and compensated first, and then accurate matching is performed.

[0070] S4, analyze the rotation direction between the fusion feature map of each image block and the pre-acquired defect prior feature vector, and perform a multi-angle rotation alignment operation with the center of the industrial part to be detected as the center in the local coordinate system according to the rotation direction to obtain a defect detection result of each image block.

[0071] In the embodiment of the application, the rotation direction refers to the angle direction required for the current image block feature to be aligned with the prior defect feature in direction.

[0072] In the embodiment of the application, the analysis of the rotation direction between the fusion feature map of each image block and the pre-acquired defect prior feature vector comprises:

[0073] Retrieving a plurality of candidate defect prior feature vectors associated with the part region and material attribute to which the current image block belongs from a pre-constructed defect knowledge graph;

[0074] compress the fusion feature map into a query feature vector by global pooling;

[0075] calculate the cosine similarity and the included angle between the query feature vector and each candidate defect prior feature vector in the multi-dimensional spherical space;

[0076] select the candidate defect prior feature vector with the smallest included angle and the cosine similarity exceeding the preset threshold as a target vector, and set the spatial included angle direction of the target vector and the query feature vector as a rotation direction;

[0077] When there is no candidate defect prior feature vector meeting the condition, calculate a default direction of each image block based on the local texture main direction of the image block, and set the default direction as the rotation direction.

[0078] In detail, the defect knowledge graph is a knowledge base organized in a graph structure, with nodes representing defect feature vectors or types and edges representing the relationship between them; the defect prior feature vector is a fixed-dimensional vector learned from historical defect samples, representing the essential features of a certain type of defect. The defect knowledge graph is constructed in the offline stage and stores feature vectors of various defect types (such as cracks, pores, scratches) in different part regions (such as raceways, end faces) and different materials (such as bearing steel, aluminum alloy). Each feature vector is associated with metadata. When processing an image block, based on the known part region and overall material properties of the image block, the nodes with the same or similar metadata in the knowledge graph are queried to obtain their associated defect feature vectors as the candidate set, thereby ensuring that the comparison is conducted within the most relevant prior knowledge range.

[0079] Specifically, for the fusion feature map of the current image block, a global average pooling operation is performed in the spatial dimension to compress it into a one-dimensional vector, i.e., a query feature vector. The query feature vector is a summary representation of the fusion feature map, which summarizes the fusion feature information of the entire image block. The multi-dimensional spherical space is an abstract space that regards feature vectors as points on a spherical surface, and the included angle between vectors is used to measure the directional difference. The query feature vector and each candidate defect prior feature vector are normalized by L2 norm, so that their lengths are 1, and they are projected onto the multi-dimensional spherical space. The cosine similarity (i.e., the dot product) between the query feature vector and each candidate vector is calculated, which is between -1 and 1, and the closer to 1 indicates the more similar the direction. At the same time, the spatial included angle (inverse cosine value) between them is calculated. For example, the cosine similarity of the query feature vector and the candidate vector is , and the included angle is . By traversing all candidate vectors, the included angle The smallest one. Simultaneously, check if the cosine similarity between the candidate vector and the query feature vector exceeds a preset confidence threshold (e.g., 0.8). If the condition is met, the candidate vector is determined as the target vector. At this point, the direction from the query feature vector to the target vector, i.e., in spherical space, needs to be... Rotate to The same direction of rotation, i.e. the direction information, can be converted into a rotation angle on the image plane, transforming the feature matching problem into an alignment problem in spherical space, and explicitly decoupling the feature content from the feature direction.

[0080] Furthermore, if the cosine similarity of all candidate vectors is below a threshold, it indicates that the features of the current image patch may differ significantly from known defect patterns, or it may be defect-free. In this case, its principal orientation is calculated by analyzing the local texture of the image patch itself (e.g., using a gray-level co-occurrence matrix or gradient histogram), and this orientation is used as the default rotation direction, ensuring that subsequent operations can still be performed even in the absence of explicit prior matching.

[0081] Furthermore, after determining the rotation direction (i.e. the rotation angle that needs to be compensated), a rotation operation with the global center of the part as the physical constraint needs to be performed in the local coordinate system of the image block itself, so that the features of the image block and the target defect features are optimally aligned, thereby achieving accurate matching.

[0082] In this embodiment of the invention, the defect detection result is information such as the location and confidence level of the defect within the image block.

[0083] In this embodiment of the invention, the step of performing a multi-angle rotation alignment operation on each image block in the local coordinate system according to the rotation direction, with the center of the industrial part to be inspected as the center, to obtain the defect detection result of each image block includes:

[0084] Using the global geometric center of the industrial part to be inspected as the rotation center, calculate the polar coordinates of the origin of the local coordinate system of the current image block relative to the rotation center;

[0085] In the local coordinate system, multiple discrete rotation angles are generated according to the rotation direction;

[0086] Based on the fused feature map of the image block, perform a rotation operation on the affine transformation matrix of each discrete rotation angle to generate a rotation feature map sequence;

[0087] The rotated feature map sequence is spatially convolved and cross-correlated with the target vector to obtain multiple response maps;

[0088] select the rotation angle with the highest peak value in the response map as the optimal alignment angle, and take the response peak position and intensity under the optimal alignment angle as the preliminary defect detection result of the image block;

[0089] perform non-maximum suppression on the preliminary defect detection result to obtain the defect detection result of the image block.

[0090] In detail, the coordinates of the known global geometric center of the part in the original image are obtained At the same time, the coordinates of the local coordinate system origin (i.e. the centroid) of the current image block in the original image are obtained The vector is calculated The vector is converted into polar coordinate form wherein is the distance, is the initial polar angle relative to the global coordinate system, thereby establishing the relationship between the local coordinate system of the image block and the global rotation center.

[0091] Specifically, the rotation direction (assuming quantized to a reference angle ) is taken as the center, and a series of discrete rotation angles are generated within a small range (for example to ) on the left and right sides, for example, with a step of 1 degree. These angles are defined under the local coordinate system of the image block itself, and the angle range covers the possibility of alignment error.

[0092] In the embodiment of the application, the rotation feature map sequence refers to a three-dimensional array containing standardized feature maps under multiple rotation angles.

[0093] In the embodiment of the application, the rotation operation on the affine transformation matrix of each discrete rotation angle according to the fusion feature map of the image block to generate a rotation feature map sequence comprises:

[0094] calculating a rotation matrix according to the discrete rotation angle, and calculating an affine transformation matrix according to the rotation matrix and the offset amount of the local coordinate system origin and the global geometric center;

[0095] geometrically transforming the fusion feature map according to each affine transformation matrix to obtain a rotated feature map;

[0096] padding the edge pixels generated by symmetric replication for each rotated feature map, and performing normalization processing on each channel of each rotated feature map to obtain a standardized feature map;

[0097] stacking all standardized feature maps generated under different rotation angles of the same image block according to a preset angle sequence to obtain a rotation feature map sequence.

[0098] In detail, for each discrete rotation angle , the transformation matrix of rotating around the local coordinate system origin is . However, in order to make the rotation operation physically centered at the global part center O, a composite affine transformation matrix needs to be constructed. This matrix is equivalent to: first translating the local coordinate system origin to the global center O (translation matrix T1), then rotating around O (rotation matrix ), and finally translating the center O back to position (translation matrix T2). Therefore, the complete affine transformation matrix , thus ensuring that each point on the image block moves along a circular arc centered at O when rotated, consistent with the physical rotation model of the part. For the fused feature map (considered as a special image), use the calculated affine transformation matrix to perform geometric transformation by methods such as bilinear interpolation to generate the rotated feature map .

[0099] Specifically, due to the blank areas appearing in the corners of the rotated feature map, symmetric replication is used to fill these areas with pixel values from the edges of the feature map to maintain the continuity of the feature map's size and boundary information. Then, each channel of the rotated feature map is independently standardized (e.g., subtract the mean and divide by the standard deviation) to eliminate the weak brightness distribution changes that may be caused by geometric transformation, ensuring the numerical distribution consistency of the feature map under different rotation angles. The standardized rotated feature maps generated for all discrete rotation angles are stacked in the batch dimension according to the angle order to form a feature map sequence (or tensor) with the shape [number of angles, number of channels, height, width], thus obtaining the sequence of the image block fused feature map under a series of rotation angles, i.e., the rotated feature map sequence.

[0100] Further, the rotated feature map sequence is matched with the target defect prior feature vector in the spatial dimension. The target vector (dimension ) is reshaped to be considered as a 1x1xC convolution kernel. This kernel is spatially convolved (actually cross-correlation operation) with the rotated feature map sequence (dimension ) angle by angle. For the feature map under each rotation angle in the sequence, a two-dimensional response map (dimension ) is obtained after convolution calculation. Finally, N response maps are obtained, forming a response map sequence, where the high-value area on the response map indicates that the feature map and the target defect pattern are highly matched in the corresponding spatial position at that rotation angle. By traversing all response maps, the global maximum (peak) and its position are found. The peak value corresponding to the rotation angle is recorded (i.e. the optimal alignment angle), and the spatial coordinates of the peak on the corresponding response map and the peak intensity The preliminary position of the defect in the local coordinate system of the current image patch is denoted as The confidence of the existence of the defect is denoted as The estimated local rotation compensation is denoted as These three together constitute the preliminary result of defect detection. Since feature matching can produce multiple local response peaks around the true position of the defect, a non-maximum suppression algorithm is applied. A neighborhood window is set around the preliminary result position, and only the peak point with the highest confidence in the neighborhood is retained, and other lower peak points are suppressed, so as to obtain the refined defect position and confidence information without redundancy, i.e. the final defect detection result of the image patch.

[0101] Further, the defect detection under rotation compensation is independently completed for each image patch. However, these results are scattered in various local coordinate systems and image patches, and in order to obtain the seamless defect distribution map of the whole part, all local results need to be fused into a unified global view.

[0102] S5, fuse the defect detection results of all image patches to obtain a pixel-level defect classification and positioning map of the industrial part to be detected.

[0103] In the embodiment of the application, the pixel-level defect classification and positioning map is a visual result finally containing defect categories, positions and contours.

[0104] In the embodiment of the application, the fusion of the defect detection results of all image patches to obtain the pixel-level defect classification and positioning map of the industrial part to be detected comprises:

[0105] A global image coordinate system is established with the geometric center of the industrial part to be detected as the origin and the design reference direction of the industrial part to be detected as the axis direction;

[0106] The original position of each image patch in the surface optical image is identified, and the defect detection result of each image patch is mapped back into the global image coordinate system according to the original position, to obtain a global defect distribution field;

[0107] For a boundary region in the global defect distribution field that is jointly covered by multiple adjacent image patches, the spatial distribution confidence of the fusion feature map response peak intensity of each image patch in the boundary region is calculated;

[0108] Based on the spatial distribution confidence, adaptive weight allocation and probability fusion are performed on the defect detection results of different image patches in the boundary region, to generate a global defect probability map that eliminates the boundary effect;

[0109] According to the theoretical structure boundary of the industrial part to be detected, the boundary region is geometrically constrained and filtered to obtain a defect region contour, and a class label of the defect region contour is determined according to the global defect probability map;

[0110] The defect region contour and the class label are superimposed on the surface optical image to generate a visual pixel-level defect classification and positioning map.

[0111] In detail, the global image coordinate system is a unified coordinate system established on the original part image and defined by the part geometric center and the design reference direction. On the original surface optical image, the part geometric center is taken as the coordinate origin O, and the design reference direction (for example, the main shaft direction or the normal direction of a key positioning surface) of the part is taken as the X-axis direction of the global image coordinate system to establish a right-hand coordinate system, which provides an absolute spatial reference framework for the entire part. For the defect detection result (local coordinate position) of each image block, the local coordinate is converted back to the actual pixel coordinate of the image block in the original image by using the correspondence between the local coordinate system and the original image position. The converted defect coordinates (and the accompanying confidence and class information) of all image blocks are collected and projected into the global image coordinate system to form a discrete defect information set covering the entire part, i.e., a global defect distribution field, wherein the global defect distribution field is a discrete defect point set or a probability distribution formed by mapping all local detection results back to the global coordinate. At this time, in the image block boundary area, the same physical defect may be detected by two or more adjacent image blocks, thereby generating multiple repeated or slightly offset detection points with close positions in the global field.

[0112] Specifically, the boundary overlap region is analyzed in detail, and for each boundary pixel, it is traced back which image blocks cover it. For each image block covering the pixel, the response peak intensity it produces at the pixel's corresponding position (or neighborhood) when performing spatial convolution cross-correlation calculation is obtained, and the spatial distribution smoothness and consistency of the response peak intensity in the boundary region is analyzed, for example, the local variance is calculated, the smaller the variance, the more stable and reliable the detection result of the image block at the boundary, and thus the spatial distribution confidence is obtained, wherein the spatial distribution confidence is an index for measuring the reliability of the detection result of the image block in the boundary region. For each pixel in the boundary region, according to the spatial distribution confidence of each image block covering it at the pixel, appropriate fusion weights are assigned to them. The image block with high confidence has high weight. The defect probability estimate (which can be obtained by normalizing the response peak intensity) given by the image block at the pixel is weighted and averaged to obtain the final fusion probability of the pixel belonging to a defect, wherein the global defect probability map is a probability map of each pixel belonging to a defect obtained by fusion. For the non-boundary region (i.e. the internal region covered by only one image block), the detection probability of the image block is directly used, thereby generating a global defect probability map in the whole image range, which is smooth and natural in transition at the boundary, effectively eliminating the discontinuity or seam phenomenon at the boundary caused by independent processing of the blocks.

[0113] Further, the geometric constraint filtering refers to a process of eliminating unreasonable defect regions by using part theoretical structure. The CAD design drawing or theoretical contour mask of the part is loaded, and the generated global defect probability map is compared with the theoretical contour, and the suspected defect probability falling outside the theoretical part region is set to zero. Meanwhile, for the high-probability region connected in the probability map, the boundary contour thereof is obtained by a contour extraction algorithm (such as Marching Squares). According to the category associated with the most matched target defect vector of the region in the defect knowledge graph, a category label (such as crack, pore) is assigned to the region. The extracted defect region contour and the category label thereof are rendered and superimposed onto the original optical image of the industrial part to be detected in the form of graphic overlay (such as contour lines of different colors, transparent fill color blocks and text labels), and finally a pixel-level defect classification and positioning map is generated, which is intuitive and can be directly interpreted by the quality inspection personnel, and the whole automatic detection process is completed. The method is especially suitable for precision parts with features of rotation body or complex three-dimensional structure, and under the imaging conditions of strong noise, uneven illumination and posture deviation in the industrial field, the method exhibits superior detection consistency, high recall rate and low false alarm rate.

[0114] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0115] Thus, the embodiments should be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the description given above, all changes which come within the meaning and range of equivalency of the claims are embraced within their scope.

[0116] Embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results.

[0117] Furthermore, it is clear that the word "comprising" does not exclude other elements or steps, and that the singular also includes the plural and vice-versa unless explicitly stated otherwise. The mention of one or more reference signs in the description with respect to a particular figure is only a reference to the figure in question and does not mean that the figure in question must necessarily contain or describe all the elements or steps mentioned in the description.

[0118] Finally, it should be noted that the above-mentioned embodiments illustrate rather than limit the application, since the scope of the application is indicated by the appended claims rather than by the description given above, and that, within said scope, equivalent changes can be made thereto without departing from the spirit and scope of the application.

Claims

1. A method for detecting defects of industrial parts based on multi-scale feature fusion, characterized in that, The method comprises: acquiring a surface optical image of an industrial part to be detected, taking the geometric center of the industrial part to be detected as a reference, dividing the surface optical image into a plurality of non-overlapping image blocks, and constructing a local coordinate system corresponding to each image block; a feature extraction network representing parallel double branches is constructed, a lateral connection is established between the double branches, the second branch in the feature extraction network is selectively capable of receiving a semantic gating signal from the first branch or related to the current spatial position during the extraction process, the convolution channel weight of the feature extraction network of the double branches is dynamically adjusted according to the type of the region to which each image block belongs on the industrial part to be detected and the semantic gating signal, the first convolution kernel in the first branch of the adjusted feature extraction network and the deep layer stack are used to capture the global structural context of each image block, and the deep feature map of each image block is generated according to the global structural context; the second convolution kernel in the second branch of the adjusted feature extraction network and the dense connection are used to focus on the local micro-texture and edge details of each image block, to obtain a middle layer feature map, the deep layer feature map is subjected to feature up-sampling to obtain a semantic enhancement feature map, and the semantic enhancement feature map is subjected to channel recalibration based on the channel attention weight of the middle layer feature map to generate a semantic guided feature map; the deep layer feature map, the semantic guided feature map and the middle layer feature map are subjected to multi-scale fusion to generate a fusion feature map; the rotation direction between the fusion feature map of each image block and a pre-acquired defect prior feature vector is analyzed, the polar coordinates of the local coordinate system origin of the current image block relative to the global geometric center of the industrial part to be detected are calculated with the global geometric center of the industrial part to be detected as the rotation center; a plurality of discrete rotation angles are generated according to the rotation direction in the local coordinate system; the affine transformation matrix of each discrete rotation angle is subjected to rotation operation according to the fusion feature map of the image block to generate a rotation feature map sequence; the rotation feature map sequence is subjected to spatial convolution cross-correlation calculation with a target vector respectively to obtain a plurality of response maps; the rotation angle with the highest peak value in the response map is selected as the optimal alignment angle, and the response peak position and intensity under the optimal alignment angle are taken as the defect detection preliminary result of the image block; non-maximum suppression is applied to the defect detection preliminary result to obtain the defect detection result of the image block; the defect detection results of all image blocks are fused to obtain a pixel-level defect classification and positioning map of the industrial part to be detected.

2. The method of claim 1, wherein the method comprises: The method comprises: mapping the surface optical image to a two-dimensional unfolded map based on the three-dimensional model topology of the industrial part to be detected; performing non-uniform grid division on the two-dimensional unfolded map according to the distribution of the functional region boundary and the stress concentration region of the surface of the industrial part to be detected to obtain a plurality of non-overlapping image blocks; A local coordinate system with a centroid of each divided image block as an origin is established, and an axis direction of the local coordinate system is aligned with a principal stress direction or a typical texture direction of the industrial part to be detected in a corresponding region, to obtain a local coordinate system corresponding to each image block.

3. The method of claim 1, wherein the method comprises: The feature up-sampling is performed on the deep feature map to obtain a semantic enhancement feature map, including: Point-by-point convolution is performed on the deep feature map to generate spatially adaptive up-sampling kernel parameters; Irregular interpolation is performed on each local region of the deep feature map using the spatially adaptive up-sampling kernel parameters to obtain an initial up-sampling feature map; A high-frequency detail guide signal is extracted from the middle layer feature map, and the initial up-sampling feature map is element-by-element fused with the high-frequency detail guide signal to generate a detail enhancement feature map; Feature pyramid pooling is performed on the detail enhancement feature map to obtain a plurality of context features of different scales, and the plurality of context features of different scales are fused to generate a semantic enhancement feature map.

4. The method of claim 1, wherein the method comprises: The semantic guide feature map is generated by recalibrating the channel of the semantic enhancement feature map based on the channel attention weight of the middle layer feature map, including: The middle layer feature map is subjected to dual-path aggregation of global average pooling and global maximum pooling to obtain an aggregated feature vector, and the aggregated feature vector is converted into an initial channel attention weight using a pre-set multi-layer perceptron with a bottleneck structure; A global semantic importance vector of the semantic enhancement feature map is extracted, and the initial channel attention weight is multiplied by the global semantic importance vector through gating to generate an enhanced channel attention weight; Each channel of the semantic enhancement feature map is re-weighted by taking the enhanced channel attention weight as a modulation coefficient; The edge structure feature in the middle layer feature map is added to the re-weighted feature map through a pre-set residual connection, and the added features are activated through a pre-set activation function to generate a semantic guide feature map.

5. The method of claim 1, wherein the method further comprises: The deep feature map, the semantic guide feature map, and the middle layer feature map are fused in multiple scales to generate a fusion feature map, including: The deep feature map, the semantic guide feature map, and the middle layer feature map are respectively input into a spatial attention module to generate respective spatial importance masks; The deep feature map, the semantic guide feature map, and the middle layer feature map are weighted and summed based on the spatial importance masks to obtain a preliminary fusion feature; A three-channel pixel-level fusion weight map is obtained by channel fusion of the preliminary fusion feature using a pre-constructed fusion decision network; The deep feature map, the semantic guide feature map, and the middle layer feature map are subjected to pixel-level soft selection and mixing according to the pixel-level fusion weight map to generate a fusion feature map.

6. The method of claim 1, wherein the method further comprises: The rotation direction between the fusion feature map of each image block and a pre-acquired defect prior feature vector is analyzed, including: A plurality of candidate defect prior feature vectors associated with a part region and material attributes of the current image block are retrieved from a pre-constructed defect knowledge graph; compress the fusion feature map into a query feature vector by global pooling; calculate the cosine similarity and the included angle between the query feature vector and each candidate defect prior feature vector in the multi-dimensional spherical space; select the candidate defect prior feature vector with the smallest included angle and the cosine similarity exceeding the preset threshold as the target vector, and take the spatial included angle direction of the target vector and the query feature vector as the rotation direction; when there is no candidate defect prior feature vector meeting the condition, calculate the default direction of each image block based on the local texture main direction of the image block, and set the default direction as the rotation direction.

7. The method of claim 1, wherein the method further comprises: the rotation operation of the affine transformation matrix of each discrete rotation angle based on the fusion feature map of the image block, to generate a rotation feature map sequence, includes: calculate the rotation matrix according to the discrete rotation angle, and calculate the affine transformation matrix according to the rotation matrix and the offset amount of the local coordinate system origin and the global geometric center; perform geometric transformation on the fusion feature map according to each affine transformation matrix to obtain a rotated feature map; fill the edge pixels generated by symmetric replication for each rotated feature map, and perform standardization processing on each channel of each rotated feature map to obtain a standardized feature map; stack all standardized feature maps generated under different rotation angles of the same image block in a preset angle sequence to obtain a rotation feature map sequence.

8. The method of claim 1, wherein the method further comprises: the fusion of the defect detection results of all image blocks to obtain a pixel-level defect classification and positioning map of the industrial part to be detected, includes: establish a global image coordinate system with the geometric center of the industrial part to be detected as the origin and the design reference direction of the industrial part to be detected as the axis; identify the original position of each image block in the surface optical image, map the defect detection result of each image block back to the global image coordinate system according to the original position, and obtain a global defect distribution field; for the boundary region covered by multiple adjacent image blocks in the global defect distribution field, calculate the spatial distribution confidence of the fusion feature map response peak intensity of each image block in the boundary region; based on the spatial distribution confidence, perform adaptive weight allocation and probability fusion on the defect detection results of different image blocks in the boundary region to generate a global defect probability map eliminating boundary effects; according to the theoretical structure boundary of the industrial part to be detected, perform geometric constraint filtering on the boundary region to obtain a defect region contour, and determine the category label of the defect region contour according to the global defect probability map; superimpose the defect region contour and the category label on the surface optical image to generate a visual pixel-level defect classification and positioning map.

Images