Metal marking image processing method and device, computer device and storage medium

Abstract

The application relates to the field of image processing, and discloses a metal marking image processing method and device, computer equipment and a storage medium, the method comprising the following steps: acquiring a metal marking image to be processed; processing the metal marking image through a multi-branch attention backbone network to obtain backbone features; processing the backbone features through a preset encoder network to obtain deep layer features; and processing the backbone features and the deep layer features through a preset decoder to obtain a metal marking segmentation image. The application can improve the efficiency and accuracy of metal marking area reexamination and meet the high-precision identification requirement of metal marking.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a metal marking image processing method and device, computer equipment and storage medium. BACKGROUND

[0002] In the field of metal material production, especially in the field of steel manufacturing, the re-inspection of the metal marking area is an important link to ensure product quality and traceability. At present, traditional machine learning and deep learning methods are mainly used for marking area recognition and segmentation. Traditional image processing methods rely on manual extraction of color, texture and shape features, and use support vector machines (SVM), random forests (RF) and other classifiers to realize detection. Although it is simple to implement, the feature expression ability is limited when facing complex and variable marking patterns and background interference, resulting in insufficient recognition accuracy. In recent years, methods based on deep learning, especially U-Net and other convolutional neural network models, have made some progress in marking area segmentation tasks due to their automatic feature extraction and end-to-end learning capabilities. Some research has introduced attention mechanisms to enhance multi-scale feature fusion and improve model performance.

[0003] However, existing image processing methods still have problems such as limited receptive field, high model complexity, weak generalization ability, and are difficult to adapt to different sizes, positions and types of marking information. Therefore, there is an urgent need for an efficient, accurate and adaptable metal marking image processing method to improve the recognition accuracy of metal marking images and meet the high-precision identification requirements of metal markings. SUMMARY

[0004] The embodiments of the present application provide a metal marking image processing method and device, computer equipment and storage medium to improve the efficiency and accuracy of metal marking area re-inspection and meet the high-precision identification requirements of metal markings.

[0005] A metal marking image processing method, comprising:

[0006] Obtaining a metal marking image to be processed;

[0007] Processing the metal marking image through a multi-branch attention backbone network to obtain backbone features;

[0008] Processing the backbone features through a preset encoder network to obtain deep features;

[0009] Processing the backbone features and the deep features through a preset decoder to obtain a metal marking segmentation image.

[0010] Optionally, the multi-branch attention backbone network includes an asynchronously stacked multi-branch feature extraction module and an improved bottleneck attention module;

[0011] The metal marking image is processed by the multi-branch attention backbone network to obtain a backbone feature, including:

[0012] The metal marking image is processed by the multi-branch feature extraction module to obtain a marking feature map;

[0013] The marking feature map is processed by the improved bottleneck attention module to obtain the backbone feature.

[0014] Optionally, the metal marking image is processed by the multi-branch feature extraction module to obtain a marking feature map, including:

[0015] The metal marking image is processed by a plurality of first extraction convolutional layers to obtain an initial marking feature;

[0016] The initial marking feature is subjected to first down-sampling and channel adjustment to obtain a channel-adjusted feature;

[0017] The channel-adjusted feature is processed by a second extraction convolutional layer and subjected to second down-sampling processing to obtain a high-level feature;

[0018] The high-level feature is processed by a multi-branch network to obtain a plurality of branch features;

[0019] The plurality of branch features are spliced along the channel dimension to obtain the marking feature map.

[0020] Optionally, the high-level feature is processed by a multi-branch network to obtain a plurality of branch features, including:

[0021] The high-level feature is subjected to average pooling and first branch channel adjustment to obtain a first branch feature;

[0022] The high-level feature is subjected to second branch channel adjustment to obtain a second branch feature;

[0023] The high-level feature is subjected to third branch channel adjustment and processed using a high-level feature convolutional layer to obtain a third branch feature;

[0024] The high-level feature is subjected to fourth branch channel adjustment and processed using two 3x3 convolutional layers to obtain a fourth branch feature.

[0025] Optionally, the marking feature map is processed by the improved bottleneck attention module to obtain the backbone feature, including:

[0026] The marking feature map is processed by a feature transformation branch network to obtain a feature transformation branch feature; the feature transformation branch network includes a first transformation convolutional layer, a second transformation convolutional layer, and a third transformation convolutional layer;

[0027] The attention weight is obtained by processing the marking feature map through an attention weight branch network; the attention weight branch network comprises an average pooling layer, a multi-layer perception and a batch normalization layer;

[0028] The weighted feature is obtained by fusing the feature transformation branch feature and the attention weight;

[0029] The backbone feature is obtained by adding the marking feature map and the weighted feature.

[0030] Optionally, the deep layer feature is obtained by processing the backbone feature through a preset encoder network, comprising:

[0031] The first encoding feature is obtained by processing the backbone feature through a first encoder convolutional layer; the first encoder convolutional layer is a 1x1 convolutional layer;

[0032] The second encoding feature is obtained by processing the backbone feature through a second encoder convolutional layer; the second encoder convolutional layer comprises a plurality of 3x3 first atrous convolutional layers; the span of the first atrous convolutional layer is less than or equal to 6;

[0033] The third encoding feature is obtained by processing the backbone feature through a third encoder convolutional layer; the third encoder convolutional layer comprises a plurality of 3x3 second atrous convolutional layers; the span of the second atrous convolutional layer is greater than or equal to 10 and less than or equal to 14;

[0034] The fourth encoding feature is obtained by processing the backbone feature through a fourth encoder convolutional layer; the fourth encoder convolutional layer comprises a plurality of 3x3 third atrous convolutional layers; the span of the third atrous convolutional layer is greater than or equal to 16 and less than or equal to 20;

[0035] The fifth encoding feature is obtained by processing the backbone feature through a maximum pooling layer;

[0036] The deep layer feature is obtained by processing the first encoding feature, the second encoding feature, the third encoding feature, the fourth encoding feature and the fifth encoding feature through a fusion convolutional layer.

[0037] Optionally, the metal marking segmentation image is obtained by processing the backbone feature and the deep layer feature through a preset decoder, comprising:

[0038] The backbone convolutional feature is obtained by processing the backbone feature through a first decoding convolutional layer;

[0039] The deep layer up-sampling feature is obtained by performing an up-sampling operation on the deep layer feature at a first rate;

[0040] The decoding feature map is obtained by stacking and merging the backbone convolutional feature and the deep layer up-sampling feature;

[0041] obtaining a decoding convolution feature map by processing the decoding feature map through a second decoding convolution layer;

[0042] performing an up-sampling operation on the decoding convolution feature map at a second rate to obtain the metal marking segmentation image.

[0043] A metal marking image processing device comprises:

[0044] an acquisition marking image module configured to acquire a metal marking image to be processed;

[0045] a multi-branch attention backbone network configured to process the metal marking image to obtain backbone features;

[0046] a preset encoder network configured to process the backbone features to obtain deep features;

[0047] a preset decoder configured to process the backbone features and the deep features to obtain a metal marking segmentation image.

[0048] A computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the metal marking image processing method described above when executing the computer program.

[0049] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the metal marking image processing method described above.

[0050] The metal marking image processing method, device, computer device, and storage medium described above enhance the feature extraction capability of fine textures and complex backgrounds in metal marking images through the multi-branch attention backbone network, effectively improve the richness and discriminability of marking feature expression, combine the encoder-decoder structure to fuse the backbone features and the deep features, realize accurate segmentation of the metal marking region, significantly improve the accuracy and robustness of segmentation, can improve the efficiency and accuracy of metal marking region review, and meet the high-precision identification requirements of metal marking. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0052] Figure 1 is an application environment schematic diagram of a metal marking image processing method in an embodiment of the present application;

[0053] Figure 2 is a flowchart of a metal marking image processing method in an embodiment of the present application;

[0054] Figure 3 is a metal marking image in an embodiment of the present application;

[0055] Figure 4a is a metal marking segmentation image processed by using a DeepLabv3+ model;

[0056] Figure 4b is a metal marking segmentation image processed by using a Mask2Former model;

[0057] Figure 4c is a metal marking segmentation image processed by using a DINOv2 model;

[0058] Figure 4d is a processed metal marking segmentation image in an embodiment of the present application;

[0059] Figure 5 is a structural schematic diagram of a multi-branch feature extraction module in an embodiment of the present application;

[0060] Figure 6 is a structural schematic diagram of an improved bottleneck attention module in an embodiment of the present application;

[0061] Figure 7 is a structural schematic diagram of a preset encoder network in an embodiment of the present application;

[0062] Figure 8 is a structural schematic diagram of a preset decoder in an embodiment of the present application;

[0063] Figure 9 is a structural schematic diagram of a preset decoder in an embodiment of the present application;

[0064] Figure 10 is a schematic diagram of a metal marking image processing device in an embodiment of the present application;

[0065] Figure 11 is a schematic diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0066] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0067] The metal marking image processing method provided in this embodiment of the invention can be applied to, for example... Figure 1 The application environment is shown. Specifically, this metal marking image processing method is applied in a metal marking image processing system, which includes, as shown in the example, [example of the system]. Figure 1 The diagram shows a client and server that communicate over a network to improve the efficiency and accuracy of metal marking area re-inspection. The client, also known as the user terminal, is the program that provides local services to the client, corresponding to the server. The client can be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be a standalone server or a server cluster consisting of multiple servers.

[0068] In one embodiment, such as Figure 2 As shown, a metal marking image processing method is provided, which is applied to... Figure 1 Taking the server in the example, the following steps S10~S40 are explained.

[0069] S10. Obtain the metal marking image to be processed.

[0070] In essence, a metal marking image refers to an image containing the marked area of ​​a metal material. Here, the metal material can be steel, aluminum, copper, etc. Metal marking images can be acquired using a high-resolution industrial camera. When acquiring metal marking images, since metal surfaces are prone to reflection, polarizing filters or multi-angle lighting schemes can be used to reduce reflection interference. In one example, the metal marking image has a resolution of 1024×1024 pixels and is in RGB format.

[0071] S20. The metal marking image is processed by a multi-branch attention backbone network to obtain backbone features.

[0072] Understandably, a multi-branch attention backbone network is a feature extraction network that integrates a multi-branch feature extraction module and an improved bottleneck attention module. The multi-branch feature extraction module performs initial processing to generate labeled feature maps of different scales. These labeled feature maps are then fed into the improved bottleneck attention module for further optimization, highlighting key features and suppressing irrelevant information to form the backbone features.

[0073] S30. Process the backbone features through a preset encoder network to obtain deep features.

[0074] Understandably, the preset encoder network is used to reduce the model complexity while expanding the receptive field. The preset encoder network includes multiple ordinary convolution layers with fixed channel numbers, dynamic span dilated convolution layers and max pooling layers, forming a deep feature extraction structure that can not only effectively capture information at different scales, but also improve the accuracy of the model in segmenting the marked region on the metal surface. The deep feature extraction structure of the preset encoder network can effectively capture and integrate feature information from multiple scales, thereby improving the accuracy and robustness of the model in segmenting the marked region on the metal surface. After processing by the preset encoder network, the deep features of the metal marking image can be obtained. These deep features will be used in the subsequent decoder part to accurately restore the marked region in the image and distinguish the marked region from the irrelevant background.

[0075] S40, processing the backbone features and the deep features through a preset decoder to obtain a metal marking segmentation image.

[0076] Understandably, in order to ensure the accuracy and efficiency of the metal marking segmentation image restoration, the preset decoder can be used to decode the backbone features and the deep features to generate the metal marking image. Among them, the preset decoder can use asymmetric upsampling technology. Through asymmetric upsampling, not only can the multi-level feature information extracted by the preset encoder network be fully utilized, but also the spatial details of the image can be effectively restored, thereby realizing high-precision segmentation of the metal marking region. As shown in FIG. 4d, Figure 3 , 4a Figure 3 is an example of a metal marking image, Figure 4a to Figure 4d is a metal marking segmentation image processed by different models. Among them, Figure 4a is a metal marking segmentation image processed by a DeepLabv3+ model, Figure 4b is a metal marking segmentation image processed by a Mask2Former model, Figure 4c is a metal marking segmentation image processed by a DINOv2 model, Figure 4d ​The metal marking segmentation image processed by the embodiment of the present application. Among them, the DeepLabv3+ model is a relatively classic model in semantic segmentation, but it lacks a lot of background area false detection, part of the character boundary is blurred, and it cannot guarantee the integrity of the marking area. For the reflective area, there is an error of point segmentation phenomenon. The Mask2Former model can better segment the target character, but there is still a missing detection phenomenon for small areas, and the character edge is still not fine enough, and there is also an error of segmentation in the reflective area. The DINOv2 model is sensitive to the texture noise of the background, and it is easy to mistakenly divide the non-target area into the character area, resulting in serious interference of the segmentation result. Compared with the above three traditional methods, the method proposed in the embodiment can accurately extract the complete character area of the steel marking by extracting local detail features and global context information, combining the improved bottleneck attention mechanism, significantly suppressing the interference of the texture noise of the background, and at the same time maintaining good coherence and clarity of the character edge, and obviously reducing the influence of the reflective area on the segmentation effect. Especially in the aspects of character details and contour preservation, the embodiment is obviously superior to the prior art.

[0077] The embodiment enhances the feature extraction capability of fine texture and complex background in the metal marking image through the multi-branch attention backbone network, effectively improves the richness and discriminability of the marking feature expression; combines the encoder-decoder structure to fuse the backbone features and deep features, realizes the accurate segmentation of the metal marking area, significantly improves the accuracy and robustness of the segmentation, can improve the efficiency and accuracy of the metal marking area review, and meets the high-precision identification requirement of the metal marking. That is, the method provided by the embodiment can distinguish characters, icons and backgrounds at pixel level accuracy, ensure edge integrity and region continuity. The method provided by the embodiment can not only identify the boundary of a string of characters and icons, but also distinguish the contour of each character, and can identify the true marking form even in the case of reflection or blur.

[0078] Optionally, the multi-branch attention backbone network comprises an asynchronous stacked multi-branch feature extraction module and an improved bottleneck attention module;

[0079] Step S20, i.e., processing the metal marking image through the multi-branch attention backbone network to obtain a backbone feature, comprising:

[0080] S201, processing the metal marking image through the multi-branch feature extraction module to obtain a marking feature map;

[0081] S202, asynchronously processing the marking feature map through the improved bottleneck attention module to obtain the backbone feature.

[0082] Understandably, the multi-branch feature extraction module can include basic feature extraction, downsampling operation, channel adjustment, high-level feature extraction, multi-branch processing, feature integration and other processing steps. Through the multi-branch feature extraction module processing, the marking feature map can be obtained. The marking feature map has rich levels, not only contains rich detailed information, but also has good scale adaptability, which can optimize the model performance.

[0083] The improved bottleneck attention module can include an input layer, parallel feature transformation branch networks and attention weight branch networks, a fusion unit and a feature addition unit. After processing by the improved bottleneck attention module, the backbone features of the metal marking image can be obtained. Here, the improved bottleneck attention module effectively enhances the expressiveness of key features and suppresses the influence of irrelevant information, which can improve the accuracy of the model in rechecking the metal marking area, and at the same time improve the robustness and efficiency of the model.

[0084] In some examples, the improved bottleneck attention module can use a self-attention mechanism to improve the segmentation accuracy of complex marking content by calculating the spatial correlation of the feature map, but it needs to increase the memory consumption.

[0085] The present embodiment preliminarily processes the metal marking image through the multi-branch feature extraction module, effectively captures the detailed features and global information in the image, and forms a marking feature map. Subsequently, the improved bottleneck attention module asynchronously processes the marking feature map, which can specifically enhance useful features and suppress irrelevant information, thereby obtaining more refined and representative backbone features. The present embodiment not only improves the sensitivity of the model to key features, but also enhances the efficiency and accuracy of feature utilization in subsequent tasks, which provides a solid foundation for realizing higher quality metal marking image analysis.

[0086] Optionally, step S201, i.e., processing the metal marking image through the multi-branch feature extraction module to obtain a marking feature map, includes:

[0087] S2011, processing the metal marking image through a plurality of first extraction convolutional layers to obtain initial marking features;

[0088] S2012, performing first downsampling and channel adjustment on the initial marking features to obtain channel adjusted features;

[0089] S2013, processing the channel adjusted features through a second extraction convolutional layer and performing second downsampling processing to obtain high-level features;

[0090] S2014, processing the high-level features through a multi-branch network to obtain a plurality of branch features;

[0091] S2015, concatenating the plurality of branch features along a channel dimension to obtain the marking feature map.

[0092] As can be understood, Figure 5 as shown, Figure 5 is a structural schematic diagram of a multi-branch feature extraction module in an example. In the process of processing a metal marking image through the multi-branch feature extraction module, the initial marking feature can be obtained by a plurality of first extraction convolutional layers for basic feature extraction. Here, the plurality of first extraction convolutional layers can be three consecutive 3x3 convolutional layers (with channel numbers of 32, 32 and 64, respectively), which are key steps for constructing basic feature representations. The stacking of multiple small convolutional kernels can effectively expand the receptive field while keeping the parameter amount relatively small, thereby gradually capturing image features from low-level edges, textures to more complex patterns and obtaining rich detailed information to provide a basis for subsequent multi-branch processing.

[0093] Then, the initial marking feature can be subjected to first down-sampling and channel adjustment to obtain a channel-adjusted feature. In an example, the first down-sampling can use a max-pooling layer with a stride of 2 to reduce the spatial size of the feature map while compressing the computational load of the model. The channel adjustment uses a 1x1 convolutional layer with a channel number of 80 to match the input channel number of the subsequent branches.

[0094] After the channel adjustment, the channel-adjusted feature is processed by a second extraction convolutional layer and subjected to second down-sampling to obtain a high-level feature. In an example, the second extraction convolutional layer can be a 3x3 convolutional layer with a channel number of 192. A 3x3 convolutional layer with a high channel number (192) is used after the channel adjustment, aiming to further extract more abstract and high-level semantic features. The second down-sampling can be a max-pooling layer with a stride of 2, aiming to compress the computational load of the model. Increasing the channel number means increasing the capacity and capacity of the feature representation, allowing the network to learn and retain more complex patterns and information. This operation helps to make the features contain richer representations before entering the multi-branch structure.

[0095] After obtaining the high-level feature, the multi-branch network processes the high-level feature to obtain a plurality of branch features. In an example, the multi-branch network includes four parallel branches. The multi-branch network allows the network to process features in parallel, from different angles and scales, to capture diverse information. Each branch can obtain a respective branch feature.

[0096] Finally, the plurality of branch features are concatenated along the channel dimension to obtain a marking feature map. By aggregating the feature information extracted by different branches in different scales, different receptive fields and different emphases, an extremely rich and comprehensive feature representation is formed.

[0097] In some examples, each extraction convolutional layer can use a depthwise separable convolution instead of a normal convolution to further reduce the parameter amount and be suitable for deployment on embedded devices.

[0098] The embodiment processes the metal marking image through multiple first extraction convolutional layers to obtain initial marking features, emphasizing details and basic structures. Then, the initial features are down-sampled and adjusted in channels to optimize feature expression, making it more suitable for subsequent processing. Subsequently, the second extraction convolutional layer further refines high-level features and processes them through a multi-branch network to generate branch features of multiple different abstraction levels. Finally, these branch features are spliced along the channel dimension to form a marking feature map containing multiple levels of information, providing a rich feature basis for accurate recognition and analysis. The embodiment effectively improves the feature extraction capability and recognition accuracy of metal marking images.

[0099] Optionally, step S2014, i.e., processing the high-level features through a multi-branch network to obtain multiple branch features, includes:

[0100] S20141, performing average pooling and first branch channel adjustment on the high-level features to obtain first branch features;

[0101] S20142, performing second branch channel adjustment on the high-level features to obtain second branch features;

[0102] S20143, performing third branch channel adjustment on the high-level features and processing them using a high-level feature convolutional layer to obtain third branch features;

[0103] S20144, performing fourth branch channel adjustment on the high-level features and processing them using two 3x3 convolutional layers to obtain fourth branch features.

[0104] Understandably, the multi-branch network can process the high-level features in parallel to generate branch features corresponding to each branch.

[0105] Specifically, in the first branch, the high-level features are subjected to average pooling and first branch channel adjustment to obtain first branch features. The average pooling operation takes the average of all values within the pooling window, which can reduce the dimension and the amount of calculation, smooth the features and suppress noise. The first branch channel adjustment uses a 64-channel 1x1 convolutional layer. The first branch features contain low-resolution global features that can enhance the model's perception of large-scale information and determine the approximate position of the marking.

[0106] In the second branch, the high-level features are subjected to second branch channel adjustment to obtain second branch features. Here, the second branch channel adjustment uses a 96-channel 1x1 convolutional layer for channel-level adjustment and transformation. The second branch can directly adjust the channel number, preserving the detail information at the original scale, maintaining the local texture features, and preserving the local line or icon edges of the marked area icons and characters.

[0107] In the third branch, the high-level features are subjected to third branch channel adjustment and processed using a high-level feature convolutional layer to obtain third branch features. Here, the third branch channel adjustment uses a 48-channel 1x1 convolutional layer. The high-level feature convolutional layer can use a 64-channel 5x5 convolutional kernel or a 7x7 convolutional kernel. The larger convolutional kernel has a larger receptive field, which is good at capturing larger spatial patterns and context information. Compared with the 5x5 convolutional kernel, the 7x7 convolutional kernel can enhance the modeling ability of large-scale marked areas, but requires an additional 15% of the computational load. The third branch features contain medium-scale features, which can enhance the model's ability to recognize larger marked characters, icons, or defect areas.

[0108] In the fourth branch, the high-level features are subjected to fourth branch channel adjustment and processed using two 3x3 convolutional layers to obtain fourth branch features. Here, the fourth branch channel adjustment uses a 64-channel 1x1 convolutional layer. Then, two 3x3 convolutional layers (each with 96 channels) are used to further extract features for extracting more complex and nonlinear features while focusing on capturing details and local structures. The fourth branch extracts high-level semantic features by stacking small convolutional kernels to learn complex detail features such as font curvature and surface reflection texture.

[0109] The present embodiment processes high-level features through a multi-branch network to generate multiple branch features with different characteristics. First, the high-level features are subjected to pooling and first branch channel adjustment to obtain first branch features that emphasize global information. Then, the second branch channel adjustment is used to obtain second branch features that focus on specific channel information. The third branch features are captured by combining channel adjustment with a 5x5 convolutional layer processing to capture more extensive context information. The fourth branch features use two 3x3 convolutional layers to enhance the ability to extract local details. The present embodiment improves the model's ability to recognize and analyze complex patterns and detail features in metal marking images through multi-level and multi-angle feature processing, enhancing the robustness and accuracy of the system.

[0110] Optionally, step S202, i.e., the processing of the marking feature map through the improved bottleneck attention module to obtain the backbone features, includes:

[0111] S2021, processing the marking feature map through a feature transformation branch network to obtain feature transformation branch features; the feature transformation branch network comprises a first transformation convolution layer, a second transformation convolution layer, and a third transformation convolution layer;

[0112] S2022, processing the marking feature map through an attention weight branch network to obtain attention weights; the attention weight branch network comprises an average pooling layer, a multi-layer perception, and a batch normalization layer;

[0113] S2023, fusing the feature transformation branch features and the attention weights to obtain weighted features;

[0114] S2024, adding the marking feature map and the weighted features to obtain the backbone features.

[0115] As can be understood, Figure 6 as shown, Figure 6 is a structural schematic diagram of an improved bottleneck attention module in an example. The improved bottleneck attention module is used to optimize the selective attention of features and enhance the expressiveness of key features.

[0116] Specifically, the marking feature map output by the multi-branch feature extraction module can be received through the input layer. Then, the marking feature map is processed through the feature transformation branch network to obtain the feature transformation branch features. The feature transformation branch network can retain the spatial structure information of the input features, extract the local marking texture and structure features output by the multi-branch feature extraction module, and form preliminary backbone features, i.e., the feature transformation branch features. The feature transformation branch network comprises a first transformation convolution layer, a second transformation convolution layer, and a third transformation convolution layer. Among them, the first transformation convolution layer is a 64-channel 1x1 convolution layer, which maps the input features to a lower-dimensional space through the 1x1 convolution layer, reduces the number of parameters, and speeds up the calculation. The second transformation convolution layer successively uses two 3x3 convolution layers, and the output channel number of each convolution layer is 32. The second transformation convolution layer helps to further extract local features and increase the nonlinear expression ability of the network. The third transformation convolution layer is a 64-channel 1x1 convolution layer, which is used to map the features back to the original channel number to complete the preliminary processing of the features.

[0117] The marking feature map is processed by an attention weight branch network to obtain an attention weight. The attention weight branch network is used to realize extraction of channel attention, enhance selective attention of global marking information, enhance learning of key information, and suppress irrelevant or noise features. The attention weight branch network includes an average pooling layer, a multi-layer perceptron, and a batch normalization layer. The average pooling layer obtains statistical information at a channel level through a global average pooling operation to obtain a fixed-size feature vector. The multi-layer perceptron uses two fully connected layers to generate the attention weight. The first fully connected layer maps the feature vector to a lower-dimensional space, and the second fully connected layer maps it back to the original dimension. The batch normalization layer performs batch normalization processing on the generated attention weight to accelerate the convergence speed of the network and improve the generalization ability.

[0118] The feature transformation branch feature and the attention weight are fused to obtain a weighted feature. Here, the feature transformation branch feature and the attention weight are multiplied by channel to strengthen the feature response of important channels and suppress irrelevant or noise channels.

[0119] The marking feature map and the weighted feature are added element by element to form a residual connection, which ensures that the network learns to enhance the features while preserving the original information and avoids gradient disappearance. The output is the enhanced backbone feature, which can be used for subsequent image segmentation.

[0120] The embodiment extracts feature transformation branch features with stronger expression ability through multi-level convolutional transformation of the marking feature map by the feature transformation branch network, and enhances the non-linear representation ability of the features. At the same time, the attention weight branch network combines the average pooling layer, the multi-layer perceptron, and the batch normalization layer to adaptively learn the importance weight of the channels and the space, and highlight the key region information. The embodiment generates a weighted feature by fusing the feature transformation branch feature and the attention weight, realizes enhancement of key features and suppression of redundant information. Finally, the weighted feature and the original marking feature map are added to preserve the original information while introducing attention-guided enhanced features, effectively improving the quality and discriminative ability of the backbone feature.

[0121] Optionally, step S30, i.e., processing the backbone feature by a preset encoder network to obtain a deep feature, includes:

[0122] S301, processing the backbone feature by a first encoder convolutional layer to obtain a first encoding feature; the first encoder convolutional layer is a 1x1 convolutional layer;

[0123] S302, processing the backbone feature by a second encoder convolutional layer to obtain a second encoding feature; the second encoder convolutional layer includes a plurality of 3x3 first atrous convolutional layers; the span of the first atrous convolutional layer is less than or equal to 6;

[0124] S303. The backbone features are processed by the third encoder convolutional layer to obtain the third encoded features; the third encoder convolutional layer includes multiple 3x3 second dilated convolutional layers; the span of the second dilated convolutional layer is greater than or equal to 10 and less than or equal to 14.

[0125] S304. The backbone features are processed by the fourth encoder convolutional layer to obtain the fourth encoded features; the fourth encoder convolutional layer includes multiple 3x3 third dilated convolutional layers; the span of the third dilated convolutional layer is greater than or equal to 16 and less than or equal to 20.

[0126] S305. The backbone features are processed by a max pooling layer to obtain the fifth encoded feature;

[0127] S306. The deep features are obtained by processing the first coding feature, the second coding feature, the third coding feature, the fourth coding feature, and the fifth coding feature through a fusion convolutional layer.

[0128] Understandably, such as Figure 7 As shown, Figure 7 This is a schematic diagram of the structure of a preset encoder network in an example. The preset encoder network includes four encoder convolutional layers, a max pooling layer, and a fusion convolutional layer, which realizes deep feature extraction and can effectively capture information at different scales, improving the accuracy of the model in segmenting the marked areas on the metal surface.

[0129] Specifically, the backbone features can be processed through the first encoder convolutional layer to obtain the first encoded features. The first encoder convolutional layer is a 1x1 convolutional layer with 256 channels. The first encoder convolutional layer helps reduce computation and accelerate information transmission.

[0130] The backbone features are processed by a second encoder convolutional layer to obtain the second encoded features. In one example, the second encoder convolutional layer consists of three 3x3 first-dilated convolutional layers with 256 channels and spans of 2, 4, and 6, respectively. The second encoder convolutional layer is able to capture medium-scale contextual information while maintaining high resolution.

[0131] The backbone features are processed by a third encoder convolutional layer to obtain the third encoded features. In one example, the third encoder convolutional layer consists of three 3x3 dilated convolutional layers with 256 channels and spans of 10, 12, and 14 respectively. Compared to the second encoder convolutional layer, the third encoder convolutional layer has a larger span, enabling it to capture a wider range of contextual information and enhancing the model's understanding of global features.

[0132] The backbone feature is processed by a fourth encoder convolutional layer to obtain a fourth encoding feature. The fourth encoder convolutional layer includes three 3x3 third hollow convolutional layers, with a channel number of 256 and a span of 16, 18 and 20 respectively. Compared with the previous two hollow convolutional layers, the fourth encoder convolutional layer can capture the most extensive context information, which helps the model to identify complex patterns and details. As shown in FIG. 3, Figure 8 Figure 8 is a schematic diagram of a hollow convolution with a span of n. As the span increases, the receptive field of the hollow convolutional layer expands, and the multi-scale information capturing capability is enhanced.

[0133] The backbone feature can be processed by a maximum pooling layer with a stride of 2 to obtain a fifth encoding feature. The maximum pooling layer is used for downsampling to reduce the spatial size of the feature map, further compressing the calculation amount of the model, which helps to reduce the complexity of the model while retaining key feature information.

[0134] Finally, the first encoding feature, the second encoding feature, the third encoding feature, the fourth encoding feature and the fifth encoding feature are processed by a fusion convolutional layer to obtain a deep feature. The fusion convolutional layer can be a 1x1 convolutional layer with a channel number of 256. The fusion convolutional layer can make the output channel number of the deep feature match the subsequent processing stage, and fuse the feature information from different scales to generate a more comprehensive and comprehensive deep feature.

[0135] The first to fourth encoder convolutional layers with different configurations in the embodiment perform multi-level and multi-scale feature extraction on the backbone feature to capture details and context information in images or data. The use of hollow convolution can expand the receptive field without increasing the calculation amount, and enhance the understanding ability of the model for large-scale feature relationships. The application of the maximum pooling layer further refines the key information of the backbone feature, ensuring that the deep feature not only contains rich detail information, but also retains important spatial structure features. Finally, the fusion convolutional layer integrates the features of the above layers to generate a more comprehensive and advanced deep feature, effectively improving the performance and robustness of the model in complex tasks.

[0136] Optionally, the step S40, i.e., the processing of the backbone feature and the deep feature by the preset decoder to obtain a metal marking segmentation image, includes:

[0137] S401, processing the backbone feature by a first decoding convolutional layer to obtain a backbone convolutional feature;

[0138] S402, performing an upsampling operation on the deep feature at a first rate to obtain a deep upsampled feature;

[0139] S403, stacking and merging the backbone convolutional feature and the deep upsampled feature to obtain a decoding feature map; ​

[0140] S404, processing the decoding feature map through a second decoding convolutional layer to obtain a decoding convolutional feature map;

[0141] S405, performing an up-sampling operation on the decoding convolutional feature map at a second rate to obtain the metal marking segmentation image.

[0142] As can be understood, Figure 9 as shown, Figure 9 is a structural schematic diagram of a preset decoder in an example. The preset decoder decodes backbone features and deep features, which can ensure the accuracy and efficiency of image restoration.

[0143] Specifically, the backbone features are first processed through a first decoding convolutional layer to obtain backbone convolutional features. The first decoding convolutional layer can be a 1x1 convolutional layer with a channel number of 64. The backbone features contain rich local detail information, which is helpful for accurately positioning the marking area. By adjusting the channel number of the backbone features through the first decoding convolutional layer, the output channel number of the subsequent processing stage is matched, which helps to reduce the amount of calculation and speed up the information transmission.

[0144] The deep features are up-sampled at a first rate to obtain deep up-sampled features. Here, the first rate can be 8. The deep features contain global context information, which is helpful for understanding the overall distribution and shape of the marking area. By 8 times up-sampling operation, the spatial size of the deep feature map is restored to close to the resolution of the original image, which helps to retain more spatial details and provides a basis for subsequent feature fusion.

[0145] Then, the backbone convolutional features and the deep up-sampled features are stacked and combined according to the channel dimension to obtain a decoding feature map. This combination method can comprehensively utilize feature information at different levels and improve the expression ability of the model.

[0146] Then, the decoding feature map is processed through a second decoding convolutional layer to obtain a decoding convolutional feature map. The second decoding convolutional layer can be a 3x3 convolutional layer with a channel number of 32. This process helps to enhance the representation ability and robustness of the features.

[0147] Finally, the decoding convolutional feature map is up-sampled at a second rate to obtain a metal marking segmentation image. The second rate can be 2. By 2 times up-sampling operation, the spatial size of the feature map is further restored, and finally the metal marking segmentation image is obtained. This process ensures that the metal marking segmentation image matches the metal marking image in spatial size, improving the accuracy of segmentation.

[0148] In some examples, the up-sampling operation in the preset decoder can be replaced by transposed convolution, which can improve the accuracy of the segmentation boundary by learning the up-sampling weight, but increases the training complexity.

[0149] The embodiment performs preliminary processing on the backbone feature through the first decoding convolutional layer, retains and optimizes the shallow layer detail information, and provides high-quality input for subsequent feature fusion; after the deep layer feature is subjected to first-rate upsampling, the deep layer feature is stacked and combined with the backbone convolutional feature, thereby realizing sufficient fusion of deep layer semantic information and shallow layer spatial details and enhancing the completeness of feature expression; the fused decoding feature map is further optimized through the second decoding convolutional layer, thereby effectively suppressing noise and improving boundary definition; and finally, second-rate upsampling is performed to gradually restore the spatial resolution, thereby generating a high-precision metal marking segmentation image and significantly improving the accuracy and detail restoration capability of the segmentation result.

[0150] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0151] In an embodiment, a metal marking image processing device is provided, which corresponds to the metal marking image processing method in the above embodiment. As shown in the figure, the metal marking image processing device comprises: Figure 10

[0152] The marking image acquisition module 10 is configured to acquire a metal marking image to be processed;

[0153] The multi-branch attention backbone network 20 is configured to process the metal marking image to obtain a backbone feature;

[0154] The preset encoder network 30 is configured to process the backbone feature to obtain a deep layer feature;

[0155] The preset decoder 40 is configured to process the backbone feature and the deep layer feature to obtain a metal marking segmentation image.

[0156] Optionally, the multi-branch attention backbone network 20 comprises an asynchronous stacked multi-branch feature extraction module and an improved bottleneck attention module; wherein,

[0157] The multi-branch feature extraction module is configured to process the metal marking image to obtain a marking feature map;

[0158] The improved bottleneck attention module is configured to asynchronously process the marking feature map to obtain the backbone feature.

[0159] Optionally, the multi-branch feature extraction module comprises:

[0160] The initial marking feature acquisition unit is configured to process the metal marking image through a plurality of first extraction convolutional layers to obtain an initial marking feature;

[0161] ​The acquisition channel adjustment feature unit is configured to perform first down-sampling and channel adjustment on the initial marking feature to obtain a channel adjustment feature.

[0162] The high-level feature acquisition unit is configured to process the channel adjustment feature by a second extraction convolutional layer and perform second down-sampling processing to obtain a high-level feature.

[0163] The multi-branch network is configured to process the high-level feature to obtain a plurality of branch features.

[0164] The marking feature map acquisition unit is configured to splice the plurality of branch features along a channel dimension to obtain the marking feature map.

[0165] Optionally, the multi-branch network comprises:

[0166] The first branch feature unit is configured to perform average pooling and first branch channel adjustment on the high-level feature to obtain a first branch feature.

[0167] The second branch feature unit is configured to perform second branch channel adjustment on the high-level feature to obtain a second branch feature.

[0168] The third branch feature unit is configured to perform third branch channel adjustment on the high-level feature and process using a high-level feature convolutional layer to obtain a third branch feature.

[0169] The fourth branch feature unit is configured to perform fourth branch channel adjustment on the high-level feature and process using two 3x3 convolutional layers to obtain a fourth branch feature.

[0170] Optionally, the improved bottleneck attention module comprises:

[0171] The feature transformation branch network is configured to process the marking feature map to obtain a feature transformation branch feature; the feature transformation branch network comprises a first transformation convolutional layer, a second transformation convolutional layer, and a third transformation convolutional layer.

[0172] The attention weight branch network is configured to process the marking feature map to obtain an attention weight; the attention weight branch network comprises an average pooling layer, a multi-layer perception, and a batch normalization layer.

[0173] The weighted feature acquisition unit is configured to fuse the feature transformation branch feature and the attention weight to obtain a weighted feature.

[0174] The backbone feature acquisition unit is configured to add the marking feature map and the weighted feature to obtain the backbone feature.

[0175] Optionally, the preset encoder network 30 comprises:

[0176] a first encoder convolutional layer configured to process the backbone feature to obtain a first encoded feature, wherein the first encoder convolutional layer is a 1x1 convolutional layer;

[0177] a second encoder convolutional layer configured to process the backbone feature to obtain a second encoded feature, wherein the second encoder convolutional layer comprises a plurality of 3x3 first atrous convolutional layers, and wherein a stride of the first atrous convolutional layers is less than or equal to 6;

[0178] a third encoder convolutional layer configured to process the backbone feature to obtain a third encoded feature, wherein the third encoder convolutional layer comprises a plurality of 3x3 second atrous convolutional layers, and wherein a stride of the second atrous convolutional layers is greater than or equal to 10 and less than or equal to 14;

[0179] a fourth encoder convolutional layer configured to process the backbone feature to obtain a fourth encoded feature, wherein the fourth encoder convolutional layer comprises a plurality of 3x3 third atrous convolutional layers, and wherein a stride of the third atrous convolutional layers is greater than or equal to 16 and less than or equal to 20;

[0180] a max-pooling layer configured to process the backbone feature to obtain a fifth encoded feature;

[0181] a fusion convolutional layer configured to process the first encoded feature, the second encoded feature, the third encoded feature, the fourth encoded feature, and the fifth encoded feature to obtain the deep feature.

[0182] Optionally, the preset decoder 40 comprises:

[0183] a first decoding convolutional layer configured to process the backbone feature to obtain a backbone convolutional feature;

[0184] a first upsampling unit configured to perform an upsampling operation on the deep feature at a first upsampling rate to obtain a deep upsampled feature;

[0185] a stack merging unit configured to stack and merge the backbone convolutional feature and the deep upsampled feature to obtain a decoding feature map;

[0186] a second decoding convolutional layer configured to process the decoding feature map to obtain a decoding convolutional feature map;

[0187] a second upsampling unit configured to perform an upsampling operation on the decoding convolutional feature map at a second upsampling rate to obtain the metal marking segmentation image.

[0188] The specific limitations of the metal marking image processing device can refer to the limitations of the metal marking image processing method in the above, which will not be repeated here. Each module in the above metal marking image processing device can be realized by software, hardware and their combination in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor calls and executes the operations corresponding to each module.

[0189] In one embodiment, a computer device, which can be a server, is provided, and an internal structure diagram thereof can be as shown in Figure 11 The computer device includes a processor, a memory, a network interface and a database connected by a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is used to store data related to the metal marking image processing method. The network interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a metal marking image processing method.

[0190] In one embodiment, a computer device is provided, including a memory, a processor and a computer program stored in the memory and executable on the processor. The processor implements the metal marking image processing method in the above embodiments when executing the computer program, and details are not repeated here. Alternatively, the processor implements the functions of each module / unit in the metal marking image processing device in this embodiment when executing the computer program, and details are not repeated here.

[0191] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by the processor to implement the metal marking image processing method in the above embodiments, and details are not repeated here. Alternatively, the computer program is executed by the processor to implement the functions of each module / unit in the metal marking image processing device in this embodiment, and details are not repeated here.

[0192] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiments of each method can be included. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0193] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0194] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A metal marking image processing method, characterized by, The method comprises the following steps: acquiring a metal marking image to be processed; processing the metal marking image through a multi-branch attention backbone network to obtain backbone features; processing the backbone features through a preset encoder network to obtain deep layer features; processing the backbone features and the deep layer features through a preset decoder to obtain a metal marking segmentation image; wherein the multi-branch attention backbone network comprises an asynchronously stacked multi-branch feature extraction module and an improved bottleneck attention module; the processing of the metal marking image through the multi-branch attention backbone network to obtain the backbone features comprises: processing the metal marking image through the multi-branch feature extraction module to obtain a marking feature map; asynchronously processing the marking feature map through the improved bottleneck attention module to obtain the backbone features; the processing of the backbone features and the deep layer features through the preset decoder to obtain the metal marking segmentation image comprises: processing the backbone features through a first decoding convolutional layer to obtain backbone convolutional features; performing an up-sampling operation on the deep layer features at a first rate to obtain deep layer up-sampled features; stacking and merging the backbone convolutional features and the deep layer up-sampled features to obtain a decoding feature map; processing the decoding feature map through a second decoding convolutional layer to obtain a decoding convolutional feature map; performing an up-sampling operation on the decoding convolutional feature map at a second rate to obtain the metal marking segmentation image.

2. The metal-marking image processing method of claim 1, wherein the processing of the metal marking image through the multi-branch feature extraction module to obtain a marking feature map comprises: processing the metal marking image through a plurality of first extraction convolutional layers to obtain initial marking features; performing first down-sampling and channel adjustment on the initial marking features to obtain channel adjusted features; processing the channel adjusted features through a second extraction convolutional layer and performing second down-sampling processing to obtain high layer features; processing the high layer features through a multi-branch network to obtain a plurality of branch features; splicing the plurality of branch features along the channel dimension to obtain the marking feature map.

3. The metal-marking image processing method of claim 2, wherein, the processing of the high layer features through the multi-branch network to obtain a plurality of branch features comprises: performing average pooling and first branch channel adjustment on the high layer features to obtain first branch features; performing second branch channel adjustment on the high layer features to obtain second branch features; performing third branch channel adjustment on the high layer features and processing using a high layer feature convolutional layer to obtain third branch features; performing fourth branch channel adjustment on the high layer features and processing using two 3x3 convolutional layers to obtain fourth branch features.

4. The metal-marking image processing method of claim 1, wherein, the asynchronously processing of the marking feature map through the improved bottleneck attention module to obtain the backbone features comprises: processing the marking feature map through a feature transformation branch network to obtain feature transformation branch features; the feature transformation branch network comprises a first transformation convolutional layer, a second transformation convolutional layer, and a third transformation convolutional layer; processing the marking feature map through an attention weight branch network to obtain attention weights; the attention weight branch network comprises an average pooling layer, a multi-layer perceptron, and a batch normalization layer; fusing the feature transformation branch features and the attention weights to obtain weighted features; Add the marking feature map and the weighted feature to obtain the backbone feature.

5. The metal-marking image processing method of claim 1, wherein, The preset encoder network is used to process the backbone feature to obtain a deep feature. The first encoder convolutional layer is used to process the backbone feature to obtain a first encoded feature; the first encoder convolutional layer is a 1x1 convolutional layer; The second encoder convolutional layer is used to process the backbone feature to obtain a second encoded feature; the second encoder convolutional layer includes a plurality of 3x3 first atrous convolutional layers; the span of the first atrous convolutional layer is less than or equal to 6; The third encoder convolutional layer is used to process the backbone feature to obtain a third encoded feature; the third encoder convolutional layer includes a plurality of 3x3 second atrous convolutional layers; the span of the second atrous convolutional layer is greater than or equal to 10 and less than or equal to 14; The fourth encoder convolutional layer is used to process the backbone feature to obtain a fourth encoded feature; the fourth encoder convolutional layer includes a plurality of 3x3 third atrous convolutional layers; the span of the third atrous convolutional layer is greater than or equal to 16 and less than or equal to 20; The maximum pooling layer is used to process the backbone feature to obtain a fifth encoded feature; The fusion convolutional layer is used to process the first encoded feature, the second encoded feature, the third encoded feature, the fourth encoded feature and the fifth encoded feature to obtain the deep feature.

6. A metal marking image processing apparatus characterized by comprising: It comprises: An image acquisition module is configured to acquire a metal marking image to be processed; A multi-branch attention backbone network is configured to process the metal marking image to obtain a backbone feature; A preset encoder network is configured to process the backbone feature to obtain a deep feature; A preset decoder is configured to process the backbone feature and the deep feature to obtain a metal marking segmentation image; The multi-branch attention backbone network comprises an asynchronous stacked multi-branch feature extraction module and an improved bottleneck attention module; The multi-branch feature extraction module is configured to process the metal marking image to obtain a marking feature map; The improved bottleneck attention module is configured to asynchronously process the marking feature map to obtain the backbone feature; The preset decoder comprises: A first decoding convolutional layer is configured to process the backbone feature to obtain a backbone convolutional feature; A first upsampling unit is configured to perform an upsampling operation on the deep feature at a first rate to obtain a deep upsampling feature; A stack merging unit is configured to stack and merge the backbone convolutional feature and the deep upsampling feature to obtain a decoding feature map; A second decoding convolutional layer is configured to process the decoding feature map to obtain a decoding convolutional feature map; A second upsampling unit is configured to perform an upsampling operation on the decoding convolutional feature map at a second rate to obtain the metal marking segmentation image.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the metal marking image processing method according to any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executed by the processor to implement the metal marking image processing method according to any one of claims 1-5.

Images