Image semantic segmentation method fusing adaptive receptive field and cross-scale feature interaction

Abstract

The application discloses a method for image semantic segmentation by fusing adaptive receptive field and cross-scale feature interaction, and belongs to the field of small target information extraction, and comprises the following steps: S1, acquiring a pointer instrument image dataset and performing standardization preprocessing, and dividing the processed standardization dataset into training data, verification data and test data; S2, building an improved DeepLabV3+ pointer scale semantic segmentation network model based on a deep learning framework; S3, using a multi-constraint compound loss function, and completing iterative training of the improved DeepLabV3+ network model by using the divided training data and verification data; and S4, inputting the test data into the trained improved DeepLabV3+ network model, and outputting a pointer and scale pixel-level semantic segmentation mask result. The image semantic segmentation method by fusing adaptive receptive field and cross-scale feature interaction solves the problem of imbalance between foreground and background categories, simultaneously strengthens segmentation constraints on the boundaries of the pointer and scale, and improves the edge segmentation precision of the foreground target.

Description

Technical Field

[0001] This invention relates to the field of small target information extraction technology, and in particular to an image semantic segmentation method that integrates adaptive receptive field and cross-scale feature interaction. Background Technology

[0002] Currently, pointer-type instruments, with their advantages of simple structure, strong anti-interference ability, and high reliability, are still widely deployed in industrial scenarios such as power, chemical, petroleum, and manufacturing, serving as important terminal equipment for industrial field parameter monitoring. With the rapid development of industrial automation and intelligent manufacturing, manual inspection methods are no longer sufficient to meet the demands of large-scale, high-frequency, and high-precision monitoring. Achieving automated and intelligent reading and status monitoring of industrial instruments has become an inevitable trend. Against this backdrop, pointer-type instrument information extraction technology based on computer vision has become a research hotspot, its core being the accurate identification and positioning of key targets such as pointers and scales within the instrument dial.

[0003] However, the pointers and scales of analog instruments are typical small, elongated targets, with their total pixel count typically only 1%-5% of the entire instrument image. Their slender shape and extremely low feature proportions pose a significant challenge to accurate semantic segmentation. Existing semantic segmentation methods still have many shortcomings when dealing with elongated targets like pointers and scales, which have a very small pixel proportion. These shortcomings are mainly reflected in the following three aspects: First, there is insufficient capability for extracting features from small targets. Existing deep convolutional neural networks continuously reduce the resolution of feature maps during feature extraction through multiple layers of pooling and stride convolution, leading to the gradual loss of fine-grained details in shallow features. For slender targets like pointers and scales, which have extremely low pixel counts, the feature information they carry is already limited. After multiple downsampling layers, they are easily submerged by background features, resulting in a lack of semantic information about small targets in deep feature maps, making it difficult to support accurate subsequent segmentation.

[0004] Second, the segmentation boundaries are inaccurately located and prone to breakage. Existing mainstream semantic segmentation models (such as DeepLabV3+) have relatively simple decoder structures and insufficient deep-shallow feature fusion mechanisms, failing to fully mine and utilize the rich edge and texture details in shallow features to refine the segmentation boundaries. For targets with continuous, slender shapes, such as pointers and scales, the lack of boundary information directly leads to local breaks and blurred contours in the segmentation results, severely affecting the accuracy of subsequent reading calculations.

[0005] Third, the extreme imbalance between positive and negative samples leads to a bias in model prediction towards the background. Since pointers and tick marks only account for 1%-5% of the total image pixels, the difference in pixel count between the foreground and background is significant, representing a typical extreme class imbalance scenario. Traditional cross-entropy loss functions assign equal weights to all types of samples, and when dealing with this type of problem, they are dominated by the extremely high proportion of background samples. This causes the model to tend to predict pixels as background, further exacerbating the missed detection problem for pointers and tick marks and reducing the model's recall rate for small targets. Summary of the Invention

[0006] The purpose of this invention is to provide an image semantic segmentation method that integrates adaptive receptive field and cross-scale feature interaction to solve the above-mentioned technical problems.

[0007] To achieve the above objectives, this invention provides an image semantic segmentation method that integrates adaptive receptive field and cross-scale feature interaction, comprising the following steps: S1. Obtain the pointer instrument image dataset and perform standardized preprocessing. Divide the processed standardized dataset into training data, validation data, and test data. S2. An improved DeepLabV3+ pointer-scale semantic segmentation network model is built based on a deep learning framework. This network model includes an encoder and a decoder. The encoder uses a lightweight EfficientNetV2-S backbone network to extract Stage 2 low-level detail features and Stage 4 high-level semantic features. It is configured with an ARF-ASPP module for adaptive receptive field, a hollow space, and a pyramid pooling module to perform multi-scale adaptive feature enhancement on the high-level semantic features. The decoder uses a CSFA cross-scale feature alignment interaction module to complete high-low layer feature space alignment, bidirectional attention enhancement, and residual fusion. Then, after convolutional thinning, upsampling, and softmax activation, it outputs a pixel-level segmentation prediction map. S3. Employ a multi-constraint composite loss function that integrates Focal cross-entropy loss, Dice loss, and boundary loss, and use the training and validation data obtained in step S1 to complete the iterative training of the improved DeepLabV3+ network model. S4. Input the test data obtained in step S1 into the improved DeepLabV3+ network model trained in step S3, and output pointer and scale pixel-level semantic segmentation mask results.

[0008] Preferably, the ARF-ASPP module described in step S2 incorporates directional dilated convolution multi-scale branches and dynamic weight prediction branches to adapt to the slender linear targets of the instrument.

[0009] Preferably, the CSFA cross-scale feature alignment interaction module in step S2 includes a deformable spatial alignment unit, a bidirectional cross-attention interaction unit, and a residual fusion unit; wherein, the deformable spatial alignment unit predicts pixel offsets in the x and y directions through depthwise separable convolution. , The original sampling coordinates of the standard convolution The deformable sampling coordinates are obtained by superimposing the predicted pixel offsets. This allows the sampling grid to fit the shape of a slender linear target, enabling adaptive correction of the position of high-level feature pixels and matching the position of the pointer scale foreground pixels of Stage2 low-level detail features and Stage4 high-level semantic features. The bidirectional cross-attention interaction unit is divided into a channel attention branch and a spatial attention branch; the channel attention branch generates channel weights using aligned Stage4 high-level semantic features, and multiplies them channel by channel to filter out invalid background channels with low-level detail features. ; in, ; ; In the formula, This is an optimized shallow feature after filtering out invalid background channels using channel attention; This is a low-level detail feature of Stage 2; This is a channel-by-channel multiplication operation; Channel attention weights; For the first Channel attention weights for channels; Use the Sigmoid activation function; , These are the weight matrices for the first layer of dimension-reduced fully connected layers and the weight matrices for the second layer of dimension-upgraded fully connected layers, respectively. For the first Global average pooling feature vectors for each channel; , These are the height and width of the feature map, respectively; For the aligned high-level features, the first Channel, First line, number The pixel value corresponding to the column position, and , Number of channels; It is the ReLU activation function; The spatial attention branch generates a spatial weight map using Stage2 low-level detail features, and multiplies it pixel-by-pixel to enhance the pointer tick foreground region response of aligned high-level features: ; in, ; ; ; ; In the formula, This refers to the optimized high-rise features after spatial attention enhancement of the foreground region; This is a single-channel spatial attention weight map; coordinates Spatial attention weights at each pixel; It is a standard 3×3 convolutional layer; This is the result of channel splicing; This involves concatenating operations along the channel dimension. For the Stage2 low-level detail features Channel, First line, number The pixel value corresponding to the column position; coordinates The maximum value of each channel; This is a pixel-by-pixel averaged feature map across channels; This is a feature map showing the maximum value across channels pixel by pixel; The residual fusion unit adds the pixel-by-pixel residuals of the Stage2 low-level detail features optimized by channel attention and the Stage4 high-level semantic features optimized by spatial attention, and outputs fine-grained fused features: ; In the formula, This refers to the finely fused features output by the bidirectional cross-attention interaction unit.

[0010] Preferably, in step S3, the training data from step S1 is input into the improved DeepLabV3+ network built in step S2, and all network parameters are updated by backpropagation based on the total loss obtained by the multi-constraint composite loss function. After each round of training, the validation data is input to calculate the segmentation evaluation index mIoU and F1-Score, and the model training is completed by iterating until the index converges. The expression for the multi-constraint composite loss function is as follows: ; in, ; ; ; In the formula, This represents the total loss value of the multi-constraint composite loss function; , , These are the fixed balance weight coefficients corresponding to the Focal cross-entropy loss, Dice loss, and boundary loss, respectively. , , ; , , These are the Focal cross-entropy loss function, the Dice loss function, and the boundary loss function, respectively. This represents the total number of pixels within a single input instrument image. As a category balance factor; To predict the probability value for the current pixel to belong to its true category for the network; Focusing factors for challenging samples; For the first The true segmentation label value corresponding to each pixel, with a value of 1 for foreground pixels and 0 for background pixels; Output the first part to the network Predicted segmentation probability for each pixel; It is a numerical stability smoothing coefficient; The first one in the instrument image The Euclidean distance from pixel to the nearest foreground object edge pixel, and , , The first The horizontal and vertical coordinates of each pixel on a two-dimensional instrument image , The first The horizontal and vertical coordinates of each pixel on a two-dimensional instrument image It is the set of all edge pixels of the pointer and scale foreground target in the real label mask.

[0011] Preferably, step S4 specifically includes the following steps: S41. Input the instrument image into the EfficientNetV2-S of the improved DeepLabV3+ pointer scale semantic segmentation network model that has been built to complete multi-level feature extraction, and output the 1 / 4 resolution low-level detail features corresponding to Stage 2 and the 1 / 16 resolution high-level semantic features corresponding to Stage 4. S42. Input the 1 / 16 resolution high-level semantic features corresponding to Stage4 output in step S41 into the ARF-ASPP module to complete multi-scale adaptive feature enhancement. The ARF-ASPP module sets four groups of directional dilated convolutions with dilation rates of 2, 4, 6, and 12 in its multi-scale branch. The sampling points of the directional dilated convolutions are arranged along a linear radial direction to capture the thin linear features of the pointer scale. The dynamic weight prediction branch takes the 1 / 16 resolution high-level semantic features corresponding to the Stage4 level output in step S31 as input, and outputs a four-dimensional weight vector after global average pooling and two fully connected layers. After Softmax normalization, adaptive weights are assigned to the four groups of directional dilated convolution branches. S43. The four weighted directional dilated convolution features are concatenated by channels, and then 1×1 convolution branch and global pooling branch features are fused. After 1×1 convolution dimensionality reduction, the enhanced high-level features after adaptive multi-scale optimization are output. S44. Input the enhanced high-level features output in step S43 into the CSFA cross-scale feature alignment interaction module of the decoder and perform 4x bilinear upsampling to obtain a feature map with the same resolution as the 1 / 4 resolution low-level detail features corresponding to the Stage2 level output in step S21. Then, send the feature map and the 1 / 4 resolution low-level detail features corresponding to the Stage2 level into the CSFA cross-scale feature alignment interaction module to complete feature alignment and bidirectional enhancement fusion to obtain fine fused features. S45. The finely fused features are fed into two consecutive 3×3 convolutional layers to complete the contour smoothing and thinning. After thinning, the features are upsampled by 4 times to restore the original resolution of the input instrument image. S46. The features upsampled in step S45 are fed into the Softmax activation layer, and the predicted probability of each pixel belonging to the background, pointer, and tick categories is output to generate the original size semantic segmentation prediction map.

[0012] Therefore, the image semantic segmentation method of the present invention, which integrates adaptive receptive field and cross-scale feature interaction, has the following beneficial effects: 1. The lightweight EfficientNetV2-S backbone network replaces the native Xception, which significantly reduces the number of model parameters and inference time while fully preserving the shallow edge details of pointers and scales. This meets the needs of edge hardware deployment in industrial settings and solves the problems of large computational load and easy loss of small target details in traditional deep networks.

[0013] 2. The ARF-ASPP adaptive receptive field void space pyramid pooling module uses radially oriented void convolution to adapt to slender linear targets, and is paired with dynamic weight prediction branches to adaptively match dials of different sizes. Compared with fixed void rate ASPP, it can accurately capture multi-scale features of pointer scales of different lengths and thicknesses, eliminating the defect of poor adaptability of global semantic extraction.

[0014] 3. The CSFA cross-scale feature alignment interaction module introduces deformable convolution to correct the misalignment of deep and shallow feature spaces. Combined with dual-channel bidirectional cross attention, it enables the bidirectional mutual guidance and fusion of shallow details and high-level semantics, effectively solving the problems of broken outlines and blurred boundaries in pointer and scale segmentation, and significantly improving the segmentation integrity of slender targets.

[0015] 4. By integrating Focal cross-entropy, Dice loss, and boundary loss into a multi-constraint composite loss function, the model simultaneously addresses three major pain points: imbalance between extreme foreground and background samples, low cross-union ratio in foreground segmentation, and insufficient edge segmentation accuracy. This reduces the false negative rate of fine scale and short pointer, and improves the model's ability to identify small, difficult-to-segment targets.

[0016] In summary, this invention improves the segmentation accuracy and boundary integrity of slender small targets by optimizing the feature extraction structure, enhancing the fusion of deep and shallow features, and improving the loss function. It eliminates the need for manually designing complex image preprocessing rules and outputs end-to-end pointers and scales with precise segmentation masks, which can directly support automatic instrument readings and fault identification, thereby improving the intelligence and automation level of industrial inspection and providing reliable technical support for the intelligent monitoring of industrial instruments.

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0018] Figure 1 This is a flowchart of the image semantic segmentation method of the present invention that integrates adaptive receptive field and cross-scale feature interaction; Figure 2 This is a diagram illustrating the overall architecture of the improved DeepLabV3+ pointer tick semantic segmentation network model built for this invention. Figure 3 ARF-ASPP architecture diagram of the improved DeepLabV3+ pointer tick semantic segmentation network model built for this invention; Figure 4 This is a visual comparison of the results of the present invention and the traditional method. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the embodiments of the present invention and are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0020] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as a process, method, system, product, or server that includes a series of steps or units, not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device.

[0021] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0022] like Figures 1-3 As shown, the image semantic segmentation method that integrates adaptive receptive field and cross-scale feature interaction includes the following steps: S1. Obtain the pointer instrument image dataset and perform standardized preprocessing. Divide the processed standardized dataset into training data, validation data, and test data. S2. An improved DeepLabV3+ pointer-scale semantic segmentation network model is built based on a deep learning framework. This network model includes an encoder and a decoder. The encoder uses a lightweight EfficientNetV2-S backbone network to extract Stage 2 low-level detail features and Stage 4 high-level semantic features. It is configured with an ARF-ASPP module and an adaptive receptive field hole spatial pyramid pooling module to perform multi-scale adaptive feature enhancement on high-level semantic features. The decoder uses a CSFA cross-scale feature alignment interaction module to complete high-low layer feature spatial alignment, bidirectional attention enhancement, and residual fusion. After convolution thinning, upsampling, and Softmax activation, a pixel-level segmentation prediction map is output. Through three major network structure improvements—lightweight backbone network, adaptive multi-scale pooling, and cross-scale alignment interaction fusion—the problems of detail loss, spatial misalignment, and segmentation fragmentation of slender small targets are solved. S3. Employ a multi-constraint composite loss function that integrates Focal cross-entropy loss, Dice loss, and boundary loss, and use the training and validation data obtained in step S1 to complete the iterative training of the improved DeepLabV3+ network model. S4. Input the test data obtained in step S1 into the improved DeepLabV3+ network model trained in step S3, and output pointer and scale pixel-level semantic segmentation mask results.

[0023] Table 1. Parameters of Lightweight EfficientNetV2-S Backbone Network

[0024] The ARF-ASPP module described in step S2 incorporates directional dilated convolution multi-scale branches and dynamic weight prediction branches to adapt to the slender linear targets of the instrument.

[0025] The CSFA cross-scale feature alignment interaction module described in step S2 includes a deformable spatial alignment unit, a bidirectional cross-attention interaction unit, and a residual fusion unit; wherein, the deformable spatial alignment unit predicts pixel offsets in the x and y directions through depthwise separable convolution. , The original sampling coordinates of the standard convolution The deformable sampling coordinates are obtained by superimposing the predicted pixel offsets. This allows the sampling grid to fit the shape of a slender linear target, enabling adaptive correction of the position of high-level feature pixels and matching the position of the pointer scale foreground pixels of Stage2 low-level detail features and Stage4 high-level semantic features. The bidirectional cross-attention interaction unit is divided into a channel attention branch and a spatial attention branch; the channel attention branch generates channel weights using aligned Stage4 high-level semantic features, and multiplies them channel by channel to filter out invalid background channels with low-level detail features. ; in, ; ; In the formula, This is an optimized shallow feature after filtering out invalid background channels using channel attention; This is a low-level detail feature of Stage 2; This is a channel-by-channel multiplication operation; Channel attention weights; For the first Channel attention weights for channels; Use the Sigmoid activation function; , These are the weight matrices for the first layer of dimension-reduced fully connected layers and the weight matrices for the second layer of dimension-upgraded fully connected layers, respectively. For the first Global average pooling feature vectors for each channel; , These are the height and width of the feature map, respectively; For the aligned high-level features, the first Channel, First line, number The pixel value corresponding to the column position, and , Number of channels; It is the ReLU activation function; The spatial attention branch generates a spatial weight map using Stage2 low-level detail features, and multiplies it pixel-by-pixel to enhance the pointer tick foreground region response of aligned high-level features: ; in, ; ; ; ; In the formula, This refers to the optimized high-rise features after spatial attention enhancement of the foreground region; This is a single-channel spatial attention weight map; coordinates Spatial attention weights at each pixel; It is a standard 3×3 convolutional layer; This is the result of channel splicing; This involves concatenating operations along the channel dimension. For the Stage2 low-level detail features Channel, First line, number The pixel value corresponding to the column position; coordinates The maximum value of each channel; This is a pixel-by-pixel averaged feature map across channels; This is a feature map showing the maximum value across channels pixel by pixel; The residual fusion unit adds the pixel-by-pixel residuals of the Stage2 low-level detail features optimized by channel attention and the Stage4 high-level semantic features optimized by spatial attention, and outputs fine-grained fused features: ; In the formula, This refers to the finely fused features output by the bidirectional cross-attention interaction unit.

[0026] In step S3, the training data from step S1 is input into the improved DeepLabV3+ network built in step S2. All network parameters are updated by backpropagation based on the total loss obtained from the multi-constraint composite loss function. After each round of training, the validation data is input to calculate the segmentation evaluation index mIoU and F1-Score. The iteration continues until the index converges to complete the model training. The expression for the multi-constraint composite loss function is as follows: ; in, ; ; ; In the formula, This represents the total loss value of the multi-constraint composite loss function; , , These are the fixed balance weight coefficients corresponding to the Focal cross-entropy loss, Dice loss, and boundary loss, respectively. , , ; , , These are the Focal cross-entropy loss function, the Dice loss function, and the boundary loss function, respectively. This represents the total number of pixels within a single input instrument image. As a category balance factor; To predict the probability value for the current pixel to belong to its true category for the network; Focusing factors for challenging samples; For the first The true segmentation label value corresponding to each pixel, with a value of 1 for foreground pixels and 0 for background pixels; Output the first part to the network Predicted segmentation probability for each pixel; It is a numerical stability smoothing coefficient; The first one in the instrument image The Euclidean distance from pixel to the nearest foreground object edge pixel, and , , The first The horizontal and vertical coordinates of each pixel on a two-dimensional instrument image , The first The horizontal and vertical coordinates of each pixel on a two-dimensional instrument image It is the set of all edge pixels of the pointer and scale foreground target in the real label mask.

[0027] Step S4 specifically includes the following steps: S41. Input the instrument image into the EfficientNetV2-S of the improved DeepLabV3+ pointer scale semantic segmentation network model that has been built to complete multi-level feature extraction, and output the 1 / 4 resolution low-level detail features corresponding to Stage 2 and the 1 / 16 resolution high-level semantic features corresponding to Stage 4. S42. Input the 1 / 16 resolution high-level semantic features corresponding to Stage4 output in step S41 into the ARF-ASPP module to complete multi-scale adaptive feature enhancement. The ARF-ASPP module sets four groups of directional dilated convolutions with dilation rates of 2, 4, 6, and 12 in its multi-scale branch. The sampling points of the directional dilated convolutions are arranged along a linear radial direction to capture the thin linear features of the pointer scale. The dynamic weight prediction branch takes the 1 / 16 resolution high-level semantic features corresponding to the Stage4 level output in step S31 as input, and outputs a four-dimensional weight vector after global average pooling and two fully connected layers. After Softmax normalization, adaptive weights are assigned to the four groups of directional dilated convolution branches. S43. The four weighted directional dilated convolution features are concatenated by channels, and then 1×1 convolution branch and global pooling branch features are fused. After 1×1 convolution dimensionality reduction, the enhanced high-level features after adaptive multi-scale optimization are output. S44. Input the enhanced high-level features output in step S43 into the CSFA cross-scale feature alignment interaction module of the decoder and perform 4x bilinear upsampling to obtain a feature map with the same resolution as the 1 / 4 resolution low-level detail features corresponding to the Stage2 level output in step S21. Then, send the feature map and the 1 / 4 resolution low-level detail features corresponding to the Stage2 level into the CSFA cross-scale feature alignment interaction module to complete feature alignment and bidirectional enhancement fusion to obtain fine fused features. S45. The finely fused features are fed into two consecutive 3×3 convolutional layers to complete the contour smoothing and thinning. After thinning, the features are upsampled by 4 times to restore the original resolution of the input instrument image. S46. The features upsampled in step S45 are fed into the Softmax activation layer, and the predicted probability of each pixel belonging to the background, pointer, and tick categories is output to generate the original size semantic segmentation prediction map.

[0028] Simulation Experiment Experimental hardware and software environment Hardware: An industrial server equipped with an NVIDIA ARTX 3090 graphics card and 64GB of RAM; Software: Python 3.8, PyTorch 1.10 deep learning framework, OpenCV image preprocessing library; Training hyperparameters: The optimizer used is AdamW, with a base learning rate of 1e-4, a batch size of 8, a total number of epochs of 120, and a weight decay coefficient of 5e-4.

[0029] Dataset Construction A total of 6200 images of various pointer-type instruments, such as pressure gauges and thermometers, were collected from industrial sites and uniformly scaled to 512×512 pixels. The LabelMe annotation tool was used to annotate the background, pointer, and scale with semantic labels pixel by pixel. The training set consisted of 4340 images, the validation set consisted of 1240 images, and the test set consisted of 620 images, divided in a 7:2:1 ratio. Data augmentation was performed using random flipping, random brightness perturbation, Gaussian noise, and slight rotation to alleviate the problem of insufficient sample size.

[0030] Comparative experimental setup Four mainstream semantic segmentation models were selected as benchmarks: native DeepLabV3+, U-Net, HRNet, and SegFormer. All experimental conditions, including unified dataset, training epochs, optimizers, and input resolution, were used to ensure fairness in the comparison.

[0031] The improved scheme of this invention fixes all structural parameters: the backbone network uses EfficientNetV2-S, and the hierarchical parameters follow... Figure 2 The table configuration shows that Stage 2 outputs shallow detail features at 1 / 4 resolution and Stage 4 outputs high-level semantic features at 1 / 16 resolution. The encoder is equipped with an ARF-ASPP module, and the decoder adopts a complete CSFA three-unit structure. The training phase uses a Focal+Dice+boundary composite loss function.

[0032] Evaluation indicators The common quantitative indicators for semantic segmentation, namely mean intersection-union ratio (mIoU), mPA, and F1 score, are used as the quantitative evaluation criteria. Mean Intersection over Union (mIoU): The arithmetic mean of the IoU calculated for the background, pointer, and tick targets is used to measure the degree of overlap between the model's predicted mask and the real labeled mask. The higher the value, the higher the segmentation integrity of the pointer and tick targets. Average pixel accuracy (mPA): The average of the proportion of correctly predicted pixels to all true pixels in each category. It reflects the accuracy of the model in classifying each type of pixel and can intuitively demonstrate the pixel recognition accuracy of small targets with fine scales and short pointers. F1 score: A comprehensive evaluation metric that combines precision and recall to balance the two types of defects: missed detections and false detections. It is specifically adapted to the extreme class imbalance scenarios in this invention where the foreground pixels only account for 1%-5% of the image. The higher the value, the fewer the number of missed and false detections by the pointer and scale.

[0033] Table 2 Quantitative Experiment Results

[0034] As shown in Table 2, the improved model of this invention outperforms all the comparison algorithms in both the core segmentation metrics of mIoU and F1, achieving the best overall segmentation performance. Compared to the baseline native DeepLabV3+, mIoU is improved by 2.63% and F1 by 2.1%; compared to HRNet, which has the best overall performance, mIoU is improved by 2.45% and F1 by 1.7%. The traditional lightweight network PSPNet has the lowest performance in all metrics. This network lacks a multi-scale feature capture mechanism, resulting in severe segmentation failures when dealing with thin pointers and scales with extremely low pixel ratios. SegFormer has some feature extraction capabilities by relying on global modeling with Transformer, but it lacks directional radial dilated convolution to adapt to linear targets, and dense scales are prone to adhesion and missed detections, resulting in an mIoU that is 6.88% lower than that of this invention. U-Net has strong shallow detail preservation capabilities, but it lacks adaptive receptive fields and cross-scale feature alignment structures, resulting in prominent pointer segmentation breakage problems, and its F1 score is 1.9 lower than that of this invention, thus proving the effectiveness of this invention.

[0035] At the same time, such as Figure 4 As shown, the segmentation results of HRNet, native DeepLabV3+, SegFormer, and U-Net all have obvious defects, such as large areas of missing fine scales, broken pointers, overlapping scale lines, and severe jagged edges on the target. HRNet can detect most pointers and scales, but the outlines of extremely fine and short scales are blurred and the edge fitting deviation is large, which cannot support the linear fitting calculation of instrument readings. The improved model of this invention outputs a complete and continuous mask, with no broken pointers, no overlapping dense scales, and the target edges smoothly fit the real lines of the instrument. There are no missed or falsely detected pixels, and it can completely restore the geometric shape of fine linear targets, directly serving upper-level applications such as automatic reading and fault identification of industrial instruments.

[0036] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. An image semantic segmentation method that integrates adaptive receptive field and cross-scale feature interaction, characterized by: Includes the following steps: S1. Obtain the pointer instrument image dataset and perform standardized preprocessing. Divide the processed standardized dataset into training data, validation data, and test data. S2. An improved DeepLabV3+ pointer-scale semantic segmentation network model is built based on a deep learning framework. This network model includes an encoder and a decoder. The encoder uses a lightweight EfficientNetV2-S backbone network to extract Stage 2 low-level detail features and Stage 4 high-level semantic features. It is configured with an ARF-ASPP module for adaptive receptive field, a hollow space, and a pyramid pooling module to perform multi-scale adaptive feature enhancement on the high-level semantic features. The decoder uses a CSFA cross-scale feature alignment interaction module to complete high-low layer feature space alignment, bidirectional attention enhancement, and residual fusion. Then, after convolutional thinning, upsampling, and softmax activation, it outputs a pixel-level segmentation prediction map. S3. Employ a multi-constraint composite loss function that integrates Focal cross-entropy loss, Dice loss, and boundary loss, and use the training and validation data obtained in step S1 to complete the iterative training of the improved DeepLabV3+ network model. S4. Input the test data obtained in step S1 into the improved DeepLabV3+ network model trained in step S3, and output pointer and scale pixel-level semantic segmentation mask results.

2. The image semantic segmentation method that integrates adaptive receptive field and cross-scale feature interaction according to claim 1, characterized in that: The ARF-ASPP module described in step S2 incorporates directional dilated convolution multi-scale branches and dynamic weight prediction branches to adapt to the slender linear targets of the instrument.

3. The image semantic segmentation method that integrates adaptive receptive field and cross-scale feature interaction according to claim 1, characterized in that: The CSFA cross-scale feature alignment interaction module described in step S2 includes a deformable spatial alignment unit, a bidirectional cross-attention interaction unit, and a residual fusion unit; wherein, the deformable spatial alignment unit predicts pixel offsets in the x and y directions through depthwise separable convolution. , The original sampling coordinates of the standard convolution The deformable sampling coordinates are obtained by superimposing the predicted pixel offsets. This allows the sampling grid to fit the shape of a slender linear target, enabling adaptive correction of the position of high-level feature pixels and matching the position of the pointer scale foreground pixels of Stage2 low-level detail features and Stage4 high-level semantic features. The bidirectional cross-attention interaction unit is divided into a channel attention branch and a spatial attention branch; the channel attention branch generates channel weights using aligned Stage4 high-level semantic features, and multiplies them channel by channel to filter out invalid background channels with low-level detail features. ； in, ； ； In the formula, This is an optimized shallow feature after filtering out invalid background channels using channel attention; This refers to the low-level detail features of Stage 2; This is a channel-by-channel multiplication operation; Channel attention weights; For the first Channel attention weights for channels; Use the Sigmoid activation function; , These are the weight matrices for the first layer of dimension-reduced fully connected layers and the weight matrices for the second layer of dimension-upgraded fully connected layers, respectively. For the first Global average pooling feature vectors for each channel; , These are the height and width of the feature map, respectively; For the aligned high-level features, the first Channel, First line, number The pixel value corresponding to the column position, and , Number of channels; It is the ReLU activation function; The spatial attention branch generates a spatial weight map using Stage2 low-level detail features, and multiplies it pixel-by-pixel to enhance the pointer tick foreground region response of aligned high-level features: ； in, ； ； ； ； In the formula, This refers to the optimized high-rise features after spatial attention enhancement of the foreground region; This is a single-channel spatial attention weight map; coordinates Spatial attention weights at each pixel; It is a standard 3×3 convolutional layer; This is the result of channel splicing; This involves concatenating operations along the channel dimension. For the Stage2 low-level detail features Channel, First line, number The pixel value corresponding to the column position; coordinates The maximum value of each channel; This is a pixel-by-pixel averaged feature map across channels; This is a feature map showing the maximum value across channels pixel by pixel; The residual fusion unit adds the pixel-by-pixel residuals of the Stage2 low-level detail features optimized by channel attention and the Stage4 high-level semantic features optimized by spatial attention, and outputs fine-grained fused features: ； In the formula, This refers to the finely fused features output by the bidirectional cross-attention interaction unit.

4. The image semantic segmentation method according to claim 3, which integrates adaptive receptive field and cross-scale feature interaction, is characterized in that: In step S3, the training data from step S1 is input into the improved DeepLabV3+ network built in step S2. All network parameters are updated by backpropagation based on the total loss obtained from the multi-constraint composite loss function. After each round of training, the validation data is input to calculate the segmentation evaluation index mIoU and F1-Score. The iteration continues until the index converges to complete the model training. The expression for the multi-constraint composite loss function is as follows: ； in, ； ； ； In the formula, This represents the total loss value of the multi-constraint composite loss function. , , These are the fixed balance weight coefficients corresponding to the Focal cross-entropy loss, Dice loss, and boundary loss, respectively. , , ; , , These are the Focal cross-entropy loss function, the Dice loss function, and the boundary loss function, respectively. This represents the total number of pixels within a single input instrument image. As a category balance factor; To predict the probability value for the current pixel to belong to its true category for the network; Focusing factors for challenging samples; For the first The true segmentation label value corresponding to each pixel, with a value of 1 for foreground pixels and 0 for background pixels; Output the first part to the network Predicted segmentation probability for each pixel; It is a numerical stability smoothing coefficient; The first one in the instrument image The Euclidean distance from pixel to the nearest foreground object edge pixel, and , , The first The horizontal and vertical coordinates of each pixel on a two-dimensional instrument image , The first The horizontal and vertical coordinates of each pixel on a two-dimensional instrument image It is the set of all edge pixels of the pointer and scale foreground target in the real label mask.

5. The image semantic segmentation method according to claim 4, which integrates adaptive receptive field and cross-scale feature interaction, is characterized in that: Step S4 specifically includes the following steps: S41. Input the instrument image into the EfficientNetV2-S of the improved DeepLabV3+ pointer scale semantic segmentation network model that has been built to complete multi-level feature extraction, and output the 1 / 4 resolution low-level detail features corresponding to Stage 2 and the 1 / 16 resolution high-level semantic features corresponding to Stage 4. S42. Input the 1 / 16 resolution high-level semantic features corresponding to Stage4 output in step S41 into the ARF-ASPP module to complete multi-scale adaptive feature enhancement. The ARF-ASPP module sets four groups of directional dilated convolutions with dilation rates of 2, 4, 6, and 12 in its multi-scale branch. The sampling points of the directional dilated convolutions are arranged along a linear radial direction to capture the thin linear features of the pointer scale. The dynamic weight prediction branch takes the 1 / 16 resolution high-level semantic features corresponding to the Stage4 level output in step S31 as input, and outputs a four-dimensional weight vector after global average pooling and two fully connected layers. After Softmax normalization, adaptive weights are assigned to the four groups of directional dilated convolution branches. S43. The four weighted directional dilated convolution features are concatenated by channels, and then 1×1 convolution branch and global pooling branch features are fused. After 1×1 convolution dimensionality reduction, the enhanced high-level features after adaptive multi-scale optimization are output. S44. Input the enhanced high-level features output in step S43 into the CSFA cross-scale feature alignment interaction module of the decoder and perform 4x bilinear upsampling to obtain a feature map with the same resolution as the 1 / 4 resolution low-level detail features corresponding to the Stage2 level output in step S21. Then, send the feature map and the 1 / 4 resolution low-level detail features corresponding to the Stage2 level into the CSFA cross-scale feature alignment interaction module to complete feature alignment and bidirectional enhancement fusion to obtain fine fused features. S45. The finely fused features are fed into two consecutive 3×3 convolutional layers to complete the contour smoothing and thinning. After thinning, the features are upsampled by 4 times to restore the original resolution of the input instrument image. S46. The features upsampled in step S45 are fed into the Softmax activation layer, and the predicted probability of each pixel belonging to the background, pointer, and tick categories is output to generate the original size semantic segmentation prediction map.

Images