Data processing method and device based on mixed attention mechanism, equipment and medium

Abstract

The invention relates to the technical field of image processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a data processing method, device and equipment based on a mixed attention mechanism and a medium, and the method comprises the steps: carrying out the preprocessing of a to-be-processed image, and generating preprocessed image data; performing multi-layer convolution through an encoder module to generate a multi-layer encoding feature map; in each level jump connection between the encoder and the decoder, processing the multi-layer encoding feature map through a mixed attention module to generate a mixed attention feature map; the mixed attention feature map is transmitted to a decoder module, and a hierarchical input feature map is generated through feature fusion; and performing up-sampling processing on the hierarchical input feature map to generate an output feature map matched with the spatial resolution of the to-be-processed image. According to the method, the mixed attention module is introduced between the encoder and the decoder to enhance multi-scale feature expression, the spatial resolution is recovered through up-sampling, and the feature extraction and semantic analysis precision in a complex scene is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a data processing method, apparatus, device and storage medium based on a hybrid attention mechanism. Background Art

[0002] In the FinTech sector, image processing technology has been widely used in scenarios such as identity authentication, document review, and risk control. For example, when opening a bank account or applying for a loan, users are required to upload images of documents such as their ID cards and driver's licenses. The system must automatically identify and extract key information from these documents, such as name, ID number, and expiration date. However, existing technologies face multiple challenges in identifying and extracting the subject matter of the image. First, in complex backgrounds, ID images may contain a large amount of interference, such as background debris, light reflections, or other non-target areas. These interference elements can cause the edges of the target document to be unclear or obscured, thereby affecting document positioning and recognition accuracy. Second, for low-resolution or blurred images, such as those captured by mobile phone cameras, existing technologies struggle to reliably extract key information, especially fine details such as characters and icons. While existing convolutional neural networks (CNNs) and attention mechanisms can improve target area recognition to a certain extent, feature extraction by existing models is less robust when faced with complex background interference and low-resolution images, making them prone to false or missed detections, and even misidentifying non-target areas as document content.

[0003] In the medical and health business field, image processing technology is widely used in medical image analysis, lesion detection and health record management. For example, in the analysis of lung CT images, the system needs to automatically identify and extract the lung area, and further detect the lesions or abnormal areas therein. However, the existing technology still has many shortcomings when facing complex medical images. First, in CT images, the boundaries between lung tissue and other organs or tissues are blurred. Especially under low-dose CT scans, the noise effect is obvious, resulting in blurred boundaries or unclear structures. Secondly, when extracting target areas (such as lung nodules or lesions), the existing technology is easily interfered by other non-target tissues, such as blood vessels, trachea or surrounding tissues, resulting in false detection or missed detection. In addition, for multi-level complex medical images, the existing convolutional neural network shows certain limitations in capturing features of different scales in the image, and cannot effectively distinguish the subtle differences between the lesion area and normal tissue. Although the attention mechanism has been introduced to improve the feature focusing ability of the model, in multi-level and multi-scale medical image processing, the existing attention mechanism has difficulty in accurately capturing the target area, especially when the lesion area is small or has low contrast with the surrounding tissue, the detection effect of the model is significantly reduced. Summary of the Invention

[0004] The main purpose of the present invention is to provide a data processing method, device, equipment and storage medium based on a hybrid attention mechanism, aiming to solve the technical problem that the existing technology lacks a cross-level hybrid attention mechanism in the encoder and decoder architecture, resulting in insufficient multi-scale feature fusion and insufficient spatial semantic alignment accuracy, which limits the image parsing capability of complex scenes.

[0005] To achieve the above objectives, the present invention provides a data processing method based on a hybrid attention mechanism, comprising:

[0006] Receive the image to be processed and perform preprocessing operations to generate preprocessed image data;

[0007] Input the pre-processed image data into the encoder module to perform multi-layer convolution processing to generate a multi-layer encoding feature map;

[0008] In each level of skip connection between the encoder module and the decoder module, the multi-layer encoding feature map is processed layer by layer by the hybrid attention module to generate a hybrid attention feature map of the corresponding layer;

[0009] The mixed attention feature maps of each layer are passed to the corresponding decoding layer of the decoder module, and the layer input feature maps are generated through feature fusion;

[0010] Performing hierarchical upsampling processing on the hierarchical input feature map to generate an output feature map that matches the spatial resolution of the image to be processed.

[0011] Furthermore, to achieve the above objectives, the present invention provides a data processing device based on a hybrid attention mechanism, comprising:

[0012] A preprocessing module is used to receive the image to be processed and perform preprocessing operations to generate preprocessed image data;

[0013] An encoder module, configured to input the pre-processed image data into an encoder module to perform multi-layer convolution processing to generate a multi-layer encoding feature map;

[0014] A hybrid attention module is configured to process the multi-layer encoding feature map layer by layer in the skip connection between the encoder module and the decoder module to generate a hybrid attention feature map of the corresponding layer;

[0015] The feature fusion module is used to transfer the mixed attention feature maps of each layer to the corresponding decoding layer of the decoder module, and generate the layer input feature map through feature fusion;

[0016] A decoder module is used to perform hierarchical upsampling processing on the hierarchical input feature map to generate an output feature map that matches the spatial resolution of the image to be processed.

[0017] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer device, which includes a memory, a processor, and a data processing program based on a hybrid attention mechanism stored in the memory and run on the processor. When the data processing program based on the hybrid attention mechanism is executed by the processor, the steps of the data processing method based on the hybrid attention mechanism as described above are implemented.

[0018] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a data processing program based on a hybrid attention mechanism is stored. When the data processing program based on the hybrid attention mechanism is executed by a processor, the steps of the data processing method based on the hybrid attention mechanism as described above are implemented.

[0019] Beneficial effects: The present invention relates to the field of image processing technology and can be applied to business scenarios such as financial technology and medical health. It discloses a data processing method, device, equipment and medium based on a hybrid attention mechanism, including: receiving an image to be processed and performing preprocessing operations to generate preprocessed image data; inputting the preprocessed image data into an encoder module to perform multi-layer convolution processing to generate a multi-layer encoding feature map; in each level jump connection between the encoder module and the decoder module, the multi-layer encoding feature map is processed layer by layer by the hybrid attention module to generate a hybrid attention feature map of the corresponding level; the hybrid attention feature map of each level is passed to the corresponding decoding level of the decoder module, and a level input feature map is generated through feature fusion; and level upsampling processing is performed on the level input feature map to generate an output feature map that matches the spatial resolution of the image to be processed. The present invention introduces a hybrid attention module in the skip connections at each level between the encoder and decoder to achieve joint modeling of channel features and spatial features, thereby enhancing the richness and accuracy of feature expression; on the decoder side, spatial resolution is gradually restored through feature fusion and layer-by-layer upsampling, ensuring that the output feature map remains consistent with the spatial structure of the input image, significantly improving the ability to extract and analyze image features in a variety of complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0021] Figure 1 A schematic diagram of an application environment of a data processing method based on a hybrid attention mechanism in one embodiment of the present invention;

[0022] Figure 2 This is a flow chart of an embodiment of a data processing method based on a hybrid attention mechanism of the present invention;

[0023] Figure 3Schematic diagram of functional modules of a preferred embodiment of a data processing device based on a hybrid attention mechanism of the present invention;

[0024] Figure 4 A schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0025] Figure 5 FIG. 2 is another structural diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0027] The data processing method based on the hybrid attention mechanism provided by the embodiment of the present invention can be applied in the following situations: Figure 1 In an application environment, the user end communicates with the server end through a network. The server end can receive the image to be processed through the user end and perform preprocessing operations to generate preprocessed image data; the preprocessed image data is input into the encoder module to perform multi-layer convolution processing to generate a multi-layer coding feature map; in the jump connection of each level between the encoder module and the decoder module, the multi-layer coding feature map is processed layer by layer by the hybrid attention module to generate a hybrid attention feature map of the corresponding level; the hybrid attention feature map of each level is passed to the corresponding decoding level of the decoder module, and a level input feature map is generated by feature fusion; the level input feature map is subjected to level upsampling processing to generate an output feature map that matches the spatial resolution of the image to be processed. The present invention realizes the joint modeling of channel features and spatial features by introducing a hybrid attention module in the jump connection of each level between the encoder and the decoder, thereby enhancing the richness and accuracy of feature expression; on the decoder side, the spatial resolution is gradually restored by feature fusion and layer-by-layer upsampling to ensure that the output feature map remains consistent with the spatial structure of the input image, significantly improving the ability to extract and analyze image features in a variety of complex scenes. The user end may be, but is not limited to, various personal computers, laptops, smartphones, tablet computers, and portable wearable devices. The server end may be implemented as an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below using specific embodiments.

[0028] See also Figure 2 , Figure 2 This is a flow chart of an embodiment of a data processing method based on a hybrid attention mechanism provided by the present invention. It should be noted that although a logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than shown here.

[0029] like Figure 2As shown, the data processing method based on the hybrid attention mechanism proposed in the present invention includes the following steps:

[0030] S10, receiving an image to be processed and performing a preprocessing operation to generate preprocessed image data;

[0031] In this embodiment, an image to be processed is received and preprocessed to generate preprocessed image data. The input image to be processed can be a static image, such as a document image, ID photo, or medical image in JPEG or PNG format, or a dynamic image frame, such as a single frame in a video stream. The preprocessing operation is intended to provide high-quality input data for subsequent feature extraction and multi-scale feature analysis.

[0032] Preprocessing begins with color space conversion, converting the input image from RGB to another color space, such as YUV, Lab, or HSV, depending on the target task. The goal of color space conversion is to enhance the information representation within specific color channels, facilitating subsequent feature extraction. For example, in medical imaging, the luminance channel Y in YUV space can better highlight tissue structure. In document image processing tasks in the financial sector, the hue channel H in the HSV color space can be used for background separation.

[0033] After color space conversion, contrast stretching is performed. Contrast stretching is an enhancement technique based on pixel intensity distribution that increases brightness differences in an image, making dark and bright areas clearer. Specifically, methods such as histogram equalization and adaptive contrast enhancement (CLAHE) can be used. For example, in ID card images, contrast stretching can make text and portraits clearer; in medical images, it can enhance the contrast between soft tissue and the background.

[0034] After contrast enhancement, the image data is normalized to adjust pixel values ​​to a preset range, such as 0-1 or -1 to 1. Normalization not only improves computational stability but also ensures that gradient instability caused by excessively large or small values ​​when propagating image data through the neural network is avoided. Specific implementations of normalization include min-max normalization and mean-standard deviation normalization. For financial document images, normalization ensures that images of varying sizes maintain the same numerical distribution when fed into the network.

[0035] After normalization, noise reduction is performed using a nonlinear filtering module. The goal of nonlinear filtering is to remove high-frequency noise from the image while maintaining edge clarity. Techniques such as median filtering, bilateral filtering, or Gaussian filtering can be used. Bilateral filtering offers the advantage of reducing noise while preserving edge detail. In financial document images, bilateral filtering can remove background texture noise while maintaining text clarity. In medical imaging, it can remove high-frequency noise within tissue regions, enhancing the clarity of lesions.

[0036] Example: In the healthcare field, one implementation approach is applied to automatic lesion detection in CT images. The received CT image is first converted to the YUV color space, and the luminance channel Y is extracted to enhance tissue contrast. Adaptive histogram equalization is then performed on the enhanced luminance image to highlight tissue structural details. Median filtering is used to remove high-frequency noise and ensure sharp edges. The resulting preprocessed image data is then fed into subsequent feature extraction and classification models to accurately identify lesion areas.

[0037] In financial applications, the images to be processed are ID card photos. The input image is converted to the HSV color space, the hue channel H is extracted for background separation, and the lightness channel V is retained to enhance the contrast between text and portraits. The enhanced image is then subjected to min-max normalization, adjusting pixel values ​​to [0, 1] to ensure computational stability. Finally, bilateral filtering is applied to remove background texture noise while maintaining the clarity of text and portraits. This preprocessed image data is fed into the subsequent identity extraction and verification model, significantly improving the accuracy of identity extraction.

[0038] This embodiment converts the input image into high-quality preprocessed image data through preprocessing operations including color space conversion, contrast stretching, pixel value normalization, and nonlinear filtering. This not only eliminates noise and redundant information but also enhances the image's feature representation. Whether used for financial ID images or medical CT images, this preprocessing method significantly improves the accuracy of subsequent feature extraction and classification.

[0039] S20, inputting the pre-processed image data into an encoder module to perform multi-layer convolution processing to generate a multi-layer encoding feature map;

[0040] In this embodiment, the preprocessed image data is input into the encoder module, where multi-layer convolution processing is performed to generate a multi-layer encoded feature map. The encoder module is a deep neural network structure specifically designed to extract multi-level feature representations from the input image. The preprocessed image data can be a single-channel grayscale image or a three-channel color image. Regardless of the number of color channels in the input image, the encoder module can extract feature information layer by layer through convolution operations, generating feature representations at different scales and depths.

[0041] First, the preprocessed image data is fed into the first convolutional layer of the encoder. A convolutional layer is a neural network layer that extracts local features from the input image using trainable convolution kernels (or filters). Each convolution kernel slides across space, calculating a weighted sum pixel by pixel to generate a feature map. The first convolutional layer typically uses a smaller convolution kernel size, such as 3×3 or 5×5, to ensure fine-grained feature extraction. The result of the convolution operation is called the initial convolution feature map, which preserves the spatial structure of the input image.

[0042] After the initial convolutional feature map is generated, it is normalized using a batch normalization layer. Batch normalization is a technique that improves training stability and model generalization by normalizing the distribution of values ​​in the feature map to zero mean and unit variance. Batch normalization not only eliminates exploding or vanishing gradients during training but also accelerates model convergence.

[0043] The normalized feature map is then fed into a nonlinear activation function, such as ReLU (Rectified Linear Unit) or Leaky ReLU. Nonlinear activation functions introduce nonlinear expressiveness to neural networks, ensuring the model can fit complex patterns and features. The ReLU function effectively suppresses negative values, setting all values ​​less than zero to zero while leaving positive values ​​unchanged. Leaky ReLU, on the other hand, allows negative values ​​to be retained with a smaller slope, mitigating the "neuron death" phenomenon in ReLU.

[0044] The activated feature map undergoes dimensionality reduction via strided convolution. Strided convolution is a convolution method that achieves spatial downsampling by increasing the convolution stride. Compared to standard convolution, strided convolution reduces the spatial resolution of the feature map while still extracting features. For example, a strided convolution with a stride of 2 halves the width and height of the feature map. This approach reduces the spatial resolution of the feature map layer by layer while increasing its level of abstraction.

[0045] The above process is repeated at each level of the encoder module. Each level includes convolution, batch normalization, activation, and strided convolution operations. The feature maps generated layer by layer form a multi-layer encoding feature map, representing simple features such as edges and textures at the low level to complex structures and semantic information at the high level. The number and size of convolution kernels at each level can be adjusted. Lower levels usually use fewer convolution kernels and smaller convolution kernels to capture detailed features; higher levels use more convolution kernels and larger convolution kernels to enhance semantic expression capabilities.

[0046] In one implementation, the preprocessed image data is a three-channel RGB image, which is fed into the first convolutional layer of the encoder module. This first convolutional layer uses 64 3×3 convolution kernels with a stride of 1 and a padding mode of "same" to ensure that the output feature map has the same size as the input. The convolution result undergoes batch normalization to standardize the distribution of values, and then a nonlinear transformation is performed using the ReLU activation function. The activated feature map undergoes dimensionality reduction using strided convolution with a stride of 2, halving its width and height.

[0047] In another implementation, the preprocessed image data is a single-channel grayscale image. The first convolutional layer uses 32 5×5 kernels with a stride of 1. The convolution results are batch normalized and activated with a Leaky ReLU activation coefficient of 0.1, allowing negative values ​​to be retained with a slope of 0.1. The activated feature map is then reduced in dimension using a strided convolution with a stride of 2. The encoder consists of five convolutional layers, with the number of feature maps in each layer increasing to 32, 64, 128, 256, and 512, respectively, capturing information from edge to high-level semantics.

[0048] In an efficient implementation, strided convolution can be replaced with a pooling operation (such as max pooling or average pooling) to achieve dimensionality reduction. Max pooling can preserve salient features in the feature map, while average pooling can remove high-frequency noise. In the activation feature map after each convolution layer, 2×2 max pooling is used for downsampling.

[0049] Example: In the healthcare field, for lung CT imaging, the input CT image is first fed into the encoder module, where it undergoes multi-layer convolution. Low-level convolutional layers extract tissue edges and vascular structures, while high-level convolutional layers capture semantic information about the lung region. The resulting multi-layer encoded feature map is then processed in the subsequent hybrid attention module to improve the accuracy of lesion recognition.

[0050] In the financial sector, ID card image processing involves preprocessing the ID card image into an encoder module, where multi-layer convolutions generate multi-layer encoded feature maps. Low-level feature maps preserve the edge information of the text and portrait on the ID card, while high-level feature maps extract the overall structure and semantic information of the ID card. These multi-layer feature maps are further processed in the subsequent decoder module to generate clear ID card region and text extraction results, facilitating identity verification tasks.

[0051] Through multi-layer convolution processing, the encoder module in this embodiment can extract multi-scale, multi-level feature representations from the input image layer by layer, from low-level edges and textures to high-level semantic information. Batch normalization and nonlinear activation functions ensure the numerical stability and expressiveness of the feature map, while strided convolution achieves effective spatial dimensionality reduction. This multi-level feature extraction method can not only capture rich visual information in complex scenes, but also provide high-quality feature input in the subsequent decoding stage, significantly improving the accuracy of image parsing.

[0052] S30, in each level of skip connection between the encoder module and the decoder module, processing the multi-layer encoding feature map layer by layer through the hybrid attention module to generate a hybrid attention feature map of the corresponding layer;

[0053] In this embodiment, in the skip connections between the encoder module and the decoder module at each level, the hybrid attention module processes the multi-layer encoding feature map layer by layer to generate the hybrid attention feature map of the corresponding layer. Skip connection is a design strategy for retaining and transmitting low-level features in deep neural networks. It can directly transmit the output feature map of each layer in the encoder to the corresponding layer of the decoder. This cross-level feature transfer can avoid the problem of low-level features being lost in deep networks and improve the expressiveness of features.

[0054] During each skip connection, each feature map from the multi-layer encoded feature maps is fed into a hybrid attention module. The hybrid attention module is a deep neural network structure that simultaneously captures the correlation between channel and spatial features, ensuring that each feature map effectively represents both global and local information of the input image. The hybrid attention module employs a dual attention mechanism: channel attention and spatial attention.

[0055] First, the encoded feature map of the current layer is input into the channel attention branch, where channel attention calculation is performed. Channel attention is a technique that improves feature representation by learning the importance weights of each channel. The channel attention branch typically uses global average pooling or global maximum pooling operations to compress the spatial information of each channel into a single value, thereby generating a channel description vector. This channel description vector is processed by a multi-layer perceptron (MLP) to generate attention weights for each channel. Finally, the attention weights are multiplied channel by channel with the original feature map to generate a channel attention map. Channel attention can enhance channels with high-importance information and suppress irrelevant or redundant features.

[0056] The encoded feature map of the current level is then divided into multiple window feature blocks of predetermined size, each representing a local region in the feature map. Self-attention calculations are performed within each window. Self-attention is a mechanism that captures the correlation between features within a window. By calculating the correlation between each feature point and other feature points, a window self-attention map is generated. Window self-attention calculations typically employ a multi-head self-attention mechanism, where each head independently learns different feature relationships, thereby enhancing feature representation.

[0057] After the channel attention map and window self-attention map are generated, they are linearly weighted and fused. The goal of weighted fusion is to balance information representation between channel-level and spatial-level features. This fusion can be performed using simple weighted addition or by further learning weight coefficients through a multi-layer perceptron. The fused feature map is called an intermediate attention feature map, which combines the expressive power of both channel and spatial features.

[0058] Finally, the intermediate attention feature map is connected to the original encoded feature map through a residual connection. Residual connections are an effective mechanism in deep neural networks to prevent gradient vanishing and preserve original information. Through residual connections, the hybrid attention feature map not only retains the information of the original feature map but also enhances the expressiveness of high-level semantic features.

[0059] The above process is repeated in the skip connections of each layer, generating hybrid attention feature maps layer by layer. The hybrid attention feature maps of each layer contain a dual enhanced representation of channel and spatial information, providing high-quality feature input for the subsequent decoder module.

[0060] In one implementation, the channel attention branch in the hybrid attention module uses global average pooling combined with a two-layer perceptron (MLP). Global average pooling compresses the spatial information of each channel into a single value. The two-layer perceptron uses a Reluctant Unit (ReLU) activation function in the first layer and a Sigmoid in the second layer, outputting attention weights for each channel. The channel attention map is combined with the input feature map through a channel-by-channel weighted multiplication.

[0061] In another implementation, window self-attention uses a multi-head self-attention mechanism, with each window size being 7×7. The window feature block generates multi-head self-attention features through self-attention calculations. The output of each head is recombined through a linear layer to generate a window self-attention map. For self-attention calculations, a standard dot-product self-attention mechanism is used, where the query vector, key vector, and value vector for each feature point are independently generated through a linear layer.

[0062] Another possible implementation is to use a combination where, after the channel attention and window self-attention maps are generated separately, instead of simply adding them together, they are further fused using a multi-layer perceptron (MLP). This MLP receives the concatenation of the channel attention map and the window self-attention map and outputs a fusion weight vector to guide the weighted fusion of the two attention maps.

[0063] Example: In lung CT image analysis in healthcare, the hybrid attention module effectively extracts salient features of lesion regions from multi-layer encoded feature maps. Channel attention enhances the representation of lung structure and lesion regions, while window self-attention captures detailed features surrounding the lesion. During the decoding phase, the hybrid attention feature map significantly improves lesion detection and segmentation accuracy.

[0064] In identity authentication scenarios in the financial sector, the hybrid attention module helps extract key information from ID card images from multi-layered encoded feature maps. Channel attention retains high-weight information in the text and portrait areas on the ID card, while window self-attention accurately identifies the ID card's edges and background. The feature map generated during the decoding phase ensures accurate segmentation of the ID card area, providing high-quality feature input for subsequent identity information extraction and verification.

[0065] This embodiment can effectively enhance feature representation in the jump connection between the encoder and decoder through the hybrid attention module. Channel attention enhances the expression of important channels in the feature map and reduces the interference of irrelevant information, while window self-attention can capture the correlation between local features and improve the expressiveness of spatial features. Through the combination of channel and spatial attention, the hybrid attention feature map has both detailed and semantic expression capabilities in feature representation. In the subsequent decoding process, this enhanced feature map can significantly improve the recognition and segmentation accuracy of objects in complex scenes.

[0066] S40, passing the mixed attention feature map of each level to the corresponding decoding level of the decoder module, and generating a level input feature map through feature fusion;

[0067] In this embodiment, after the multi-layer encoding feature maps are processed by the hybrid attention module and the hybrid attention feature maps are generated, these feature maps need to be passed to each layer of the decoder module, and the hierarchical input feature maps are generated through feature fusion. The hybrid attention feature maps are not directly passed in this process, but are first adapted to the spatial and channel features to ensure that they can be correctly applied and processed in the decoder.

[0068] The hybrid attention feature map at each level first undergoes spatial resolution matching. This adjustment ensures that the spatial size of the hybrid attention feature map is consistent with the input resolution of the corresponding decoder layer. Spatial resolution matching can be achieved in a variety of ways, such as bilinear interpolation, bicubic interpolation, or transposed convolution. For feature maps with higher resolution, bilinear interpolation is often used to improve computational efficiency, while for feature maps with lower resolution, transposed convolution can be used to ensure that the generated feature map can capture more detailed information.

[0069] After spatial resolution matching, the hybrid attention feature map also requires channel number adjustment. This adjustment is due to the potential discrepancies in the channel counts between the encoder and decoder feature maps, especially when fusion of multiple layers of feature maps is required. Channel number adjustment is typically implemented using 1×1 convolutions, an efficient channel transformation operation that flexibly adjusts the number of feature map channels without changing the spatial resolution. This channel number adjustment ensures that the hybrid attention feature map is compatible with the decoder-level feature maps in terms of the channel dimension, laying the foundation for subsequent feature fusion.

[0070] After completing spatial and channel adaptation, the decoder module retrieves the decoded output feature map from the previous layer. This map represents the features generated layer by layer by the decoder and represents the intermediate results of gradually restoring spatial resolution. The purpose of obtaining the previous layer's output feature map is to fuse high-level features with low-level features through skip connections. This cross-level information fusion improves the decoder's feature representation capabilities.

[0071] After the hybrid attention feature map and the previous layer's decoded output feature map are prepared, they are concatenated along the channel dimension. Channel-wise concatenation is an efficient feature fusion method that preserves the complete information of both features and further learns the associations between them during the subsequent decoding process. The concatenated feature map is called the target-level input feature map. It contains both the hybrid attention feature information passed from the encoder and the high-level feature information recovered layer by layer from the decoder.

[0072] Finally, the target-level input feature map is fed into the corresponding decoding layer of the decoder module. After receiving the target-level input feature map, the decoder performs convolution, batch normalization, and activation operations to further process the feature map. Through this feature fusion mechanism, the decoder is able to gradually restore the spatial resolution and detail information of the image at each level while preserving the high-quality semantic features extracted by the hybrid attention feature.

[0073] In one implementation, spatial resolution matching is achieved through bilinear interpolation. Bilinear interpolation is an efficient spatial resizing method that can quickly adjust the resolution of feature maps while preserving image edge details. During the channel number adjustment stage, a 1×1 convolution is used to adjust the number of channels in the mixed attention feature map to the number of channels in the corresponding decoder layer, for example, from 256 channels to 128 channels.

[0074] In another implementation, spatial resolution matching is achieved using transposed convolutions. Transposed convolutions are a method that progressively upsamples feature maps while maintaining efficient computation. Channel adjustment still utilizes 1×1 convolutions, but batch normalization and activation functions are added to the convolutional layers to further enhance feature representation.

[0075] Another approach is to use multi-resolution fusion. In this approach, the hybrid attention feature map is replicated into multiple versions at different resolutions, corresponding to the inputs of different layers in the decoder. Each resolution version of the hybrid attention feature map is then concatenated with the feature maps from each layer of the decoder after adjusting the number of channels. This multi-resolution fusion approach can further enhance the decoder's ability to express features at different scales.

[0076] Example: In the healthcare field of lung CT image segmentation, the mixed attention feature maps at each level are passed to the decoder module, where they are resized to match the resolution and channel count of the decoder input through bilinear interpolation and 1×1 convolution. Each layer's mixed attention feature map is concatenated with the decoder's high-level features and then fed into the decoder. The detailed features of lung tissue are restored layer by layer, ensuring that the final segmentation output accurately labels lesions and normal tissue.

[0077] In identity authentication image processing in the financial sector, the hybrid attention feature map is resized to the decoder input resolution through bilinear interpolation before being passed to the decoder module, and the number of channels is adjusted through 1×1 convolution. The resulting hierarchical input feature map after feature fusion preserves key information in ID card images, such as text areas and head contours. In the decoder, these features are further processed and enhanced, enabling the output to segment and identify important information in ID card images with high accuracy.

[0078] This embodiment can significantly improve the efficiency of feature transfer and fusion by transferring the hybrid attention feature map of each level to the corresponding decoding level of the decoder module, and generating a hierarchical input feature map through feature fusion. Spatial resolution matching adjustment ensures that the feature map can be consistent with the decoder input in spatial scale, avoiding information loss due to resolution difference. Channel number adjustment ensures the compatibility of the feature map in the channel dimension, avoiding feature loss or inability to splice caused by channel number mismatch. Cross-level feature fusion improves the feature expression ability and detail retention effect by combining the hybrid attention feature with the high-level features of the decoder, enabling the decoder to more accurately restore the image spatial structure.

[0079] S50, performing hierarchical upsampling processing on the hierarchical input feature map to generate an output feature map that matches the spatial resolution of the image to be processed.

[0080] In this embodiment, after the hierarchical input feature map is generated, it is further restored to the same spatial resolution as the image being processed through hierarchical upsampling to generate the output feature map. Hierarchical upsampling is a process of restoring spatial resolution layer by layer, ensuring that the final output feature map retains high-precision spatial information while preserving the high-level semantic features extracted by the encoder and hybrid attention module.

[0081] First, the layer-by-layer upsampling process is performed on each layer in the decoder module. The input feature map of each layer is first subjected to a transposed convolution operation. Transposed convolution is a convolution method that can achieve upsampling. Its essence is to increase the spatial resolution by inserting zero padding between the input feature maps. The kernel size and stride of the transposed convolution determine the upsampling factor. For example, a 3×3 convolution kernel with a stride of 2 can double the resolution of the input feature map.

[0082] In the primary upsampled feature maps generated by transposed convolution, batch normalization is performed immediately because the distribution of feature values ​​may be unstable. Batch normalization is a standardization method that normalizes the values ​​in the feature map to a standard distribution range by calculating the mean and standard deviation of the feature map within the batch. The purpose of batch normalization is to stabilize gradient propagation, avoiding gradient vanishing or gradient exploding phenomena in the feature map values, thereby improving the stability of feature transfer during the upsampling process.

[0083] After batch normalization, the normalized feature maps are activated using a nonlinear activation function. Common activation functions include ReLU (Rectified Linear Unit), Leaky ReLU, and Swish. Activation functions introduce nonlinear features, ensuring that the upsampled feature maps not only have high resolution but also retain nonlinear information in the feature representation. For example, ReLU sets negative numbers to 0 and retains positive numbers, improving the sparsity of the feature map, while Swish allows for smoother preservation of negative features.

[0084] After transposed convolution, batch normalization, and activation function processing, an upsampled feature map is generated for the current layer. This feature map is then passed to the next layer of the decoder, where the upsampling and processing process repeats. Each layer of the decoder gradually restores the spatial resolution based on the upsampled feature map of the previous layer, until the final layer generates an output feature map with the same resolution as the original input image to be processed.

[0085] This layer-by-layer upsampling and restoration process ensures that the feature map retains mixed attention features and high-level semantic features while gradually increasing spatial resolution, avoiding feature loss or blurred details caused by large-scale upsampling at one time. Furthermore, batch normalization and activation function processing at each level further improve the stability and expressiveness of the feature map.

[0086] In one implementation, layer-by-layer upsampling is achieved through transposed convolutions, with a kernel size of 3×3 and a stride of 2 at each layer. This configuration doubles the resolution of each layer, restoring the spatial resolution to match the processed image layer by layer. Batch normalization uses the standard BatchNorm operation, with the normalized values ​​set to a standard distribution (mean 0, standard deviation 1), and the ReLU activation function.

[0087] In another implementation, bilinear interpolation can be used instead of transposed convolution for upsampling. Bilinear interpolation is a non-convolutional upsampling method that generates a high-resolution feature map by weighting adjacent pixels. This method is computationally efficient, but does not introduce additional trainable parameters during the upsampling process. Batch normalization and activation function remain unchanged, using ReLU.

[0088] Another approach is to combine multi-level upsampling with a pyramid structure. Specifically, at each decoder level, upsampling not only increases resolution through transposed convolution but also preserves low-resolution feature maps and fuses them with the upsampled feature maps through skip connections. This approach preserves more multi-scale features and improves the decoder's ability to recover detail.

[0089] Example: In medical image segmentation in healthcare, layer-by-layer upsampling can gradually restore the spatial details of individual tissues and lesions in CT images. For example, in lung CT images, layer-by-layer upsampling of feature maps can finely restore the boundary details of alveoli, bronchi, and lung nodules, ensuring that the segmentation results accurately cover the target area.

[0090] In identity authentication image processing in the financial sector, layer-by-layer upsampling can gradually restore the high-resolution features of ID card and passport images. During this layer-by-layer upsampling process, the feature map retains the boundary details of text and portraits, enabling the final output feature map to accurately segment text and face regions. This feature significantly improves the accuracy of document recognition and information extraction.

[0091] This embodiment uses a layer-by-layer upsampling process to gradually restore the spatial resolution of the feature map while retaining high-level semantic feature information. Transposed convolution ensures that the upsampling process can capture spatial details, batch normalization improves feature stability, and the activation function introduces nonlinear features, making the output feature map not only high-resolution but also rich in semantic information. Compared with direct upsampling to the target resolution, this layer-by-layer upsampling method can effectively avoid feature loss and detail blurring, improving the detail fidelity and semantic expression ability of the output feature map.

[0092] The present invention relates to the field of image processing technology and can be applied to business scenarios such as financial technology and medical health. A data processing method, device, equipment and medium based on a hybrid attention mechanism are disclosed, including: receiving an image to be processed and performing preprocessing operations to generate preprocessed image data; inputting the preprocessed image data into an encoder module to perform multi-layer convolution processing to generate a multi-layer encoding feature map; in each level of skip connection between the encoder module and the decoder module, processing the multi-layer encoding feature map layer by layer through a hybrid attention module to generate a hybrid attention feature map of the corresponding level; transmitting the hybrid attention feature map of each level to the corresponding decoding level of the decoder module, generating a level input feature map through feature fusion; performing level upsampling processing on the level input feature map to generate an output feature map that matches the spatial resolution of the image to be processed. The present invention introduces a hybrid attention module in each level of skip connection between the encoder and the decoder to achieve joint modeling of channel features and spatial features, thereby enhancing the richness and accuracy of feature expression; on the decoder side, the spatial resolution is gradually restored through feature fusion and level-by-level upsampling, ensuring that the output feature map remains consistent with the spatial structure of the input image, significantly improving the ability to extract and analyze image features in various complex scenarios.

[0093] In one embodiment, the above step S10 includes:

[0094] S101, performing color space conversion processing on the input image to be processed to generate intermediate image data;

[0095] S102, performing a contrast stretching operation on the intermediate image data to generate enhanced image data;

[0096] S103, normalizing the pixel values ​​of the enhanced image data to a preset value range to generate normalized image data;

[0097] S104: Perform noise elimination processing on the normalized image data through a nonlinear filtering module to generate pre-processed image data.

[0098] In this embodiment, the process of receiving an image to be processed and performing preprocessing operations to generate preprocessed image data is designed to ensure the quality and data feature stability of the input image to meet the requirements of subsequent feature extraction and processing. The preprocessing operation improves the usability of the image data through a series of image enhancement and optimization steps, ensuring that key image information can be accurately recognized and extracted even in complex scenes or low-quality images.

[0099] Image data processing begins with the receipt of the image to be processed. The input image can come from a variety of sources, such as real-time image acquisition by a camera, document images scanned by a scanner, medical images acquired by medical imaging equipment (such as CT or MRI), or existing image data retrieved from an image database. The input image can be an RGB color image, a grayscale image, or a multi-channel image in another format.

[0100] After receiving an image, color space conversion is first performed to generate intermediate image data. Color space conversion is a fundamental operation in image processing, used to convert an image from one color representation to another. Common color space conversions include converting from RGB (red, green, and blue) to YCbCr (luminance-chrominance) color space, or from RGB to HSV (hue-saturation-value) color space. The purpose of this conversion is to separate luminance and chrominance information, making subsequent contrast adjustment and noise processing more convenient. For example, in ID card recognition, using the HSV space can more easily enhance the contrast of text areas, while in medical image processing, the YCbCr color space is more helpful in distinguishing tissue structures.

[0101] The intermediate image data after color space conversion is then subjected to contrast stretching. Contrast stretching is an image enhancement technique used to expand the brightness range of an image, making dark areas clearer and bright areas more prominent. Common contrast stretching methods include linear stretching and histogram equalization. In linear stretching, the brightness values ​​of all pixels are stretched to a preset range by setting minimum and maximum brightness values. For example, the original brightness range can be adjusted from 50-200 to 0-255. In histogram equalization, the brightness distribution of the image is made more uniform by redistributing pixel brightness values. This is particularly suitable for medical images with complex details or images captured in low light conditions.

[0102] After contrast stretching, the enhanced image data is subjected to pixel value normalization, converting the pixel values ​​into a preset numerical range (usually 0-1 or -1 to 1). Normalization can eliminate the interference of brightness changes on subsequent feature extraction, while improving the training stability of the model. In deep learning models, normalized image data can speed up the convergence of gradient descent and avoid gradient explosion or gradient vanishing problems. The normalization method can be linear normalization (scaling the pixel values ​​directly to the target range) or Z-score normalization (normalizing the pixel values ​​to a standard normal distribution).

[0103] Finally, the normalized image data is processed for noise removal through a nonlinear filtering module to generate the final preprocessed image data. Nonlinear filtering is an efficient noise reduction method that can remove noise while preserving image details. For example, median filtering is a classic nonlinear filtering method that eliminates salt and pepper noise by replacing pixel values ​​with the median of neighboring pixels. Bilateral filtering can smooth noisy areas while preserving edge details, and is suitable for medical images or document images with complex backgrounds. The parameters of nonlinear filtering (such as filter window size and strength) can be dynamically adjusted according to different application scenarios.

[0104] This embodiment significantly improves the quality and feature stability of input images through preprocessing operations. Color space conversion separates luminance and chrominance information, making subsequent enhancement processing more flexible. Contrast stretching improves the image's brightness contrast and enhances detail. Normalization eliminates the impact of brightness differences on feature extraction, improving the stability of model training. Nonlinear filtering effectively removes noise while preserving image detail. This overall preprocessing operation ensures consistent quality of input image data, improving the accuracy of subsequent feature extraction and analysis.

[0105] In one embodiment, the above step S20 includes:

[0106] S201, inputting the pre-processed image data into the first convolutional layer of the encoder module to perform feature extraction operation to generate an initial convolution feature map;

[0107] S202, processing the initial convolutional feature map through a batch normalization layer of an encoder module to generate a normalized feature map;

[0108] S203, applying a nonlinear activation function to the normalized feature map to generate an activated feature map;

[0109] S204, performing strided convolution dimensionality reduction processing on the activation feature map to generate a reduced dimensionality feature map, and using the reduced dimensionality feature map as the first-level encoding feature map;

[0110] S205, repeatedly performing the feature extraction operation, batch normalization processing, application of nonlinear activation function and strided convolution dimensionality reduction processing, generating the coding feature map of each convolution layer layer by layer, and using the coding feature map of each convolution layer as a multi-layer coding feature map.

[0111] In this embodiment, the preprocessed image data is fed into the encoder module for multi-layer convolution processing. The process of generating a multi-layer encoded feature map aims to extract multi-level feature information from the image, ensuring that both detailed features and high-level semantic information in the image are effectively extracted and represented. The encoder module's multi-layer convolutional structure captures feature information at different scales. Through layer-by-layer convolution and dimensionality reduction operations, it gradually extracts spatial and channel features from the image, forming a multi-layer feature representation.

[0112] First, the preprocessed image data is input into the first convolutional layer of the encoder module to perform feature extraction. The convolution layer is a core image feature extraction structure that performs spatial convolution operations on the input image through a set of learnable convolution kernels to extract low-level features such as edges, textures, and color distribution. The size of the convolution kernel can be set as needed, such as 3×3, 5×5, or 7×7. In each convolution operation, the convolution kernel slides pixel by pixel and generates a feature response map by performing a dot product operation with the image area. For color images, the convolution kernel performs convolution calculations on each channel (such as R, G, B) separately, and accumulates the convolution results to generate a single-channel or multi-channel feature map.

[0113] After the convolution operation, the initial convolutional feature map is processed through the batch normalization layer to generate a normalized feature map. Batch normalization is a standardization method that can stabilize feature distribution and avoid training instability caused by exploding or vanishing gradients. The batch normalization layer calculates the mean and standard deviation of the feature maps of the current batch and normalizes the feature maps, ensuring that the feature data distribution has a relatively consistent mean (usually 0) and standard deviation (usually 1). This not only accelerates model convergence but also improves the model's generalization ability. In practical applications, batch normalization can also dynamically update the mean and variance to adapt to changes in different batches of data.

[0114] The normalized feature map is further applied with a nonlinear activation function to generate an activated feature map. The activation function is a core component of convolutional neural networks, introducing nonlinear transformations and enabling the model to possess nonlinear expressive capabilities. Common nonlinear activation functions include ReLU (Rectified Linear Unit), Leaky ReLU, Sigmoid, and Tanh. ReLU is a commonly used activation function whose output is defined as max(0, x), effectively eliminating negative features while preserving positive ones. Leaky ReLU builds on this by preserving negative features, with its output defined as max(αx, x), where α is a constant less than 1. Sigmoid and Tanh functions are suitable for mapping output features to a specific numerical range (such as 0-1 or -1 to 1). In multi-layer convolutional structures, the nonlinear transformations introduced by nonlinear activation functions significantly enhance feature expressiveness.

[0115] The activated feature map then undergoes strided convolution dimensionality reduction to generate a reduced-dimensional feature map. Strided convolution is a special convolution operation that reduces the spatial resolution of the feature map by increasing the convolution stride. For example, a strided convolution stride of 2 means that every other pixel is sampled, thereby halving the width and height of the feature map. This dimensionality reduction operation can reduce the computational complexity of the feature map while maintaining the expressive power of the features, thereby improving processing efficiency. Strided convolution dimensionality reduction not only reduces the size of the feature map in space, but also avoids feature loss caused by pooling operations, allowing the feature information to be more completely preserved.

[0116] The reduced-dimensional feature map serves as the first-level encoding feature map and enters the subsequent layers of the encoder module, where feature extraction, batch normalization, nonlinear activation, and strided convolution dimensionality reduction are repeatedly performed. The convolution kernel, activation function, and strided convolution parameters at each layer can be dynamically configured based on application requirements. For example, the initial layers use 3×3 convolution kernels to capture detailed features, while higher layers use 5×5 or 7×7 convolution kernels to extract semantic information. The stride of the strided convolution can also be dynamically adjusted at different layers, for example, using a stride of 1 at shallow layers and a stride of 2 at deeper layers for greater dimensionality reduction.

[0117] As the number of layers increases, the encoded feature maps gradually extract low-level features (such as edges and textures) to high-level semantic features (such as object shape and category information). Ultimately, the encoded feature maps generated by all layers are combined to form a multi-layer encoded feature map. In a multi-layer encoded feature map, the features at each layer express different semantic information. Low-level features focus on details, while high-level features focus on global structure and semantics.

[0118] This embodiment performs multi-layer convolution processing by inputting pre-processed image data into the encoder module to generate a multi-layer encoded feature map, which can extract rich multi-level feature information from the image. Low-level feature maps retain detail information such as edges, textures, and color distributions, while high-level feature maps extract semantic information such as object shape and category. This multi-level feature representation not only improves the comprehensiveness and semantic expression ability of feature expression, but also provides a more discriminative feature foundation for the subsequent hybrid attention module, significantly improving the feature expression and target parsing capabilities in complex backgrounds and multi-target images.

[0119] In one embodiment, the above step S30 includes:

[0120] S301, performing channel attention processing on the encoding feature map of the current level to generate a channel attention map;

[0121] S302, dividing the coding feature map of the current level into a plurality of window feature blocks of predetermined sizes;

[0122] S303, performing self-attention association processing in each window feature block to generate a window self-attention map;

[0123] S304, performing linear weighted fusion on the channel attention map and the window self-attention map to generate an intermediate attention feature map;

[0124] S305, performing a residual connection between the intermediate attention feature map and the encoding feature map of the current level to generate a mixed attention feature map.

[0125] In this embodiment, in the skip connections between the encoder and decoder modules, the hybrid attention module processes the multi-layered encoded feature maps layer by layer to generate hybrid attention feature maps for the corresponding layers. This process aims to fully utilize the multi-layer feature maps generated by the encoder and enhance feature expression capabilities through the hybrid attention mechanism, ensuring high-precision image feature analysis in complex scenes.

[0126] First, channel attention processing is performed on the encoded feature map of the current level to generate a channel attention map. Channel attention is a mechanism that focuses on the information correlation between channels in the feature map. By calculating the global statistical information of each channel (such as global average pooling or global maximum pooling), the importance of each channel in the overall feature is evaluated. Taking global average pooling as an example, channel attention can be achieved by calculating the global average of each channel and converting it into a weight coefficient through a fully connected layer and an activation function (such as Sigmoid or ReLU). The higher the weight coefficient, the more important the channel information is. The features of each channel are reweighted according to their corresponding weights to generate a channel attention map. The introduction of channel attention can effectively improve the channel selectivity of the feature map, allowing the network to pay more attention to the feature channels with discriminative properties.

[0127] Then, the encoded feature map of the current level is divided into multiple window feature blocks of predetermined sizes. Window partitioning is an operation that divides the feature map into small areas. Common window sizes include 3×3, 5×5, 7×7, etc. This partitioning can generate multiple local areas on the feature map, and each area is processed independently. The core of window partitioning is to ensure that each window can cover sufficient spatial information while retaining spatial local features. The size of the window can be dynamically adjusted according to the image resolution, layer depth, and computing resources. For example, a larger window size can be used in a high-resolution feature map to reduce the amount of computation, while a smaller window can be used in a low-resolution feature map to enhance the capture of detailed features.

[0128] Self-attention association processing is performed within each window feature block to generate a window self-attention map. Self-attention is a mechanism that captures the correlation between pixels within a feature map. It generates an attention matrix by calculating the similarity between each pixel and other pixels. Self-attention is typically implemented based on a dot product calculation: First, the window feature block undergoes a linear transformation to generate query, key, and value matrices. The dot product between the query matrix and the key matrix is ​​then calculated to generate attention weights. These weights are normalized using a softmax function to represent the degree of correlation between each pixel and other pixels. Finally, the attention weights are multiplied by the value matrix to generate a window self-attention map. The self-attention mechanism can capture long-range dependencies between pixels and improve the spatial representation of feature maps.

[0129] Next, the channel attention map and the window self-attention map are linearly weighted fused to generate an intermediate attention feature map. Linear weighted fusion is a way to integrate multiple attention results into a single feature representation. In specific implementations, weight coefficients (such as α and β) can be set for channel attention and window self-attention respectively, and the sum of the weights is 1. The calculation formula for this weighted fusion is:

[0130] Intermediate attention feature map = α × channel attention map + β × window self-attention map

[0131] The weighting coefficients can be automatically learned during the training process, ensuring that the attention weights are dynamically adjusted in different scenarios to achieve optimal feature enhancement effects.

[0132] Finally, the intermediate attention feature map is connected to the encoding feature map of the current layer through a residual connection to generate a mixed attention feature map. The residual connection is an operation that adds the input features and the enhanced features, ensuring that the original feature information is retained while enhancing the feature expression capability. The formula for the residual connection is expressed as:

[0133] Mixed attention feature map = intermediate attention feature map + encoding feature map of the current level

[0134] This design not only improves the trainability of features, but also prevents information loss caused by over-enhancement of features by the attention module. The hybrid attention feature map retains the combination of original features and attention-enhanced features, making the feature representation more comprehensive.

[0135] This embodiment achieves channel and spatial dual attention enhancement by introducing a hybrid attention module in the jump connection of each level between the encoder and the decoder. Channel attention ensures that the semantic expression of each feature channel conforms to the global information distribution, and self-attention captures the spatial correlation between each feature pixel. Linear weighted fusion balances the contribution of the two attention features, and the residual connection retains the original feature information, significantly improving the diversity and accuracy of feature expression. The hybrid attention feature map can effectively enhance the feature expression of the target object in high-complexity scenes (such as low light and complex background), enabling the subsequent decoder to more accurately parse the target object.

[0136] In one embodiment, the above step S40 includes:

[0137] S401, performing spatial resolution matching adjustment on the hybrid attention feature map of the current level to generate a hybrid attention feature map with matching spatial size;

[0138] S402, performing a channel number adjustment convolution operation on the hybrid attention feature map with matching spatial size to generate an adjusted feature map with matching channel number;

[0139] S403, obtaining a previous decoding output feature map generated by the decoder module when processing the previous decoding level;

[0140] S404, performing channel-dimensional splicing on the adjusted feature map with the adapted number of channels and the decoded output feature map of the previous layer to generate an input feature map of the target layer;

[0141] S405: Input the target level input feature map into the corresponding decoding level of the decoder module.

[0142] In this embodiment, when the hybrid attention feature maps of each layer are passed to the corresponding decoding layer of the decoder module, the hierarchical input feature map is generated through feature fusion. This process is designed to ensure that the hybrid attention feature maps can be effectively parsed layer by layer in the decoder, while improving the feature expression capability of the decoder through multi-scale feature fusion.

[0143] First, spatial resolution matching adjustment is performed on the hybrid attention feature map of the current level to generate a hybrid attention feature map with matching spatial size. Spatial resolution matching adjustment is an operation that adjusts the width and height of the input feature map to the target size to ensure that the spatial structure remains consistent during feature transfer. Common spatial resolution matching methods include bilinear interpolation, bicubic interpolation, and transposed convolution. Bilinear interpolation is an efficient interpolation method that is suitable for most feature map scaling operations; bicubic interpolation can better preserve details at high resolutions; transposed convolution is a method of upsampling by learning parameters, which can dynamically adjust the spatial resolution during training. The choice of different spatial matching methods depends on the resolution of the feature map level and the computing resources of the model. Bilinear interpolation can be used to improve speed at high levels (low resolution), while transposed convolution can be used at low levels (high resolution) to ensure detailed expression.

[0144] Next, a channel-number adjustment convolution operation is performed on the spatially sized hybrid attention feature map to generate an adjusted feature map with the same channel number. Channel-number adjustment convolution is an operation that converts the number of channels in a feature map to a specified number of channels, typically implemented using a 1×1 convolution kernel. A 1×1 convolution is a spatially insensitive convolution operation that only performs linear transformations in the channel dimension, enabling rapid adjustment of the number of channels in a feature map without changing the spatial resolution. The convolution kernel weights are automatically learned through training to ensure consistent feature representation across different layers. Channel number adjustment is an integral part of decoder feature map delivery, ensuring channel compatibility between the hybrid attention feature map and the decoder-level features, thus avoiding size conflicts during feature fusion.

[0145] Next, the decoder module obtains the previous decoded output feature map generated when processing the previous decoding layer. The decoder is a structure that restores image resolution layer by layer. The output feature map of each layer serves as the input of the next layer or is fused with the feature map of the jump connection. The previous decoded output feature map is a feature representation that the decoder has processed and generated in the previous layer. It usually retains higher-level semantic information and can provide a reference for decoding at the current layer. The method for obtaining the previous decoded output feature map is usually to extract it from the decoder's feature cache.

[0146] Next, the adjusted feature map with the adapted number of channels is spliced ​​with the previous layer decoded output feature map in the channel dimension to generate the target layer input feature map. Channel dimension splicing is an operation that connects multiple feature maps together in the channel dimension. For example, assuming that the size of the adjusted feature map is (H, W, C1), the size of the previous layer decoded output feature map is (H, W, C2), and the size of the target layer input feature map after channel splicing is (H, W, C1+C2). The advantage of channel splicing is that it retains the features from the encoder side (enhanced details) and the semantic information from the decoder side (enhanced overall semantics). This fusion method can improve the feature expression capability of the decoder and ensure that multi-scale information is parsed in the decoder at the same time.

[0147] Finally, the target-level input feature map is fed into the corresponding decoding layer of the decoder module. The decoder module is responsible for parsing and gradually restoring spatial resolution layer by layer. Feeding the fused target-level input feature map into the current decoder layer means that the feature map will participate in the subsequent decoding process. The decoder typically restores spatial resolution layer by layer through transposed convolution or upsampling operations, while gradually refining the feature representation within each layer through convolution and activation functions. The fusion of mixed attention feature maps at each layer, transmitted via skip connections, improves the decoder's ability to balance detail restoration and global semantic parsing.

[0148] This embodiment achieves seamless cross-level feature fusion by performing spatial resolution matching and channel number adjustment convolution operations in each layer of skip connections between the encoder and decoder. Spatial resolution matching ensures that the hybrid attention feature map at each layer is spatially consistent with the decoder feature map, preventing information loss; channel number adjustment convolution ensures channel compatibility between different feature maps, avoiding feature splicing conflicts. Feature fusion uses channel splicing to preserve detailed information on the encoder side and global semantics on the decoder side, enhancing the diversity of feature expression and the decoder's feature restoration capabilities.

[0149] In one embodiment, the above step S50 includes:

[0150] S501, performing a transposed convolution operation on the level input feature map of the current level to generate a primary up-sampled feature map;

[0151] S502, performing batch normalization processing on the primary up-sampled feature map to generate a normalized feature map;

[0152] S503, processing the normalized feature map through an activation function to generate an upsampled feature map of the current level;

[0153] S504, repeatedly performing the transposed convolution operation, batch normalization processing and activation function processing, generating upsampling feature maps layer by layer, until the output feature map of the final decoding layer is obtained, and the output feature map of the final decoding layer is used as the final output feature map.

[0154] In this embodiment, hierarchical upsampling is performed on the hierarchical input feature maps to generate output feature maps that match the spatial resolution of the processed image. This process ensures that the decoder preserves multi-level feature information while restoring spatial resolution layer by layer. Hierarchical upsampling is a bottom-up operation that restores the spatial resolution of feature maps layer by layer. By parsing the feature maps layer by layer, spatial details are gradually restored, making it particularly suitable for reconstructing high-resolution images and semantically parsing complex scenes.

[0155] First, a transposed convolution operation is performed on the layer input feature map of the current layer to generate a primary upsampled feature map. Transposed convolution is an upsampling operation commonly used in convolutional neural networks. Unlike traditional convolution (which reduces the size of the feature map), transposed convolution can expand the spatial resolution of the feature map. The working principle of transposed convolution is to achieve spatial expansion by inserting zero values ​​in the feature map and filling these zero values ​​with the convolution kernel. The convolution kernel size and stride of the transposed convolution determine the upsampling ratio. For example, a transposed convolution with a 3×3 convolution kernel and a stride of 2 can expand the feature map size to twice the original size. The advantage of choosing transposed convolution is that it can dynamically adjust the feature expression by learning the convolution kernel parameters during training. In the specific implementation, the convolution kernel of the transposed convolution can be initialized to a random number, and the parameters are updated through backpropagation training.

[0156] Next, batch normalization is performed on the primary upsampled feature maps to generate normalized feature maps. Batch normalization is a technique that improves the stability of neural network training and accelerates training convergence. Its basic principle is to calculate the mean and standard deviation of each channel of the feature map on each batch of data, and then normalize the feature map based on these statistics to ensure a stable feature distribution. Batch normalization not only avoids the problem of gradient vanishing or gradient exploding, but also improves the generalization performance of the network. During the decoder upsampling process, batch normalization can ensure that the numerical distribution of the upsampled feature map is stable at each level, thereby improving the decoder's ability to restore detailed information.

[0157] Then, the normalized feature map is processed by the activation function to generate the upsampled feature map of the current layer. The activation function is a function that introduces nonlinear expression. Common activation functions include ReLU (Rectified Linear Unit), LeakyReLU and Sigmoid. ReLU is a nonlinear activation function that outputs the original value when the input is greater than zero and outputs zero when it is less than zero. It can effectively avoid the gradient disappearance problem. Leaky ReLU is an improved version of ReLU. When the input is less than zero, the output is a negative slope (such as 0.01 times the input value), avoiding the "neuron death" problem. During the decoder upsampling process, the activation function ensures that the feature map has nonlinear expression capabilities at each level, enhancing the feature parsing effect.

[0158] After the above operations are completed, upsampled feature maps are generated layer by layer by repeatedly performing transposed convolution, batch normalization, and activation function processing. This layer-by-layer upsampling design ensures that the decoder retains multi-scale feature information while restoring spatial resolution layer by layer. The number of layers of layer-by-layer upsampling depends on the structure of the decoder. For example, in the case of a five-layer encoder, the decoder usually also contains five layers of layer-by-layer upsampling. The upsampled feature maps at each layer retain the high-level semantic information of the previous layer while introducing spatial detail expression at the current layer.

[0159] Ultimately, the upsampled feature map at the final decoding layer is determined as the final output feature map. This final output feature map not only has the same spatial resolution as the processed image, but also preserves the multi-scale feature information parsed and enhanced layer by layer from the encoder to the decoder, ensuring high-precision representation for image reconstruction or object recognition. The final output feature map can be directly applied to subsequent object detection, image segmentation, or other computer vision tasks.

[0160] This embodiment uses layer-by-layer upsampling to enable the decoder to preserve and integrate multi-scale feature information while restoring spatial resolution layer by layer. Transposed convolution ensures consistent spatial structure during upsampling, batch normalization stabilizes feature distribution, activation functions introduce nonlinear expressions, and layer-by-layer parsing improves feature restoration accuracy. The final output feature map has the same spatial resolution as the original image and incorporates feature information that is enhanced layer by layer from encoder to decoder, making it suitable for a variety of computer vision tasks.

[0161] In one embodiment, after the above step S50, the method further includes:

[0162] S601, inputting the output feature map into a task processing network, performing target detection processing on the output feature map, and generating an initial detection result;

[0163] S602, performing non-maximum suppression processing on the initial detection result to generate target detection box coordinates and category labels;

[0164] S603: Mark the target object in the image to be processed according to the target detection frame coordinates and the category label, and generate a final detection result image.

[0165] In this embodiment, after performing hierarchical upsampling processing on the hierarchical input feature map to generate an output feature map that matches the spatial resolution of the image to be processed, the output feature map can be further input into the task processing network to realize automated processing of downstream tasks. The task processing network is a neural network that can perform feature recognition, information extraction or object positioning according to predefined targets. The input of the task processing network is the output feature map, which means that the task processing network will extract effective information from the feature map that has been restored to high resolution. The task processing network can adopt a variety of structures, such as YOLO (You Only Look Once) network, Faster R-CNN (Region-Based Convolutional Neural Network), RetinaNet, etc. These networks can efficiently identify and locate target objects on high-resolution feature maps.

[0166] First, the output feature map is input into the task processing network for target detection processing, generating initial detection results. Object detection is a computer vision task whose core is to identify and locate specific objects from an input image or feature map. The task processing network traverses the spatial and channel information of the output feature map, extracting target features through operations such as convolution, pooling, and fully connected layers, and generates multiple candidate detection boxes and a confidence score for each detection box. Each detection box consists of rectangular coordinates (xmin, ymin, xmax, ymax) and a category label, and the confidence score indicates the reliability of the detection result. In specific implementations, the task processing network can adopt a multi-level feature extraction strategy, such as FPN (Feature Pyramid Network), to ensure effective detection of target objects at multiple scales.

[0167] Then, non-maximum suppression (NMS) is performed on the initial detection results to generate the final target detection box coordinates and category labels. Non-maximum suppression is a post-processing technique that aims to eliminate overlapping redundant detection boxes and retain the detection results with the highest confidence. The working principle of NMS is: all detection boxes are sorted by confidence score, the detection box with the highest score is retained, the remaining detection boxes are traversed and the intersection over union (IoU) with the retained boxes is calculated. When the IoU is greater than a preset threshold (such as 0.5), the duplicate boxes are deleted. This process ensures that only one detection box with the highest confidence is retained for each object in the final detection result, avoiding multiple redundant boxes from interfering with the detection accuracy.

[0168] Finally, the target object is annotated in the processed image based on the target detection box coordinates and category label, generating a final detection result image. This annotation process involves plotting the detection box's location information and category label on the original image, forming a visual output image with the detection results. In practice, the annotation operation can be implemented using image processing libraries such as OpenCV. The detection box is displayed on the image as a rectangular box or polygon, and the category label and confidence level are annotated around or within the box. The annotated detection result image can be used as a basis for subsequent business processing, such as extracting ID card information in the financial field or identifying lesion areas in the medical field.

[0169] In one embodiment, the task processing network uses the YOLOv5 network architecture. The output feature map is passed through the YOLOv5 multi-scale detection head for object detection, generating multiple candidate detection boxes, each with a confidence score and category label. Non-maximum suppression is performed using an Intersection of Union (IoU) threshold of 0.5, retaining only the detection boxes with the highest confidence. The annotation process uses OpenCV's rectangle and putText functions to draw the detection boxes and category labels on the processed image.

[0170] In another embodiment, the task processing network adopts the Faster R-CNN structure. The output feature map first generates candidate regions through the region proposal network (RPN), and then performs classification and regression processing on each candidate region to generate detection boxes and category labels. Non-maximum suppression processing adopts Soft-NMS (soft non-maximum suppression) to avoid sudden deletion of valid detection boxes by gradually reducing the confidence of overlapping detection boxes. The annotation process is drawn through PIL (Python Imaging Library), and the category labels and detection box coordinates are superimposed on the image to facilitate the subsequent export of detection results.

[0171] In another implementation, the task processing network uses the EfficientDet network structure, combined with FPN (Feature Pyramid Network) and BiFPN (Bidirectional Feature Pyramid Network) to improve multi-scale detection. Non-maximum suppression uses class-wise NMS, where each category is independently NMSed to avoid overlapping detection results from multiple categories. The annotation process is implemented using a Qt graphical interface, supporting real-time visualization and interaction of detection results.

[0172] Example: In the financial sector, such as a bank's smart identity verification scenario, users upload an ID card image via their mobile device, and the system automatically extracts and verifies their identity information. This method first receives the ID card image uploaded by the user and performs a series of image enhancements via the image preprocessing module. This includes converting the input image from RGB to HSV color space, performing contrast stretching to enhance the text and portrait areas in the ID card image, and then normalizing the enhanced image to map pixel values ​​to a range of 0-1. Finally, nonlinear filtering is used to remove noise artifacts such as light speckle and texture interference. The preprocessed image data is fed into the encoder module, where features are extracted layer by layer through a multi-layer convolutional neural network. Each convolutional layer extracts features and stabilizes the feature distribution through batch normalization and activation functions. Dimensionality reduction is achieved through strided convolution, allowing the feature map to extract high-level semantic information layer by layer in the depth direction, generating a multi-layer encoded feature map. Between the encoder and decoder, the hybrid attention module processes the multi-layer encoded feature map layer by layer, generating a hybrid attention feature map for each layer. For each feature map output by the encoder layer, the hybrid attention module first uses channel-wise attention to capture the importance between feature channels. It then divides the feature map into multiple windowed feature blocks and performs self-attention within each window to identify correlations between spatial locations. Channel-wise attention and window-wise self-attention are linearly weighted fused to generate an intermediate attention feature map. Finally, a residual connection is performed with the original encoded feature map to ensure that the attention-enhanced features retain the original information. The hybrid attention feature maps from each layer are passed to the decoder layer, where feature fusion is performed to generate the layer-wise input feature map. In each decoder layer, the spatial resolution of the hybrid attention feature map is first adjusted to match the decoder layer input. Then, channel-wise convolution is performed to ensure that the number of channels in the feature map is consistent with the decoder input. The adjusted feature map is concatenated with the output feature map of the previous decoder layer, channel by channel, to form the layer-wise input feature map. This feature map is then fed into the current decoder layer for further processing. In the decoder module, upsampling is performed layer by layer to restore the spatial resolution of the feature map. Each decoder layer generates an upsampled feature map through transposed convolution, and batch normalization and activation functions are applied to enhance feature representation. The upsampling process is performed layer by layer, and finally an output feature map that matches the resolution of the original ID card image is generated. The output feature map is input into the task processing network (such as the YOLOv5 model) for target detection processing to automatically identify information such as the name, ID card number, and expiration date on the ID card. The detection results are processed through non-maximum suppression to eliminate overlapping redundant detection frames and retain the unique detection frame for each text area. In the detection results, each text area is marked with a detection frame and a category label (such as "name" and "ID card number"). The system maps the detection frame coordinates to the original image and generates an ID card image with the recognition result. The recognition result can be directly used for customer identity verification or automatic form filling.

[0173] In the healthcare field, the system receives CT or MRI images as images to be processed. The image preprocessing module first performs a series of image enhancement operations. Image preprocessing involves converting the input medical image from grayscale to pseudocolor to enhance the visual contrast of the lesion area. The system then performs a contrast stretch on the pseudocolor image to make the edges of the lesion more visible. The enhanced image is normalized, and the pixel values ​​are mapped to a range of 0-1 to ensure a stable data distribution during feature extraction. Finally, the system uses nonlinear filtering (such as bilateral filtering) to remove noise and reduce artifacts, generating preprocessed image data. The preprocessed image data is then input to the encoder module, where a multi-layer convolutional neural network is used to extract multiple layers of encoded feature maps. Each convolutional layer extracts local features using a convolution kernel, stabilizes the feature distribution using a batch normalization layer, and enhances the nonlinear representation using an activation function. Strided convolution further compresses the spatial resolution, allowing each layer of the feature map to focus on increasingly higher-level semantic information. This ultimately generates a multi-layer encoded feature map, each layer preserving both local and global information. These layers of encoded feature maps are processed layer by layer using a hybrid attention module. The hybrid attention module first performs channel-wise attention, using global pooling and weighted computation to enhance focus on key features. The feature map is then divided into multiple window feature blocks. Within each window, a self-attention mechanism is used to identify inter-pixel correlations. The channel-wise and self-attention features are linearly weighted and fused to generate an intermediate attention feature map. This is then connected to the original feature map via a residual connection, preserving the input feature information to generate a hybrid attention feature map. The hybrid attention feature maps from each layer are passed to the decoder layer, where feature fusion generates the layer-wise input feature map. In each decoder layer, the hybrid attention feature map is first spatially resized to match the input resolution of the decoder layer. Channel-wise convolution ensures that the number of channels in the feature map matches the decoder input. The resized feature map is then channel-wise concatenated with the feature map output from the previous decoder layer to generate the layer-wise input feature map. In the decoder module, upsampling is performed layer by layer, and the spatial resolution of the feature map is restored layer by layer through transposed convolution. Batch normalization and activation functions are applied to each layer to enhance feature representation. Ultimately, an output feature map with the original medical image resolution is generated. The output feature map is fed into a task processing network (such as the Faster R-CNN model) for automatic lesion region detection. The task processing network automatically identifies the lesion region in the output feature map and generates multiple candidate detection boxes. Non-maximum suppression eliminates redundant detection boxes, retaining only the lesion detection boxes with the highest confidence. The detection results are mapped onto the original medical image, generating a medical image with lesion annotations, which is provided to doctors for diagnostic reference. Each detection box labels the lesion region and includes a confidence score, allowing doctors to quickly identify the lesion location and make a diagnosis.

[0174] This embodiment further performs target detection processing within the task processing network on the output feature map, efficiently extracting target objects from the restored high-resolution feature map. Non-maximum suppression ensures the uniqueness and accuracy of detection results, eliminating redundant detection boxes. Finally, by annotating the detection results on the processed image, a clear and intuitive detection result image is provided. Overall, high-precision target recognition and annotation can be achieved in a variety of complex scenarios.

[0175] In one embodiment, a data processing device based on a hybrid attention mechanism is provided, and the data processing device based on the hybrid attention mechanism corresponds one-to-one to the data processing method based on the hybrid attention mechanism in the above embodiment. Figure 3 , Figure 3 This is a functional module diagram of a preferred embodiment of a data processing device based on a hybrid attention mechanism of the present invention. It includes a preprocessing module 10, an encoder module 20, a hybrid attention module 30, a feature fusion module 40, and a decoder module 50. Each functional module is described in detail as follows:

[0176] The preprocessing module 10 is used to receive the image to be processed and perform preprocessing operations to generate preprocessed image data;

[0177] The encoder module 20 is configured to input the pre-processed image data into the encoder module to perform multi-layer convolution processing to generate a multi-layer encoding feature map;

[0178] A hybrid attention module 30 is configured to process the multi-layer encoding feature map layer by layer in each level jump connection between the encoder module and the decoder module through the hybrid attention module to generate a hybrid attention feature map of the corresponding level;

[0179] A feature fusion module 40 is used to transfer the mixed attention feature maps of each level to the corresponding decoding level of the decoder module, and generate a level input feature map through feature fusion;

[0180] The decoder module 50 is configured to perform hierarchical upsampling processing on the hierarchical input feature map to generate an output feature map that matches the spatial resolution of the image to be processed.

[0181] In one embodiment, the preprocessing module 10 is specifically configured to:

[0182] Performing color space conversion processing on the input image to be processed to generate intermediate image data;

[0183] performing a contrast stretching operation on the intermediate image data to generate enhanced image data;

[0184] Normalizing the pixel values ​​of the enhanced image data to a preset value range to generate normalized image data;

[0185] A noise elimination process is performed on the normalized image data through a nonlinear filtering module to generate pre-processed image data.

[0186] In one embodiment, the encoder module 20 is specifically configured to:

[0187] Inputting the preprocessed image data into the first convolutional layer of the encoder module to perform feature extraction operations to generate an initial convolutional feature map;

[0188] Processing the initial convolutional feature map through a batch normalization layer of an encoder module to generate a normalized feature map;

[0189] Applying a nonlinear activation function to the normalized feature map to generate an activated feature map;

[0190] Performing strided convolution dimensionality reduction processing on the activation feature map to generate a reduced dimensionality feature map, and using the reduced dimensionality feature map as the first-level encoding feature map;

[0191] The feature extraction operation, batch normalization processing, application of nonlinear activation function and strided convolution dimensionality reduction processing are repeatedly performed to generate the encoding feature map of each convolution layer layer by layer, and the encoding feature map of each convolution layer is used as a multi-layer encoding feature map.

[0192] In one embodiment, the hybrid attention module 30 is specifically configured to:

[0193] Perform channel attention processing on the encoded feature map of the current layer to generate a channel attention map;

[0194] Dividing the encoding feature map of the current level into a plurality of window feature blocks of predetermined sizes;

[0195] Perform self-attention association processing within each window feature block to generate a window self-attention map;

[0196] Performing linear weighted fusion on the channel attention map and the window self-attention map to generate an intermediate attention feature map;

[0197] Perform a residual connection between the intermediate attention feature map and the encoding feature map of the current level to generate a mixed attention feature map.

[0198] In one embodiment, the feature fusion module 40 is specifically configured to:

[0199] Perform spatial resolution matching adjustment on the hybrid attention feature map of the current level to generate a hybrid attention feature map with matching spatial size;

[0200] Performing a channel number adjustment convolution operation on the hybrid attention feature map with matching spatial size to generate an adjusted feature map with matching channel number;

[0201] Get the previous decoding output feature map generated by the decoder module when processing the previous decoding layer;

[0202] Perform channel-wise concatenation of the adjusted feature map with the adapted number of channels and the decoded output feature map of the previous layer to generate an input feature map of the target layer;

[0203] The target level input feature map is input into the corresponding decoding level of the decoder module.

[0204] In one embodiment, the decoder module 50 is specifically configured to:

[0205] Perform a transposed convolution operation on the level input feature map of the current level to generate a primary upsampled feature map;

[0206] Performing batch normalization on the primary upsampled feature map to generate a normalized feature map;

[0207] Processing the normalized feature map through an activation function to generate an upsampled feature map of the current level;

[0208] Repeat the transposed convolution operation, batch normalization processing and activation function processing to generate up-sampled feature maps layer by layer until the output feature map of the final decoding layer is obtained, and the output feature map of the final decoding layer is used as the final output feature map.

[0209] In one embodiment, the decoder module 50 is specifically configured to:

[0210] Inputting the output feature map into a task processing network, performing target detection processing on the output feature map, and generating an initial detection result;

[0211] Performing non-maximum suppression processing on the initial detection results to generate target detection frame coordinates and category labels;

[0212] According to the target detection frame coordinates and category labels, the target object is marked in the image to be processed to generate a final detection result image.

[0213] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, memory, network interface and database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external user terminal via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a data processing method based on a hybrid attention mechanism.

[0214] In one embodiment, a computer device is provided. The computer device may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, memory, network interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the user side of a data processing method based on a hybrid attention mechanism.

[0215] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0216] Receive the image to be processed and perform preprocessing operations to generate preprocessed image data;

[0217] Input the pre-processed image data into the encoder module to perform multi-layer convolution processing to generate a multi-layer encoding feature map;

[0218] In each level of skip connection between the encoder module and the decoder module, the multi-layer encoding feature map is processed layer by layer by the hybrid attention module to generate a hybrid attention feature map of the corresponding layer;

[0219] The mixed attention feature maps of each layer are passed to the corresponding decoding layer of the decoder module, and the layer input feature maps are generated through feature fusion;

[0220] Performing hierarchical upsampling processing on the hierarchical input feature map to generate an output feature map that matches the spatial resolution of the image to be processed.

[0221] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0222] Receive the image to be processed and perform preprocessing operations to generate preprocessed image data;

[0223] Input the pre-processed image data into the encoder module to perform multi-layer convolution processing to generate a multi-layer encoding feature map;

[0224] In each level of skip connection between the encoder module and the decoder module, the multi-layer encoding feature map is processed layer by layer by the hybrid attention module to generate a hybrid attention feature map of the corresponding layer;

[0225] The mixed attention feature maps of each layer are passed to the corresponding decoding layer of the decoder module, and the layer input feature maps are generated through feature fusion;

[0226] Performing hierarchical upsampling processing on the hierarchical input feature map to generate an output feature map that matches the spatial resolution of the image to be processed.

[0227] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the user side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0228] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0229] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0230] It should be noted that if any software tools or components other than those of the Company appear in the embodiments of this application, they are merely for illustration and do not represent actual use. The above embodiments are intended only to illustrate the technical solutions of the present invention, not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some of the technical features therein with equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A data processing method based on a hybrid attention mechanism, characterized in that: The following steps are involved: Receive the image to be processed and perform preprocessing operations to generate preprocessed image data; Input the pre-processed image data into the encoder module to perform multi-layer convolution processing to generate a multi-layer encoding feature map; In each level of skip connection between the encoder module and the decoder module, the multi-layer encoding feature map is processed layer by layer by the hybrid attention module to generate a hybrid attention feature map of the corresponding layer; The mixed attention feature maps of each layer are passed to the corresponding decoding layer of the decoder module, and the layer input feature maps are generated through feature fusion; Performing hierarchical upsampling processing on the hierarchical input feature map to generate an output feature map that matches the spatial resolution of the image to be processed.

2. The data processing method based on the hybrid attention mechanism according to claim 1, characterized in that: Receive the image to be processed and perform preprocessing operations to generate preprocessed image data, including: Performing color space conversion processing on the input image to be processed to generate intermediate image data; performing a contrast stretching operation on the intermediate image data to generate enhanced image data; Normalizing the pixel values ​​of the enhanced image data to a preset value range to generate normalized image data; A noise elimination process is performed on the normalized image data through a nonlinear filtering module to generate pre-processed image data.

3. The data processing method based on the hybrid attention mechanism according to claim 1, characterized in that: The pre-processed image data is input into the encoder module to perform multi-layer convolution processing to generate a multi-layer encoding feature map, including: Inputting the preprocessed image data into the first convolutional layer of the encoder module to perform feature extraction operations to generate an initial convolutional feature map; Processing the initial convolutional feature map through a batch normalization layer of an encoder module to generate a normalized feature map; Applying a nonlinear activation function to the normalized feature map to generate an activated feature map; Performing strided convolution dimensionality reduction processing on the activation feature map to generate a reduced dimensionality feature map, and using the reduced dimensionality feature map as the first-level encoding feature map; The feature extraction operation, batch normalization processing, application of nonlinear activation function and strided convolution dimensionality reduction processing are repeatedly performed to generate the encoding feature map of each convolution layer layer by layer, and the encoding feature map of each convolution layer is used as a multi-layer encoding feature map.

4. The data processing method based on the hybrid attention mechanism according to claim 1, characterized in that: In each level of skip connection between the encoder module and the decoder module, the multi-layer encoding feature map is processed layer by layer by the hybrid attention module to generate a hybrid attention feature map of the corresponding layer, including: Perform channel attention processing on the encoded feature map of the current layer to generate a channel attention map; Dividing the encoding feature map of the current level into a plurality of window feature blocks of predetermined sizes; Perform self-attention association processing within each window feature block to generate a window self-attention map; Performing linear weighted fusion on the channel attention map and the window self-attention map to generate an intermediate attention feature map; Perform a residual connection between the intermediate attention feature map and the encoding feature map of the current level to generate a mixed attention feature map.

5. The data processing method based on the hybrid attention mechanism according to claim 1, characterized in that: The mixed attention feature maps of each layer are passed to the corresponding decoding layer of the decoder module, and the layer input feature maps are generated through feature fusion, including: Perform spatial resolution matching adjustment on the hybrid attention feature map of the current level to generate a hybrid attention feature map with matching spatial size; Performing a channel number adjustment convolution operation on the hybrid attention feature map with matching spatial size to generate an adjusted feature map with matching channel number; Get the previous decoding output feature map generated by the decoder module when processing the previous decoding layer; Perform channel-wise concatenation of the adjusted feature map with the adapted number of channels and the decoded output feature map of the previous layer to generate an input feature map of the target layer; The target level input feature map is input into the corresponding decoding level of the decoder module.

6. The data processing method based on the hybrid attention mechanism according to claim 1, characterized in that: Performing hierarchical upsampling processing on the hierarchical input feature map to generate an output feature map that matches the spatial resolution of the image to be processed, comprising: Perform a transposed convolution operation on the level input feature map of the current level to generate a primary upsampled feature map; Performing batch normalization on the primary upsampled feature map to generate a normalized feature map; Processing the normalized feature map through an activation function to generate an upsampled feature map of the current level; Repeat the transposed convolution operation, batch normalization processing and activation function processing to generate up-sampled feature maps layer by layer until the output feature map of the final decoding layer is obtained, and the output feature map of the final decoding layer is used as the final output feature map.

7. The data processing method based on the hybrid attention mechanism according to claim 1, characterized in that: After performing hierarchical upsampling processing on the hierarchical input feature map to generate an output feature map that matches the spatial resolution of the image to be processed, the method further includes: Inputting the output feature map into a task processing network, performing target detection processing on the output feature map, and generating an initial detection result; Performing non-maximum suppression processing on the initial detection results to generate target detection frame coordinates and category labels; According to the target detection frame coordinates and category labels, the target object is marked in the image to be processed to generate a final detection result image.

8. A data processing device based on a hybrid attention mechanism, characterized in that: The data processing device based on the hybrid attention mechanism includes: A preprocessing module is used to receive the image to be processed and perform preprocessing operations to generate preprocessed image data; An encoder module, configured to input the pre-processed image data into an encoder module to perform multi-layer convolution processing to generate a multi-layer encoding feature map; A hybrid attention module is configured to process the multi-layer encoding feature map layer by layer in the skip connection between the encoder module and the decoder module to generate a hybrid attention feature map of the corresponding layer; The feature fusion module is used to transfer the mixed attention feature maps of each layer to the corresponding decoding layer of the decoder module, and generate the layer input feature map through feature fusion; A decoder module is used to perform hierarchical upsampling processing on the hierarchical input feature map to generate an output feature map that matches the spatial resolution of the image to be processed.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a data processing program based on a hybrid attention mechanism that is stored in the memory and can be run on the processor. When the data processing program based on the hybrid attention mechanism is executed by the processor, the steps of the data processing method based on the hybrid attention mechanism are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores a data processing program based on a hybrid attention mechanism, and when the data processing program based on the hybrid attention mechanism is executed by a processor, the steps of the data processing method based on the hybrid attention mechanism as described in any one of claims 1 to 7 are implemented.

Images