Image feature recognition method based on adaptive collaborative feature fusion and shape perception optimization

Abstract

The invention provides a self-adaptive collaborative feature fusion and shape perception optimization image feature recognition method, which comprises the following steps of: splitting an image feature into global guide sub-features and local guide sub-features through a double-branch design of a context embedded feature enhancement network, and splicing the global guide sub-features and the local guide sub-features after attention enhancement and lightweight convolution processing respectively, collaborative characterization of global context and local details is achieved, and the problem that global and local features of an existing model are unbalanced is solved; through a dynamic fusion mechanism of the adaptive feature focusing network, focus features are enhanced, semantic and detail interaction is realized, and multi-scale focus requirements are adapted; through the loss function design of the shape perception optimization strategy, the sensitivity of bounding box regression to the focus shape change is improved, and the problem of small focus positioning gradient disappearance is solved; the three modules are connected in sequence, a closed loop is formed from feature extraction, multi-scale fusion to positioning optimization, the tiny focus detection rate and positioning precision are remarkably improved, and the complex background false detection rate is reduced.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and in particular to an image feature recognition method that combines adaptive collaborative feature fusion and shape-aware optimization. Background Technology

[0002] Chest imaging techniques, including chest X-ray (CXR) and computed tomography (CT), currently rely on manual image interpretation. However, due to limitations such as individual experience and workload, missed diagnoses and misdiagnoses are common. With the development of deep learning technology, models such as convolutional neural networks (CNN) and vision transformers (ViT) have become core tools for automated chest image detection.

[0003] The existing technology still has the following shortcomings: (1) Imbalance between global and local information fusion leads to missed detection of small or low-contrast lesions: Existing CNN models capture global semantics by increasing the number of network layers, but shallow local details are gradually lost during feature propagation; while ViT models can model global dependencies, their self-attention mechanism has weak perception of local fine-grained features. Both lead to global information overshadowing local details, resulting in the missed detection of small lesions and low-contrast lesions due to their insignificant features.

[0004] (2) Staticization of multi-scale feature fusion leads to insufficient representation ability of small lesions: Traditional FPN uses fixed weights to fuse features at different levels, failing to consider the differences between small lesions requiring more superficial detailed features and large lesions requiring more deep semantic features. Furthermore, static weights cannot adapt to changes in lesion scale, resulting in the effective features of small lesions being diluted by deep semantics and insufficient representation ability.

[0005] (3) Bounding box regression is insensitive to shape, resulting in low localization accuracy for small lesions: Metrics such as IoU and GIoU only focus on the position and overlap of the bounding box, ignoring the diversity of lesion geometry. For small lesions, the overlap of their bounding boxes is more sensitive to positional shifts, which can easily cause the IoU value to drop sharply to 0, leading to gradient vanishing and ultimately low localization accuracy. Summary of the Invention

[0006] To address the problems existing in the background technology, this invention proposes an image feature recognition method that combines adaptive collaborative feature fusion and shape-aware optimization.

[0007] An image feature recognition method with adaptive collaborative feature fusion and shape-aware optimization includes the following steps: S100: Acquire the target image and preprocess the target image; S200. After performing feature transformation convolution on the preprocessed target image, it is split into a first sub-feature and a second sub-feature along the channel dimension. The first sub-feature is used as input to the attention enhancement branch, and the second sub-feature is used as input to the detail preservation branch. The first sub-feature is then subjected to intermediate feature extraction, and then linear transformation processing and attention weighting processing are performed in sequence. S300: Perform convolution processing on the initial first sub-feature and the first sub-feature processed in step S200 respectively, and then perform feature fusion. Perform linear transformation and multilayer perceptron transformation processing on the fused feature map. S400. After performing lightweight convolution processing on the second sub-feature, it is concatenated with the fused feature map output in step S300 to obtain a concatenated feature map. Multi-layer feature maps are then extracted and processed based on the concatenated feature map. S500: After performing channel attention and spatial attention enhancement processing on the initial feature map, the stitched feature map and the multi-layer feature map respectively, a progressive fusion strategy combining top-down and lateral connection is then used for processing. S600: The output of step S500 is processed by adaptive spatial fusion attention, and the processed feature map is residually fused with the global semantic support layer feature map in the multi-layer feature map and then output to the detection head. The detection head performs shape-aware optimization processing and then performs the final image feature recognition output.

[0008] Based on the above, the preprocessing in step S100 is as follows:

[0009] in, For input preprocessing, a 3×3 convolution kernel with a stride of 2 is used to achieve dimensionality compression; This is the initial feature map; For the target image after standardization, H and W represent the height and width of the input image, and 3 represents the number of RGB channels.

[0010] Based on the above, in step S200, the preprocessed target image is subjected to feature transformation convolution and then split into a first sub-feature and a second sub-feature along the channel dimension, as follows: (2) in, For feature transformation convolution operations, a 1×1 convolution kernel and stride are used. ; For channel splitting operations; This is the initial feature map; This is the transformed feature map. , Indicates the number of channels; first sub-feature For global guiding sub-features, The second sub-feature is a detail-oriented sub-feature. ; Intermediate feature extraction is performed on the first sub-feature as follows:

[0011]

[0012] in, This is an intermediate feature map, where the number of channels C, height H, and width W are all related to... Consistent; Using 3×3 convolution kernel, stride Norm(⋅) is the layer normalization operation function; The characteristic mean; The characteristic variance; To prevent tiny values ​​with a denominator of 0; γ and β are learnable scaling and offset parameters; The normalized feature map has the following parameters: number of channels, height H, width W, and... Consistent;

[0013] middle, , , It is an independent linear transformation layer; The query features are used to calculate similarity with K to locate key regions; The key feature represents the semantic attributes of the feature, and its similarity with Q determines the attention weight; Value characteristics; Attention-weighted processing is as follows:

[0014]

[0015] in, For the attention weight matrix, each element This represents the attention weight from position j to position i. This is the transpose operation for the key feature K; Key feature dimension; This is the scaling factor; attention-weighted feature map

[0016] Based on the above, in step S200, the initial first sub-feature and the first sub-feature after processing The convolution processing and feature fusion are performed separately as follows: (8) in, , All are 3×3 convolution operations; This is the refined attention feature map; Feature maps that preserve original details; For the fused feature map The fused feature maps are then subjected to linear transformation and multilayer perceptron processing as follows: (9) in, They are independent linear transformation functions; These are the operation functions for a multilayer perceptron. , These are feature maps after linear transformation and multilayer perceptron transformation, respectively. The final output feature map for the attention enhancement branch

[0017] Based on the above, in step S400, the second sub-feature undergoes lightweight convolution processing as follows:

[0018] in, For lightweight convolution operations, a 1×1 convolution kernel and stride S=1 are used; The final output feature map of the branch that preserves details

[0019] Based on the above, in step S400, the spliced ​​feature map obtained after concatenating the fused feature map output in step S300 is as follows:

[0020] in, To enhance the final output of the network, embed features into the context; ; The final output feature map for the attention enhancement branch; Based on the concatenated feature maps, multi-layer feature maps are extracted and processed, as follows:

[0021] (13)

[0022] in, For further extraction of intermediate layer transition features; For the context embedding feature enhancement network model, the calculation process is shown in formulas (2)-(11); It is a 1×1 convolution; for Intermediate features after downsampling and semantic enhancement; For max pooling, a 2×2 pooling kernel, step size s=2, and zero padding are used; For depthwise separable convolution; For pointwise convolution, a 1×1 convolution kernel, stride s=1, and zero padding are used; This is an adaptive channel feature fusion module; for Downsampling features are used with a 2×2 pooling kernel and a step size s=2; Feature map of global semantic support layer

[0023] Based on the above, in step S500, the feature maps at each level Channel attention and spatial attention enhancement processes are performed sequentially, as follows: (15) in, This is a feature map after being aligned in size and channels by upsampling or downsampling. ; For channel attention; These represent average pooling and max pooling in channel attention, respectively. It is a fully connected layer; It is the sigmoid function; This is the feature map after channel attention weighting; This is element-wise multiplication; Spatial attention; These represent the average pooling and max pooling of spatial attention on the channel, respectively; This is a convolution operation with a 7×7 kernel; For splicing operations; Feature maps enhanced by the attention module of the convolutional block

[0024] Based on the above, the progressive fusion strategy combining top-down and lateral connections is processed as follows: (16) in, For feature map The projection; This refers to convolution with a kernel size of 1×1 and batch normalization operation; For feature map The projection; For Conv+BN+ReLU operations; This is the i-th layer feature map output during the progressive focusing stage; This represents a refined semantic representation resulting from the fusion of top-down and horizontal approaches. The learnable scaling factor; intermediate feature map

[0025] Based on the above, in step S600, the output of step S500 is processed by adaptive spatial fusion attention as follows: (17) in, This is a feature map with dimensions aligned to the channels; To standardize resolution and unified channel ; Represented as each scale and its source Calculate the location-level rating map; for The scoring subnetwork; To Apply Softmax normalized weights; The processed feature map is then residually fused with the global semantic support layer feature map from the multi-layer feature map, as follows:

[0026]

[0027] in, For identity mapping, to further align channels; Features at different scales ; P1 represents the i-th scale output feature after ASF fusion; P5 represents the global semantic support layer feature map. is a learnable scalar; Fᵢ is the final output feature of AFFN.

[0028] Based on the above, in step S600, shape-aware optimization is performed by fusing spatial distribution similarity with geometric constraints. The shape-aware loss function is as follows:

[0029] in, The shape-aware loss function; These are shape-perceived weighting coefficients; For medical shape-weighted IoU, Shape weighting factor; To normalize the Wasserstein distance similarity, the value range is [0,1]. To predict the bounding box, This represents the true bounding box corresponding to the lesion features in the data.

[0030] This invention has outstanding substantive features and significant progress compared to the prior art. Specifically, this invention: (1) A Context-Embedded Feature Enhancement Network (CEFEN) was designed to achieve dynamic coordination between global contextual information and local detail features, solve the problem of global-local feature imbalance, and improve the ability to identify small lesions and low-contrast lesions. (2) An Adaptive Feature Focusing Network (AFFN) is proposed, which realizes the adaptive fusion of cross-layer features through dynamic weight allocation, adapts to the feature requirements of lesions of different scales, and enhances the feature representation ability of small lesions. (3) A Shape-Aware Optimization Strategy (SAOS) was constructed. The loss function was designed by combining the spatial distribution and geometric shape of the lesions. This improved the sensitivity of bounding box regression to shape changes, solved the problem of gradient vanishing in the localization of small lesions, and improved the localization accuracy. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the overall framework structure of the recognition model of this invention.

[0032] Figure 2 This is a schematic diagram of the architecture of the context embedding feature enhancement network of this invention.

[0033] Figure 3 This is a schematic diagram of the architecture of the adaptive feature focusing network of the present invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] An image feature recognition method with adaptive collaborative feature fusion and shape-aware optimization includes the following steps: S100, acquiring a target image and preprocessing the target image; S200, performing feature transformation convolution on the preprocessed target image, and then splitting it into a first sub-feature and a second sub-feature along the channel dimension. The first sub-feature is used as input to the attention enhancement branch, and the second sub-feature is used as input to the detail preservation branch; intermediate features are extracted from the first sub-feature, and then linear transformation processing and attention weighting processing are performed sequentially; S300, the initial first sub-feature and the first sub-feature processed in step S200 are convolved and then fused, and the fused feature map is subjected to linear transformation and multilayer perceptron transformation processing; S4 S00. After performing lightweight convolution processing on the second sub-feature, it is concatenated with the fused feature map output in step S300 to obtain a concatenated feature map. Multi-layer feature maps are then extracted and processed based on the concatenated feature map. S500. The initial feature map, concatenated feature map, and multi-layer feature map are sequentially enhanced with channel attention and spatial attention, and then processed using a progressive fusion strategy combining top-down and lateral connections. S600. The output of step S500 is processed through adaptive spatial fusion attention, and the processed feature map is residually fused with the global semantic support layer feature map in the multi-layer feature map and output to the detection head. The detection head performs shape-aware optimization processing and then performs the final image feature recognition output.

[0036] Because the size, shape, and location of lesions in chest images are highly variable, especially small lesions, they often go undetected and are poorly located. Therefore, this embodiment provides an image feature recognition method based on adaptive collaborative feature fusion and shape-aware optimization. The overall framework is as follows: Figure 1 As shown, firstly, by constructing a Context Embedding Feature Enhancement Network (CEFEN), global contextual information is captured while retaining key local details, improving the ability to identify small lesions in complex anatomical backgrounds. Secondly, an adaptive feature focusing network is proposed, which improves the feature extraction capability of lesions at different scales through adaptive spatial fusion operations by modifying the progressive feature pyramid network architecture. Finally, to address the issue of bounding box regression accuracy, a shape-aware optimization strategy loss function is proposed by combining normalized Wasserstein distance and shape weights, effectively improving the localization accuracy of small lesions. Specifically: (1) Context Embedding Feature Enhancement Network In chest image feature recognition, it is difficult to simultaneously capture global semantic information and retain key local details. Most existing detection methods rely on deep features to obtain global contextual information, but as network depth increases, shallow spatial details are easily lost during propagation, leading to insufficient accuracy in the identification and localization of small lesions. Especially in complex chest anatomy, the imbalanced fusion between local and global features limits the model's ability to perceive small and low-contrast lesion features, ultimately resulting in missed and false detections. To address this issue, this invention proposes a Context-Embedded Feature Enhancement Network (CEFEN), such as... Figure 2 As shown, the aim is to achieve efficient collaborative representation of global information and local details.

[0037] During the backbone feature extraction process, the original chest images, due to their high pixel dimensionality and abundance of redundant information such as background noise and non-lesion tissue texture, are difficult to directly support subsequent refined feature analysis. Therefore, the chest images are subjected to a uniform size operation to obtain the backbone feature. (H, W are the height and width of the input image, and 3 is the number of RGB channels). After processing by the basic convolutional layer, the initial feature map P1 is obtained, as shown in Equation 1:

[0038] in, For input preprocessing, a 3×3 convolution kernel with a stride of 2 is used to achieve dimensionality compression; This is the initial feature map.

[0039] To achieve parallel optimization of "global-local" features, CEFEN was designed based on the "path specialization" design concept, such as... Figure 2 As shown. For the initial feature map First, perform feature transformation convolution, then perform channel segmentation operation along the feature map. The channel dimension is split into two sub-features , A two-branch structure is constructed, consisting of an attention enhancement branch and a detail preservation branch. The implementation process is shown in Equation 2: (2) in, For feature transformation convolution operations, a 1×1 convolution kernel (to reduce computational cost) and a stride of [missing information] are used. ; This is a channel splitting operation, which is the inverse operation of Concat concatenation; This is the transformed feature map. , Indicates the number of channels; For global guiding sub-features ( The input is fed into the attention enhancement branch, which focuses on global anatomical semantic capture, such as the spatial distribution of large-scale structures like lung lobe features and heart features, to help narrow the lesion search range. For detail-oriented sub-features Input to the detail preservation branch, focusing on the preservation of details of local lesions, such as the edge texture of small nodules, grayscale differences of low-contrast lesions, etc.

[0040] To improve the semantic accuracy of subsequent attention calculations, the attention enhancement branch first processes the transformed feature map. Intermediate features are extracted, and the semantics of the middle layer are refined by the local perception capability of the 3×3 convolution kernel, as shown in formula (3). At the same time, Layer Normalization (LN) is introduced to solve the problem of gradient explosion / vanishing caused by the drastic fluctuations in the mean and variance of features during training. LN stabilizes the training by standardizing the feature distribution, as shown in formula (4).

[0041]

[0042] in, This is an intermediate feature map, where the number of channels C, height H, and width W are all related to... Consistent; Using 3×3 convolution kernel, stride Norm(⋅) is the layer normalization operation function; The characteristic mean; The characteristic variance; To prevent tiny values ​​with a denominator of 0; γ and β are learnable scaling and offset parameters; The normalized feature map has the following parameters: number of channels, height H, width W, and... It is consistent, the distribution is more stable, and it is suitable for subsequent linear transformations.

[0043] Subsequently, to achieve key information filtering, query, key, and value features are generated through three linear layers, and attention weights are calculated through "dot product operation + Softmax normalization", as shown in formula (5):

[0044] in, , Each layer is an independent linear transformation layer, and each layer includes "weight matrix multiplication + bias term superposition"; The query features are used to calculate similarity with K to locate key regions; The key feature represents the semantic attributes of the feature, and its similarity with Q determines the attention weight; The value feature is the object on which attention weights are applied, carrying the semantic information that needs to be strengthened.

[0045] By calculating the similarity (dot product) between Q and K, the weight of each location feature is obtained. Then, Softmax normalization is used to ensure that the sum of the weights is 1, so that subsequent weighting operations can focus on semantically important regions (such as regions in the lung lobe where lesions may exist) and weaken irrelevant regions (such as the thoracic cage and background). The calculation process is shown in formula (6):

[0046] in, For the attention weight matrix, each element This represents the attention weight from position j to position i. The transpose operation on the key feature K swaps the channel dimension and spatial dimension of K, ensuring that the dot product with Q is computable; where The key feature dimension is used to scale the dot product result to avoid numerical overflow. As a scaling factor, to alleviate when When the value is increased, the dot product becomes too large, leading to the vanishing softmax gradient problem.

[0047] Attention weights are applied to the value feature V, as shown in Equation (7), through the weight matrix. Multiplication with V strengthens the features of key regions with high weights and weakens the features of irrelevant regions with low weights, thereby focusing global contextual information.

[0048] in, The attention-weighted feature map semantically highlights information from key global regions.

[0049] Attention-weighted While it enhances global information, it may weaken it. The original local details (such as the edges of tiny lesions). Therefore, for Perform convolution thinning, and simultaneously directly... Convolution is performed to preserve the original details, and then the two are fused by element-wise addition to achieve a balance between "global focus + local preservation". The calculation process is shown in formula (8): (8) in, , Both are 3×3 convolution operations, used to refine attention features and preserve original details, respectively; This is the refined attention feature map; Feature maps that preserve original details; This is the fused feature map, which contains both global attention information and original local details.

[0050] To further enrich feature representation, fusion features are... Perform linear transformation and multilayer perceptron (MLP) transformation, and then combine the elements of the two transformations to enhance the nonlinear representation, thus preserving... The basic semantics are enhanced, and its ability to represent complex anatomical structures (such as overlapping lung lobes and lesions) is further improved. The calculation process is shown in formula (9): (9) in, These are independent linear transformation functions used to preserve... The basic semantics should be understood to avoid information distortion caused by excessive nonlinearity. The operation function of the multilayer perceptron is to introduce nonlinearity through the ReLU activation function to fit complex feature associations, such as the gray-level difference between the lesion and the surrounding tissue. , These are the feature maps after linear transformation and MLP transformation, respectively; The final output feature map of the attention enhancement branch combines basic semantics with complex nonlinear representations, providing accurate global contextual information.

[0051] While the attention enhancement branch preserves some details through fusion operations, it still primarily focuses on the global context. The detail preservation branch refines the details through convolutional operations. The local semantics, such as enhancing the edges of small nodules and the grayscale texture of low-contrast lesions, avoid excessive transformation that leads to loss of detail and ensure that sufficient local information can be supplemented during subsequent stitching. As shown in formula (10):

[0052] in, For lightweight convolution operations, a 1×1 convolution kernel is used (to reduce parameters and computational cost), with a stride S=1; The final output feature map of the branch retains details, while the core retains key local information such as the edges of tiny nodules and the texture of patch shadows.

[0053] Finally, the attention-enhancing branch is concatenated using the channel dimension. With detail-preserving branches The fusion, as shown in formula (11), achieves a collaborative representation of global semantics and local details.

[0054]

[0055] in, The final output feature map of the Context Embedding Feature Enhancement Network (CEFEN) contains both global anatomical semantics such as lung lobes to narrow the lesion search range and local details of small lesions to improve the detection rate. It can be directly input into the subsequent adaptive feature focusing network for scale feature fusion. This is a channel-dimensional concatenation operation that ensures the output features contain both global and local information.

[0056] Obtaining the intermediate transition characteristics ,exist Based on this, the semantic information of the Backbone deep modules is further integrated to achieve a smooth transition from local details to mid-level semantics. The calculation process is shown in formula (12):

[0057] in, For further extraction of intermediate layer transition features; For the context embedding feature enhancement network model, the calculation process is shown in formulas (2)-(11); It is a 1×1 convolution used to match the number of feature channels.

[0058] This approach reduces computational cost by employing depthwise separable convolution (DSConv) and integrates more abstract global semantics, focusing on enhancing cross-scale feature associations of larger lesions. Furthermore, it utilizes an adaptive channel-to-feature fusion module (A2c2f) with dynamic weight generation to adjust the channel fusion ratio based on the lesion probability in the feature region, resulting in a cross-scale semantic association layer. The calculation process is shown in formula (13): (13) in, for Intermediate features after downsampling and semantic enhancement; For max pooling, a 2×2 pooling kernel, stride s=2, and zero padding are used to reduce spatial resolution and preserve the most significant semantic information in the features. For depthwise separable convolution, spatial features are extracted channel by channel to enhance the detailed representation of lesion edges and local structures; For pointwise convolution operations, a 1×1 convolution kernel, stride s=1, and zero padding (padding=0) are used to fuse the single-channel features extracted by depthwise convolution and construct semantic associations between channels; Adaptive channel-to-feature fusion module.

[0059] This is the highest-level deep feature, carrying the most abstract global anatomical semantics, such as the overall distribution of lung lobes and the macroscopic anatomical location of lesions, providing high-level semantic constraints for the "global search-shape perception" of small lesions. The calculation process is shown in formula (14):

[0060] in, for The downsampling features are implemented using a 2×2 pooling kernel with a stride of s=2 to further compress the spatial dimension and highlight the global semantics. This is the feature map for the global semantic support layer.

[0061] (2) Adaptive Feature Focusing Network Because lesions in chest images vary significantly in size, shape, and contrast, traditional feature pyramid structures are prone to semantic misalignment and loss of detail when fusing features from different levels, leading to decreased performance in detecting small lesions and recognizing them in complex backgrounds. To address this, the Adaptive Feature Focusing Network (AFFN) builds upon the AFPN structure to construct an attention-guided multi-scale feature focusing framework, such as... Figure 2 As shown, the CBAM feature enhancement and adaptive spatial fusion (ASF) mechanism enables dynamic focusing and fine fusion of multi-scale features.

[0062] The input image is processed by a backbone network to extract features at different levels. To enhance the saliency and discriminability of features at each layer, AFFN embeds a Convolutional Block Attention Module (CBAM) at each layer for feature enhancement. CBAM effectively suppresses redundant background information and highlights significant regions related to lesions through cascaded operations of channel attention and spatial attention, enabling each feature map to have stronger discriminative power and providing high-quality input for subsequent multi-scale fusion. The calculation process is shown in formula (15): (15) in, This is a feature map after being aligned in size and channels by upsampling or downsampling. ; For channel attention; These represent average pooling and max pooling in channel attention, respectively. It is a fully connected layer; It is the sigmoid function; This is the feature map after channel attention weighting; This is element-wise multiplication; Spatial attention; These represent the average pooling and max pooling of spatial attention on the channel, respectively; This is a convolution operation with a 7×7 kernel; For splicing operations; This is the feature map output after CBAM enhancement.

[0063] Due to the unidirectional top-down information flow of traditional FPN, deep semantics gradually decay during propagation at lower levels. Therefore, a progressive fusion strategy combining top-down and lateral connections is adopted. High-level semantic features propagate to lower levels through downsampling paths, providing semantic guidance for local regions; simultaneously, low-level features supplement the spatial details of high levels through lateral connections, achieving bidirectional interaction between semantics and space. The top-down semantic guidance calculation process is shown in formula (16): (16) in, For feature map The projection; The convolution kernel size is 1×1 + BN (Batch Normalization) operation, aligns channels; For feature map The projection; For Conv+BN+ReLU operations; This is the i-th layer feature map output during the progressive focusing stage; This represents a refined semantic representation resulting from the fusion of top-down and horizontal approaches. The learnable scaling factor; This is an intermediate feature map.

[0064] To achieve spatially adaptive dynamic fusion between features at different levels and improve the model's ability to detect lesions at multiple scales, especially small lesions, Adaptive Spatial Fusion (ASF) is introduced. During the multi-level feature fusion process, weights are adaptively assigned to each spatial location, enabling the model to automatically adjust the contribution of features at different levels based on lesion features.

[0065] First, output the scale for each target. (Corresponding to ASF-i) Performs lightweight transformation to align channels, adaptively from all source scales. Gathering information and spatial location The learnable weights are calculated to balance the residual contributions. The calculation process is shown in formula (17): (17) in, This is a feature map with dimensions aligned to the channels; To standardize resolution and unified channel ; Represented as each scale and its source Calculate the location-level rating map; for The scoring subnetwork; To Perform Softmax normalized weights.

[0066] By using the obtained positional weights to perform a linear combination of the aligned source vectors, the output features of the target layer at different positions can be obtained. The vector representation of is shown in formula (18):

[0067] in, For identity mapping, to further align channels; Features at different scales ; This represents the i-th scale output feature after ASF fusion.

[0068] Finally, global semantic compensation of P5 is introduced to... and Residual fusion is performed, and the output retains the global context semantics, as shown in formula (19):

[0069] in, Fᵢ is a learnable scalar; Fᵢ is the final output feature of AFFN (F1, F2, F3, F4). Due to the fusion of global semantics of P5, it can significantly improve the scale adaptability and localization accuracy of small lesions, and can be directly input into subsequent detection heads for lesion classification and bounding box regression.

[0070] (3) Shape-aware optimization strategy loss function Traditional bounding box regression loss functions (such as IoU, GIoU, DIoU, or CIoU) primarily measure the similarity between the target and the predicted bounding box based on the overlapping region of the boxes. However, when the lesion target is small, irregular in shape, or slightly offset in position, these metrics based on the intersection-union ratio (IoU) of regions can lead to unstable gradient problems, resulting in insufficient sensitivity of the model to the localization of small-scale lesions. To address this, this invention proposes a Shape-Aware Optimization Strategy (SAOS), which combines Normalized Wasserstein Distance (NWD) and Medical Shape-weighted IoU (MSIoU) for joint modeling, enhancing the model's ability to perceive geometric changes while maintaining scale invariance.

[0071] First, we calculate the normalized Wasserstein similarity for spatial distribution modeling, assuming the predicted bounding box and the ground truth bounding box are respectively: (20) in, To predict the bounding box, Let (x, y) be the true bounding box corresponding to the lesion in the data, and (x, y) represent the center coordinates of the bounding box. They are width and height, respectively.

[0072] To improve the continuity of spatial alignment, the predicted bounding box and the ground truth bounding box are treated as a two-dimensional Gaussian distribution, as shown in Equation (21): (twenty one) in The mean is a Gaussian distribution. The covariance matrix represents the spatial extent of the bounding box; It represents a Gaussian distribution and is used for distribution similarity calculation.

[0073] Therefore, the calculation process of the second-order Wasserstein distance between the two bounding box distributions is shown in formula (22):

[0074] in, This represents the second-order Wasserstein distance, used to measure the overall difference in position and shape between two Gaussian distributions; Using Euclidean norm, calculate the squared distance between two vectors to measure the difference in the center positions of the bounding boxes; It is the Frobenius norm, used for matrix difference calculation, measuring the difference in the shape (width and height) of the bounding box.

[0075] To accommodate targets of different scales, a normalized Wasserstein similarity is defined, as shown in formula (23):

[0076] in, is the scale normalization coefficient, used to control the similarity decay rate, in order to alleviate the gradient vanishing problem of IoU class loss in small target detection; To normalize the Wasserstein similarity, which measures the consistency between the predicted bounding box and the ground truth bounding box in spatial distribution, the value range is [0,1].

[0077] Lesions in medical images exhibit significant shape variations (e.g., round, sheet-like, cord-like), and their boundaries are often interfered with by anatomical structures. To enhance the model's perception of morphological differences, this invention introduces a shape adaptation weight term on top of the IoU (Interval in Units) model. Taking into account the consistency of aspect ratio, area ratio, and boundary structure, the calculation process is shown in formula (24): (twenty four) in, The aspect ratio difference reflects the consistency between the predicted bounding box and the ground truth bounding box. The area ratio difference measures the relative deviation between the predicted bounding box and the actual bounding box area. The boundary gradient difference characterizes the difference in boundary structure between the predicted lesion and the actual lesion; Images of the lesion region are used to calculate boundary gradient differences. ; The height and width of the image are used for gradient normalization; It is a shape weighting factor that takes into account the dynamic weights of aspect ratio differences, area ratios, and boundary differences. The shape weighting coefficient controls different geometric parameters. The weighting ratio in.

[0078] When the predicted bounding box is similar in shape to the ground truth bounding box, It approaches 1, and conversely, it decays exponentially. The medical shape-weighted IoU is calculated as shown in formula (25):

[0079] in, Intersection over Union (IoU) is the ratio of the overlap area between the predicted bounding box and the ground truth bounding box to the area of ​​their union. For medical shape-weighted IoU, shape weights are introduced on top of IoU. In addition to differences in overall morphology, MSIoU, unlike CIoU or EIoU, can still provide effective gradient constraints in lesions with significant shape variations or irregular boundaries.

[0080] By fusing spatial distribution similarity with geometric shape constraints, the final shape-aware loss function (SAOSLoss) is defined:

[0081] in, The shape-aware loss function; The SAOS weighting coefficient controls the relative importance of MSIoU and NWD in the final loss.

[0082] In this embodiment, precision, recall, mean precision (mAP@0.5 vs. mAP@0.5:0.95), and detection speed (FPS) are used as evaluation metrics. Among these, mAP is a core evaluation metric in object detection, and its calculation relies on the intersection-union ratio (IOU). IOU quantifies the degree of spatial overlap between the predicted and ground truth bounding boxes by calculating the ratio of the overlap area to the union area, as shown below:

[0083] Where A represents the predicted bounding box position and B represents the labeled bounding box position, the IOU threshold is set to 0.5 in this experiment.

[0084] Positive samples represent images containing lesion areas, while negative samples represent images containing no task-relevant lesion areas. Precision refers to the proportion of instances that the model classifies as positive, but which are actually positive. Recall refers to the proportion of all true positive samples that are correctly identified by the model. Their definitions are as follows:

[0085]

[0086] Among them, true positive (TP) represents the number of real lesions correctly detected; false positive (FP) represents the number of negative samples or background areas misclassified as lesions; and false negative (FN) represents the number of real lesions that were not detected.

[0087] mAP is used to measure the detection performance of a model at different levels by calculating the area under the precision-recall curve at multiple IOU thresholds. mAP@0.5 is the mean precision calculated at an IOU threshold of 0.5, suitable for benchmark evaluation of general object detection tasks. mAP@0.5:0.95 is the average of the mean precision (AP) calculated at multiple IOU thresholds from 0.5 to 0.95 (step size 0.05). It is suitable for tasks requiring high localization accuracy. The calculation process is shown below:

[0088]

[0089] in A function representing the change in precision as a function of recall; The value represents the average precision of the i-th category; N is the total number of categories, which is 14 in this invention.

[0090] The performance results of the model in this embodiment are compared with existing models, and are shown in Table 1 below: Model mAP@50 mAP@[50-95] Number of parameters (M) RefineDet 0.128 0.075 33.51 RT-DERT 0.266 0.149 32.01 Yolov5-s 0.363 0.162 7.2 Yolov9-c 0.383 0.188 60.8 This invention 0.399 0.196 25.0 Table 1 As shown in Table 1, the adaptive fusion mechanism in this embodiment effectively integrates features at different levels, improves multi-scale robustness, and increases the mAP index by more than 7%.

[0091] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. An image feature recognition method with adaptive collaborative feature fusion and shape-aware optimization, characterized in that, Including the following steps: S100: Acquire the target image and preprocess the target image; S200. After performing feature transformation convolution on the preprocessed target image, it is split into a first sub-feature and a second sub-feature along the channel dimension. The first sub-feature is used as input to the attention enhancement branch, and the second sub-feature is used as input to the detail preservation branch. The first sub-feature is subjected to intermediate feature extraction, and then linear transformation processing and attention weighting processing are performed in sequence. S300: Perform convolution processing on the initial first sub-feature and the first sub-feature processed in step S200 respectively, and then perform feature fusion. Perform linear transformation and multilayer perceptron transformation processing on the fused feature map. S400. After performing lightweight convolution processing on the second sub-feature, it is concatenated with the fused feature map output in step S300 to obtain a concatenated feature map. Multi-layer feature maps are then extracted and processed based on the concatenated feature map. S500: After performing channel attention and spatial attention enhancement processing on the initial feature map, the stitched feature map and the multi-layer feature map respectively, a progressive fusion strategy combining top-down and lateral connection is then used for processing. S600: The output of step S500 is processed by adaptive spatial fusion attention, and the processed feature map is residually fused with the global semantic support layer feature map in the multi-layer feature map and then output to the detection head. The detection head performs shape-aware optimization processing and then performs the final image feature recognition output.

2. The image feature recognition method based on adaptive collaborative feature fusion and shape-aware optimization according to claim 1, characterized in that, The preprocessing in step S100 is as follows: ； in, For input preprocessing, a 3×3 convolution kernel with a stride of 2 is used to achieve dimensionality compression; This is the initial feature map; For the target image after standardization, H and W represent the height and width of the input image, respectively, and 3 represents the number of RGB channels.

3. The image feature recognition method based on adaptive collaborative feature fusion and shape-aware optimization according to claim 1, characterized in that, In step S200, the preprocessed target image is subjected to feature transformation convolution and then split into a first sub-feature and a second sub-feature along the channel dimension, as follows: （2）； in, For feature transformation convolution operations, a 1×1 convolution kernel and stride are used. ; For channel splitting operations; This is the initial feature map; This is the transformed feature map. , Indicates the number of channels; first sub-feature For global guiding sub-features, Second sub-feature For detail-oriented sub-features, ; Intermediate feature extraction is performed on the first sub-feature as follows: ； ； in, This is an intermediate feature map, where the number of channels C, height H, and width W are all related to... Consistent; Using 3×3 convolution kernels, stride Norm(⋅) is the layer normalization operation function; The characteristic mean; The characteristic variance; To prevent tiny values ​​with a denominator of 0; γ and β are learnable scaling and offset parameters; The normalized feature map has the following parameters: number of channels, height H, width W, and... Consistent; Linear transformations are handled as follows: ； in, , It is an independent linear transformation layer; The query features are used to calculate similarity with K to locate key regions; The key feature represents the semantic attributes of the feature, and its similarity with Q determines the attention weight; Value characteristics; Attention-weighted processing is as follows: ； ； in, For the attention weight matrix, each element This represents the attention weight from position j to position i. This is the transpose operation for the key feature K; Key feature dimension; This is the scaling factor; This is the attention-weighted feature map.

4. The image feature recognition method with adaptive collaborative feature fusion and shape-aware optimization according to claim 3, characterized in that, In step S200, the initial first sub-feature and the first sub-feature after processing Convolution processing and feature fusion are performed separately as follows: （8）； in, , All are 3×3 convolution operations; This is the refined attention feature map; Feature maps that preserve original details; For the fused feature map The fused feature maps are then subjected to linear transformation and multilayer perceptron processing as follows: （9）； in, They are independent linear transformation functions; These are the operation functions for a multilayer perceptron. , These are feature maps after linear transformation and multilayer perceptron transformation, respectively. The final output feature map for the attention enhancement branch.

5. The image feature recognition method based on adaptive collaborative feature fusion and shape-aware optimization according to claim 1, characterized in that: In step S400, the second sub-feature undergoes lightweight convolution processing as follows: ； in, For lightweight convolution operations, a 1×1 convolution kernel and stride S=1 are used; The final output feature map of the branch retains details.

6. The image feature recognition method based on adaptive collaborative feature fusion and shape-aware optimization according to claim 5, characterized in that: In step S400, the fused feature map output in step S300 is concatenated to obtain the following concatenated feature map: ； in, To enhance the final output of the network, embed features into the context; ; The final output feature map for the attention enhancement branch; Based on the concatenated feature maps, multi-layer feature maps are extracted and processed, as follows: ； （13）； ； in, For further extraction of intermediate layer transition features; For the context embedding feature enhancement network model, the calculation process is shown in formulas (2)-(11); It is a 1×1 convolution; for Intermediate features after downsampling and semantic enhancement; For max pooling, a 2×2 pooling kernel, step size s=2, and zero padding are used; For depthwise separable convolution; For pointwise convolution, a 1×1 convolution kernel, stride s=1, and zero padding are used; This is an adaptive channel feature fusion module; for Downsampling features are used with a 2×2 pooling kernel and a step size s=2; This is the feature map for the global semantic support layer.

7. The image feature recognition method with adaptive collaborative feature fusion and shape-aware optimization according to claim 1, characterized in that: In step S500, feature maps at each level Channel attention and spatial attention enhancement processes are performed sequentially, as follows: （15）； in, This is a feature map after being aligned in size and channels by upsampling or downsampling. ; For channel attention; These represent average pooling and max pooling in channel attention, respectively. It is a fully connected layer; It is the sigmoid function; This is the feature map after channel attention weighting; This is element-wise multiplication; Spatial attention; These represent the average pooling and max pooling of spatial attention on the channel, respectively; This is a convolution operation with a 7×7 kernel; For splicing operations; This is the feature map output after being enhanced by the attention module of the convolutional block.

8. The image feature recognition method based on adaptive collaborative feature fusion and shape-aware optimization according to claim 7, characterized in that: The progressive fusion strategy, combining top-down and lateral connectivity, is processed as follows: （16）； in, For feature map The projection; This refers to convolution with a kernel size of 1×1 and batch normalization operation; For feature map The projection; For Conv+BN+ReLU operations; This is the i-th layer feature map output during the progressive focusing stage; This represents a refined semantic representation resulting from the fusion of top-down and horizontal approaches. The learnable scaling factor; This is an intermediate feature map.

9. The image feature recognition method based on adaptive collaborative feature fusion and shape-aware optimization according to claim 8, characterized in that: In step S600, the output of step S500 is processed by adaptive spatial fusion attention as follows: （17）； in, This is a feature map with dimensions aligned to the channels; To standardize resolution and unified channel ; Represented as each scale and its source Calculate the location-level rating map; for The scoring subnetwork; To Apply Softmax normalized weights; The processed feature map is then residually fused with the global semantic support layer feature map from the multi-layer feature map, as follows: ； ； in, For identity mapping, to further align channels; Features at different scales ; P1 represents the i-th scale output feature after ASF fusion; P5 represents the global semantic support layer feature map. is a learnable scalar; Fᵢ is the final output feature of AFFN.

10. The image feature recognition method based on adaptive collaborative feature fusion and shape-aware optimization according to claim 1, characterized in that, In step S600, shape-aware optimization is performed by fusing spatial distribution similarity with geometric constraints. The shape-aware loss function is as follows: ； in, The shape-aware loss function; These are shape-perceived weighting coefficients; For medical shape-weighted IoU, Shape weighting factor; To normalize the Wasserstein distance similarity, the value range is [0,1]. To predict the bounding box, This represents the true bounding box corresponding to the lesion features in the data.

Images