Spectrum-aware multi-scale color attention module and cervical image classification system

Abstract

The application discloses a spectrum perception multi-scale color attention module and a cervical image classification system, comprising a plurality of parallel feature extraction sub-modules, temperature prediction sub-modules, attention calculation sub-modules, local geometric compensation sub-modules and feature residual fusion sub-modules, wherein the feature extraction sub-modules extract a plurality of different types of features based on input images, the temperature prediction sub-modules generate attention temperature parameters based on global features of the input images, which are used to control the concentration degree of channel attention distribution, the attention calculation sub-modules calculate collaborative attention based on the attention temperature parameters and the different types of features, the local geometric compensation sub-modules comprise a depth convolution layer and a point-by-point convolution layer to retain the edges of local color regions, small white areas and local textures, and the feature residual fusion sub-modules fuse the channel collaborative attention results and local geometric details in a residual manner. The module helps to screen out invalid or insufficient phenotype images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a spectral sensing multi-scale color attention module and a classification system for cervical images. Background Technology

[0002] In the effective image screening and color phenotype classification of cervical ...

[0003] Such schemes typically characterize color phenotypes based on single color features or fixed-weight channel attention, failing to establish synergistic relationships between different color response channels. Therefore, they struggle to jointly analyze the relative color changes between acetowhite regions and surrounding red tissue, as well as between brownish responses and uncolored areas in iodine-stained images, resulting in a relatively simplistic characterization of color phenotypes. Furthermore, existing channel attention mechanisms often employ fixed attention intensity or distribution, failing to adaptively adjust the concentration of attention distribution based on the global features of the input image. This limits their adaptability, making it difficult to simultaneously address scenarios with significant color differences and continuous color changes. Methods based on global channel attention, when modeling color synergy along the channel dimension, tend to weaken spatial details, leading to the loss of spatial information such as the edges of local color regions, small acetowhite regions, and local textures, and excessive spatial diffusion of color responses. This is particularly problematic when adjacent phenotype categories have similar colors and macroscopic morphologies, making it difficult to further differentiate lesions using differences in lesion boundaries and local textures. In addition, existing schemes often rely on single-scale features for discrimination, or the fusion between scales is insufficient, making it difficult to simultaneously retain information on shallow color boundaries and subtle color changes, as well as information on deep overall acetic acid whitening reaction and tissue distribution. This limits the effectiveness of cervical acetic acid maps and the ability to discriminate phenotypic categories. Summary of the Invention

[0004] To address some or all of the problems in existing technologies, and in order to filter out invalid or insufficiently phenotypic images, reduce the computational load of subsequent image segmentation and recognition operations, and improve recognition efficiency, the first aspect of this invention provides a Spectral Perception Multi-Scale Color Attention Module (SpecMCA), comprising: Several parallel feature extraction submodules are configured to extract several different types of features based on the input image; A temperature prediction submodule is used to generate attention temperature parameters based on global features of the input image, wherein the attention temperature parameters are used to control the concentration of channel attention distribution. The attention calculation submodule calculates collaborative attention based on the attention temperature parameter and the different types of features to establish the correlation between different color responses in the channel dimension.

[0005] Furthermore, the feature extraction submodule includes: The query feature (Q) extraction submodule includes a first convolutional layer, wherein the first convolutional layer is a 3x3 convolutional kernel with a stride of 2, and the query feature extraction submodule is used to generate query features; The key feature (K) extraction submodule includes a second convolutional layer, wherein the second convolutional layer has a 3x3 convolutional kernel and a stride of 2, and the key feature extraction submodule is used to generate key features; The value feature (V) extraction submodule includes a third convolutional layer, wherein the third convolutional layer is a 1x1 convolutional kernel, and the value feature extraction submodule is used to generate value features; The intermediate color association feature (A) extraction submodule includes a first convolutional layer and a third convolutional layer connected in series, and the intermediate color association feature extraction submodule is used to generate intermediate color association features.

[0006] Furthermore, the temperature prediction submodule includes: Pooling layer, which is used to obtain global features of the input image through pooling; A convolutional layer (Conv), connected to the pooling layer, is used to perform convolutional transformations on the global features; A nonlinear activation layer, connected to the convolutional layer, is used to generate the attention temperature parameter based on the output after the convolutional transformation.

[0007] Furthermore, the spectral sensing multi-scale color attention module also includes: The local geometry compensation submodule includes a depth convolutional layer and a pointwise convolutional layer, and is used to preserve the edges of local color regions, small acetic white regions, and local textures. The feature residual fusion submodule, which is connected to the attention calculation submodule and the local geometry compensation submodule, is used to fuse the channel collaborative attention results with local geometric details in a residual manner.

[0008] Furthermore, the spectral sensing multi-scale color attention module also includes: The projection output submodule is used to perform channel projection on the fused features.

[0009] Based on the spectral-sensing multi-scale color attention module described above, a second aspect of the present invention provides a classification system for cervical images, comprising: The feature extraction module includes a serially connected original feature extraction module and several spectral sensing multi-scale color attention modules as described above, wherein the original feature extraction module and the spectral sensing multi-scale color attention modules, as well as the individual spectral sensing multi-scale color attention modules, are connected through a reshaping and fusion module. The first cross-layer fusion module, which is connected to each remodeling fusion module of the feature extraction module, is used for upsampling and cross-layer stitching from deep to shallow, so as to transfer and reconstruct high-resolution features from deep to shallow layers. The second cross-layer fusion module is connected to the output of each layer of the first cross-layer fusion module. It is used for feature transfer from shallow to deep and cross-layer splicing to supplement the shallow boundary information to the deep layer. A multi-scale classification head, connected to the outputs of each layer of the first cross-layer fusion module and / or the second cross-layer fusion module, is used to output classification results based on the fusion features of different resolution levels.

[0010] Furthermore, the original feature extraction module includes two basic convolutional layers.

[0011] Furthermore, the feature extraction module also includes: The Spatial Pyramid Fast Pooling (SPPF) submodule is connected to the last layer reshaping and fusion module of the feature extraction module, and is used to perform multi-scale pooling and stitching on deep features. The Channel-Spatial Joint Attention Aggregation Submodule (C2PSA) is connected to the Spatial Pyramid Fast Pooling Submodule and is used to perform joint attention weighting on deep features in both the channel and spatial dimensions.

[0012] Furthermore, the first cross-layer fusion module includes: The upsampling submodule is used to restore the spatial resolution of the fused features from deeper layers, align them with the corresponding shallow features in size, and then send them to the feature stitching submodule. The feature concatenation submodule is used to concatenate the upsampled features with the features from the previous layer of the feature extraction module in the channel dimension. The reshaping and fusion submodule is used to reshape and fuse the stitched features, and output high-resolution fused features at the corresponding scale.

[0013] Furthermore, the second cross-layer fusion module includes: The convolutional submodule is used to perform convolutional transformations and downsampling on the features; The feature splicing submodule is used to splice downsampled features with features from the previous layer of the first cross-layer fusion module in the channel dimension. The reshaping and fusion submodule is used to reshape and fuse the spliced ​​features and output the fused features at the corresponding scale.

[0014] This invention provides a spectral-sensing multi-scale color attention module and a classification system for cervical images. It expands single-color feature extraction to channel-related modeling, enabling joint analysis of relative color changes between acetic acid-white regions, surrounding red tissue, and iodine-stained brown regions. Simultaneously, by controlling the concentration of attention distribution through dynamic temperature parameters, different images no longer employ fixed color attention intensities, improving adaptability to scenarios with significant color differences and continuous color changes. Furthermore, a local geometric compensation branch is added outside of channel attention to supplement edge, small lesion, and local texture information, reducing the spatial detail weakening caused by global channel modeling. The overall classification system employs multi-scale feature transfer and cross-layer fusion, preserving both shallow color boundary information and deep overall phenotypic information. This improves the ability to discriminate the validity and phenotypic categories of cervical acetic acid images, helping to filter out invalid or insufficiently phenotypic images, thereby reducing the number of low-value images, reducing the computational load for subsequent image segmentation and target recognition, and improving overall efficiency and accuracy. Attached Figure Description

[0015] To further illustrate the above and other advantages and features of the various embodiments of the present invention, a more specific description of the various embodiments of the present invention will be presented with reference to the accompanying drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are therefore not intended to limit its scope. In the drawings, identical or corresponding parts will be indicated by identical or similar reference numerals for clarity.

[0016] Figure 1 This diagram illustrates the structure of a spectral sensing multi-scale color attention module according to an embodiment of the present invention. Figure 2 This diagram illustrates the structure of a cervical image classification system according to an embodiment of the present invention. Detailed Implementation

[0017] In the following description, the invention is described with reference to various embodiments. However, those skilled in the art will recognize that the embodiments may be practiced without one or more specific details or in conjunction with other alternatives and / or additional methods or components. In other instances, well-known structures or operations are not shown or described in detail so as not to obscure the inventive points of the invention. Similarly, for illustrative purposes, specific numbers and configurations are set forth to provide a comprehensive understanding of embodiments of the invention. However, the invention is not limited to these specific details.

[0018] In this specification, references to "an embodiment" or "this embodiment" mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. The phrase "in one embodiment" appearing throughout this specification does not necessarily refer to the same embodiment in all instances.

[0019] It should be noted that this invention does not involve methods for diagnosing and treating diseases, but only provides medically relevant information. The methods and systems are applied to the processing of cervical images, and are not related to the diagnosis and treatment of diseases. Rather, the corresponding diagnoses and treatments should be provided to users by hospitals / doctors.

[0020] To filter out invalid or insufficiently phenotypic images, thereby reducing the computational load of subsequent detection, this invention discloses a cervical image classification system. In the backbone feature extraction stage, a spectral-aware multi-scale color attention module (SpecMCA) is incorporated. This module generates query features, key features, value features, and color association features through multiple sets of convolutional projections. It models the collaborative relationships between different color responses at the channel dimension and dynamically generates attention temperature parameters based on the global features of the input image to adaptively adjust the intensity of color channel selection. Simultaneously, a local geometric compensation branch is introduced to preserve the edges of local color regions, small acetic acid-white areas, and texture details. The classification network further combines multi-scale feature transfer and cross-layer fusion structures to jointly analyze color boundaries, local color changes, overall acetic acid-white response, and tissue distribution in cervical images, outputting acetic acid image validity or phenotypic classification results.

[0021] The technical solution of the present invention will be further described below with reference to the accompanying drawings of the embodiments.

[0022] Figure 1 This diagram illustrates the structure of a spectral sensing multi-scale color attention module according to an embodiment of the present invention. Figure 1 As shown, a spectral sensing multi-scale color attention module includes several parallelly configured feature extraction submodules, a temperature prediction submodule 102, and an attention calculation submodule 103. The feature extraction submodules extract several different types of features based on the input image. The temperature prediction submodule 102 generates attention temperature parameters based on global features of the input image, whereby the attention temperature parameters control the concentration of channel attention distribution. The attention calculation submodule 103 calculates collaborative attention based on the attention temperature parameters and the different types of features to establish correlations between different color responses along the channel dimension. This enables the system to analyze and determine which channels express similar color phenotypes, which channels provide complementary evidence, and which color responses should be enhanced or suppressed. This allows for joint modeling of the relative changes between acetowhite regions and surrounding red tissue, and the differences between brownish responses and uncolored regions in iodine-stained images.

[0023] like Figure 1As shown, in one embodiment of the present invention, the feature extraction submodule includes a query feature (Q) extraction submodule 111, a key feature (K) extraction submodule 112, a value feature (V) extraction submodule 113, and an intermediate color association feature (A) extraction submodule 114. The query feature extraction submodule 111 includes a first convolutional layer with a depth of 3x3 and a stride of 2, used to generate query features. The key feature extraction submodule 112 includes a second convolutional layer with a depth of 3x3 and a stride of 2, used to generate key features. The value feature extraction submodule 113 includes a third convolutional layer with a depth of 1x1 and a stride of 1, used to generate value features. The intermediate color association feature extraction submodule 114 includes a first and a third convolutional layer connected in series, used to generate intermediate color association features.

[0024] In one embodiment of the present invention, the temperature prediction submodule 102 includes a pooling layer, a convolutional layer, and a nonlinear activation layer. The pooling layer is used to obtain global features of the input image through pooling. These global features are then transformed by the convolutional layer and activated by the nonlinear activation layer to generate an attention temperature parameter T. The attention temperature parameter T is used as a control signal to the attention calculation submodule 103 to dynamically control the concentration of channel attention distribution. When there are significant differences in image color, a more concentrated channel selection is formed; when color changes are continuous or color evidence is similar, a relatively smooth channel fusion is used, thus preventing different images from using a fixed color attention intensity. Based on this, the attention calculation submodule 103 can ultimately complete temperature-adjusted collaborative attention calculation in the channel dimension. The nonlinear activation layer can, for example, use the Sigmoid activation function.

[0025] In one embodiment of the present invention, the spectral sensing multi-scale color attention module further includes a local geometric compensation submodule 104, a feature residual fusion submodule 105, and a projection output submodule 106. The local geometric compensation submodule 104 includes a deep convolutional layer and a pointwise convolutional layer, which are used to preserve the edges of local color regions, small white areas, and local textures to compensate for the weakening of spatial details that may be caused by global channel correlation calculation and to reduce excessive spatial diffusion of color response. The deep convolutional layer has a 3x3 kernel, and the pointwise convolutional layer has a 1x1 kernel. The feature residual fusion submodule 105 is connected to the attention calculation submodule 103 and the local geometric compensation submodule 104, and is used to fuse the channel collaborative attention results with local geometric details in a residual manner, avoiding the loss of spatial details caused by global channel modeling. The projection output submodule 106 is used to perform channel projection on the fused features, and includes a 1x1 projection output.

[0026] Based on the spectral perception multi-scale color attention module described above Figure 2 This diagram illustrates the structure of a cervical image classification system according to an embodiment of the present invention. Figure 2 As shown, a cervical image classification system includes a feature extraction module 201, a first cross-layer fusion module 202, a second cross-layer fusion module 203, and a multi-scale classification head 204. The feature extraction module 201 is used for progressive downsampling and multi-scale feature extraction of the backbone, including the extraction of features such as color boundaries, local color changes, overall acetic acid whitening reaction, and tissue distribution. The first cross-layer fusion module 202 is connected to the outputs of each layer of the backbone feature extraction module and is used for upsampling from deep to shallow and cross-layer stitching to transfer and reconstruct high-resolution features from deep to shallow layers. The second cross-layer fusion module 203 is connected to the outputs of each layer of the first cross-layer fusion module 202 and is used for feature transfer from shallow to deep and cross-layer stitching to supplement shallow boundary information to deep layers. The multi-scale classification head 204 is connected to the outputs of each layer of the first cross-layer fusion module 202 and / or the second cross-layer fusion module 203 and is used to output the validity of the acetic acid image or phenotypic classification results based on the fused features at different resolution levels.

[0027] like Figure 2As shown, the feature extraction module 201 includes a raw feature extraction module 211 and several spectral sensing multi-scale color attention modules 212 as described above. The raw feature extraction module 211 is serially connected to each spectral sensing multi-scale color attention module 212, and the raw feature extraction module 211 and the spectral sensing multi-scale color attention modules 212, as well as the individual spectral sensing multi-scale color attention modules 212, are connected through a reorganization and fusion submodule C3K2. The reorganization and fusion module is used to reorganize and fuse cross-layer transferred features. Figure 2 As shown, in one embodiment of the present invention, the original feature extraction module 211 includes two basic convolutional submodules Conv.

[0028] like Figure 2 As shown, the feature extraction module 201 further includes a spatial pyramid fast pooling submodule (SPPF) and a channel-space joint attention aggregation submodule (C2PSA). The spatial pyramid fast pooling submodule (SPPF) is connected to the last layer reshaping and fusion module of the feature extraction module and is used to perform multi-scale pooling and concatenation on deep features. The channel-space joint attention aggregation submodule (C2PSA) is connected to the spatial pyramid fast pooling submodule and is used to perform joint attention weighting on deep features in the channel dimension and the spatial dimension.

[0029] like Figure 2 As shown, the first cross-layer fusion module 202 includes an upsampling submodule Upsample, a feature stitching submodule Concat, and a re-fusion submodule C3K2. The upsampling submodule is used to restore the spatial resolution of the fusion features from deeper layers, aligning them with the corresponding shallower features in size before sending them to the feature stitching submodule. The feature stitching submodule is used to stitch the upsampled features with the features from a shallower layer of the feature extraction module in the channel dimension. The re-fusion submodule is used to re-fuse the stitched features and output high-resolution fusion features at the corresponding scale.

[0030] like Figure 2 As shown, the second cross-layer fusion module 203 includes a convolution submodule Conv, a feature splicing submodule Concat, and a re-fusion submodule C3K2. The convolution submodule is used to perform convolution transformation and downsampling on the features. The feature splicing submodule is used to splice the downsampled features with the features from a deeper layer of the first cross-layer fusion module in the channel dimension. The re-fusion submodule is used to re-fuse the spliced ​​features and output the fused features at the corresponding scale.

[0031] like Figure 2As shown, in one embodiment of the present invention, the multi-scale classification head 204 includes three classification output terminals 241 connected to three different fusion level C3K2 output terminals, constituting a multi-scale classification output terminal. The three classification output terminals respectively receive the fusion features output by the first cross-layer fusion module 202 and / or the second cross-layer fusion module 203. The shallow features mainly preserve color boundaries and subtle color variations, while the deep features are used to express a larger range of acetic acid whitening reaction, tissue distribution, and overall cervical phenotype. The classification outputs at different scales collectively provide the basis for analyzing local color changes and overall tissue appearance.

[0032] Although various embodiments of the invention have been described above, it should be understood that they are presented by way of example only and not as limitations. It will be apparent to those skilled in the art that various combinations, modifications, and alterations can be made without departing from the spirit and scope of the invention. Therefore, the breadth and scope of the invention disclosed herein should not be limited by the exemplary embodiments disclosed above, but should be defined solely by the appended claims and their equivalents.

Claims

1. A spectral-sensing multi-scale color attention module, characterized in that, include: Several parallel feature extraction submodules are configured to extract several different types of features based on the input image. These submodules include a query feature extraction submodule, a key feature extraction submodule, a value feature extraction submodule, and an intermediate color association feature extraction submodule. The query feature extraction submodule includes a first convolutional layer with a 3x3 kernel and a stride of 2, configured to generate query features. The key feature extraction submodule includes a second convolutional layer with a 3x3 kernel and a stride of 2, configured to generate key features. The value feature extraction submodule includes a third convolutional layer with a 1x1 kernel, configured to generate value features. The intermediate color association feature extraction submodule includes a first and a third convolutional layer connected in series, configured to generate intermediate color association features. A temperature prediction submodule is configured to generate attention temperature parameters based on global features of the input image, wherein the attention temperature parameters are configured to control the concentration of channel attention distribution. The attention calculation submodule is configured to calculate collaborative attention based on the attention temperature parameter and the different types of features, so as to establish the correlation between different color responses in the channel dimension; The local geometry compensation submodule includes a depth convolutional layer and a pointwise convolutional layer, and is configured to preserve the edges of local color regions, small acetic white regions, and local textures. The feature residual fusion submodule, which is connected to the attention calculation submodule and the local geometry compensation submodule, is configured to fuse the channel collaborative attention results with local geometric details in a residual manner.

2. The spectral sensing multi-scale color attention module as described in claim 1, characterized in that, The temperature prediction submodule includes: A pooling layer, configured to acquire global features of the input image through pooling; A convolutional layer, which is connected to the pooling layer, is configured to perform a convolutional transformation on the global features; A nonlinear activation layer, connected to the convolutional layer, is configured to generate the attention temperature parameter based on the output after the convolutional transformation.

3. The spectral sensing multi-scale color attention module as described in claim 1, characterized in that, Also includes: The projection output submodule is configured to perform channel projection on the fused features.

4. A classification system for cervical images, characterized in that, include: The feature extraction module includes a serially connected original feature extraction module and a plurality of spectral sensing multi-scale color attention modules as described in any one of claims 1 to 3, wherein the original feature extraction module and the spectral sensing multi-scale color attention modules, as well as the spectral sensing multi-scale color attention modules, are connected through a reshaping and fusion module. The first cross-layer fusion module is connected to each remodeling fusion module of the feature extraction module and is configured to perform upsampling and cross-layer stitching from deep to shallow to transfer and reconstruct high-resolution features from deep to shallow layers. The second cross-layer fusion module is connected to the outputs of each layer of the first cross-layer fusion module and is configured to perform feature transfer and cross-layer splicing from shallow to deep to supplement the shallow boundary information to the deep layer. A multi-scale classification head is connected to the outputs of each layer of the first cross-layer fusion module and / or the second cross-layer fusion module, and is configured to output classification results based on the fusion features of different resolution levels.

5. The classification system as described in claim 4, characterized in that, The original feature extraction module includes two basic convolutional layers.

6. The classification system as described in claim 4, characterized in that, The feature extraction module further includes: The spatial pyramid fast pooling submodule is connected to the last layer reshaping and fusion module of the feature extraction module and is configured to perform multi-scale pooling and splicing on deep features. The channel-space joint attention aggregation submodule is connected to the spatial pyramid fast pooling submodule and is configured to perform joint attention weighting on deep features in both the channel and spatial dimensions.

7. The classification system as described in claim 4, characterized in that, The first cross-layer fusion module includes: The upsampling submodule is configured to restore the spatial resolution of the fused features from deeper layers, align them in size with the corresponding shallow features, and then send them to the feature stitching submodule. The feature concatenation submodule is configured to concatenate the upsampled features with the features from the previous layer of the feature extraction module in the channel dimension. The remodeling and fusion submodule is configured to remodel and fuse the stitched features, and output high-resolution fused features at the corresponding scale.

8. The classification system as described in claim 4, characterized in that, The second cross-layer fusion module includes: The convolutional submodule is configured to perform convolutional transformations and downsampling on the features; The feature splicing submodule is configured to splice downsampled features with features from the previous layer of the first cross-layer fusion module in the channel dimension. The remodeling and fusion submodule is configured to remodel and fuse the stitched features and output fused features at the corresponding scale.

Images