A tuberculosis detection method based on a multi-scale feature fusion algorithm

Abstract

The application discloses a kind of tuberculosis detection methods based on multi-scale feature fusion algorithm, it is related to tuberculosis detection field, including the following steps: tuberculosis Xray color image is input to network and feature map is extracted;By transverse connection, the channel of different scale feature map is aligned;F3 and F4 features are fused to obtain new F4 features, then new F4 features and F5 features are fused, to obtain final fusion features;Characteristics are mapped to classification results, output whether the judgment of suffering from tuberculosis.The application compared with prior art has the advantages that (1) the context information between each level feature is mined, the semantic consistency of multilevel feature is enhanced, and the expression ability of feature is significantly improved;(2) in the feature fusion process, the local area feature and local channel of each level are deeply mined, so that the feature expression after fusion is more refined;(3) by dynamically adjusting the weight distribution of local feature and channel, stronger task adaptability and robustness are shown.

Description

Technical Field

[0001] This invention relates to the field of tuberculosis detection, specifically to a tuberculosis detection method based on a multi-scale feature fusion algorithm. Background Technology

[0002] Tuberculosis lesions exhibit complex and varied characteristics in X-ray images, with diverse sizes, shapes, and locations, posing a challenge to accurate identification. Multi-scale feature fusion is a promising solution to this problem. Multi-scale features contain rich information; low-level features often include detailed information such as lesion outlines and textures, while high-level features contain rich semantic information, such as lesion type and location. Effectively fusing this information from different levels can significantly enhance lesion identification capabilities and improve the robustness of detecting complex and variable lesions.

[0003] While multi-scale feature fusion methods have demonstrated certain advantages in combining features from different levels, they still require improvement in several aspects. First, most of these methods only integrate features across different scales without delving into the local features of each level. Since each level of features contains important local features, assigning equal weights to all local features can easily lead to the neglect of key features, resulting in insufficient adaptability of the model when handling complex and variable tasks. Second, current multi-scale fusion methods fail to effectively utilize the contextual information between local features at different levels. Most current methods tend to extract key local features from each level individually and then combine them. However, this fusion approach is not necessarily efficient, as it ignores the dependencies between local features at different levels and cannot dynamically allocate the weights of key components in each level of features according to complex and variable tasks. Finally, current multi-scale fusion methods also neglect the information interaction between feature channels at different levels. Different channels reflect different semantics or specific attributes in the features. Without a reasonable mechanism to coordinate the connections between these channels, consistency across features will be lost, and it will be impossible to dynamically strengthen task-relevant key channels and suppress irrelevant or noisy channels. Summary of the Invention

[0004] This invention enhances the characterization capability of multi-scale feature fusion, thereby improving the detection performance of tuberculosis in X-ray images.

[0005] To solve the above-mentioned technical problems, the technical solution provided by this invention is: a tuberculosis detection method based on a multi-scale feature fusion algorithm, comprising the following steps:

[0006] S1. Input the tuberculosis X-ray color image into the network and extract the feature map;

[0007] S2. Align channels of feature maps at different scales by lateral connection;

[0008] S3. Fuse the F3 and F4 features to obtain a new F4 feature, and then fuse the new F4 feature with the F5 feature to obtain the final fused feature.

[0009] S4. Map the features to the classification results and output the judgment of whether the patient has tuberculosis.

[0010] Furthermore, in S1, a 224*224 tuberculosis X-ray color image is input into the network using ResNet50 as the backbone. The image size is reduced to 112*112 and the number of channels is increased to 64 through the Conv1 layer (7*7 convolutional kernels and max pooling). The Conv1 layer contains 7*7 convolutional kernels and max pooling. Subsequently, feature maps are extracted layer by layer from Conv2 to Conv5, with the resolution gradually decreasing and the number of channels gradually increasing. The feature maps include C2 with a size of 56×56×256, C3 with a size of 28×28×512, C4 with a size of 14×14×1024, and C5 with a size of 7×7×2048.

[0011] Furthermore, in S2, the lateral connections adjust the number of channels in the feature maps of each layer to a uniform 256 through 1×1 convolutions.

[0012] Furthermore, the specific steps of S3 include:

[0013] S3.1 and F3 reduce their size by downsampling, making their spatial dimensions 14×14×256 and matching the spatial dimensions of F4. F3 and F4 are fused by cross-attention. The output of F3 is added to the original F4 features to obtain new F4 features.

[0014] S3.2 and F4 reduce their size by downsampling, making their spatial dimensions 7×7×128 and matching the spatial dimensions of F5. F4 and F5 are then fused with cross-attention. The output of F4 is added to the original F5 features to obtain new F5 features.

[0015] Furthermore, the specific steps of the cross-attention fusion method include:

[0016] (1) Spatial cross-attention fusion: low-level feature F i After downsampling, its size and high-level feature F i+1 They are identical, both are R in size. H*W*C Where C is the number of channels, H is the height, and W is the width;

[0017] F i+1 and F i Shape converted to R N*C ,in:

[0018] N = H * W

[0019] In the formula, N represents N local features;

[0020] Then add high-level features F i+1 Multiplying by a trainable matrix yields the vector Q. i+1 Low-level features F i Multiplying by a trainable matrix yields two vectors K. i and V i ,in:

[0021] Q i+1 ,K i V i ∈R N*C

[0022] In the formula, Q i+1 It was through F i+1 K contains information about high-level characteristics. i and V i It contains information about low-level features;

[0023] Q i+1 With K i After multiplying by the transpose and applying softmax, the correlation matrix can be obtained. in:

[0024]

[0025] Will With V i Multiplication yields the characteristics of spatial cross attention. in:

[0026]

[0027] In the formula, Transform it into R H*W*C ;

[0028] (2) Channel cross-attention fusion: low-level feature F i After downsampling, its size and high-level feature F i+1 They are identical, both are R in size. H*W*C In the formula, C is the number of channels, H is the height, and W is the width; F i+1 and F i Average pooling was performed to obtain g respectively i+1 and g i ,in:

[0029]

[0030] In the formula, p and q represent the pixels in the p-th row and q-th column, respectively, where g i+1 and g i The size is R C ;

[0031] g is processed using fully connected layers FC1 and FC2, the ReLU activation function, and the Sigmoid activation function. i+1 and g i Perform a nonlinear mapping, where:

[0032] g i =Sigmiod(FC2(ReLU(FC1(g i ))))

[0033] g i+1 =Sigmoid(FC2(ReLU(FC1(g)) i+1 ))))

[0034] In the formula, FC1 reduces the dimensionality of the features to R. C / 8 To introduce non-linearity, the ReLU activation function FC2 restores the dimension to the original R. C The Sigmoid function normalizes the weights.

[0035] g i+1 transpose and g i Matrix multiplication is performed between the channels, and finally a softmax layer is applied to obtain channel attention. in:

[0036]

[0037] Multiplying the obtained attention weights by the original features, the features after channel cross-attention are:

[0038]

[0039] In the formula, Transform it into R H*W*C ;

[0040] (3) Fusion of spatial attention and channel attention: a new high-level feature Equal to the original F i+1 Spatial cross-attention features and spatial cross-attention features The sum of, where:

[0041]

[0042] Furthermore, a tuberculosis detection system based on a multi-scale feature fusion algorithm is also provided, including:

[0043] The bottom-up multi-scale feature extraction module borrows the design method of FPN. In the process of extracting multi-scale features from the bottom up, ResNet50 is used as the backbone network. The feature maps of different layers capture features of different scales. The lower the layer, the more attention is paid to detailed information, while the higher the layer, the more attention is paid to semantic information.

[0044] The horizontal connection module ensures the consistency of the number of channels during feature fusion through horizontal connections. After adjusting the number of channels, the C3, C4, and C5 feature maps are converted into F3, F4, and F5.

[0045] The multi-scale feature fusion module consists of two parts: the first part is to fuse F3 and F4 features to obtain a new F4 feature, and the second part is to fuse the new F4 feature with F5 features to obtain the final feature.

[0046] The classification module receives the fused features, maps the features to classification results through a fully connected layer or classifier, and outputs a judgment on whether the patient has tuberculosis.

[0047] Furthermore, a computer-readable storage medium is also provided, characterized in that: the storage medium stores a computer program, which, when executed by a processor, implements the method of any one of claims 1-5.

[0048] Furthermore, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: when the processor executes the program, it implements the method of any one of claims 1-5.

[0049] The advantages of this invention compared with the prior art are: (1) It fully explores the contextual information between features at each level, enhances the semantic consistency of multi-level features, and significantly improves the expressive power of features; (2) In the feature fusion process, it deeply explores the local regional features and local channels at each level, making the fused feature expression more refined; (3) By dynamically adjusting the weight distribution of local features and channels, the model can better adapt to complex and ever-changing task requirements, showing stronger task adaptability and robustness. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0051] Figure 2 This is a schematic diagram of the spatial cross-attention fusion process of the present invention.

[0052] Figure 3 This is a schematic diagram of the channel cross-attention fusion process of the present invention. Detailed Implementation

[0053] The present invention will now be described in further detail with reference to the accompanying drawings.

[0054] Example 1

[0055] like Figure 1 , Figure 2 and Figure 3 As shown, this embodiment provides a multi-scale feature fusion algorithm based on spatial and channel cross-attention mechanisms for tuberculosis detection. This algorithm fully utilizes the contextual information of multi-scale features through the cross-attention mechanism. By calculating the dependencies between local features at different levels, it can effectively filter out low-level detailed features that are more relevant to high-level semantic information, thereby constructing a more comprehensive and hierarchical feature representation. Simultaneously, by calculating the similarity between channels at different levels, low-level channel features are reasonably enhanced or suppressed under the guidance of high-level channels, further strengthening the model's expressive power. This operation enables the model to adaptively adjust the distribution of local features and channel weights in each layer, better adapting to different task requirements. This flexible weight allocation mechanism endows the model with both global perception and local detail attention capabilities when facing complex lesion morphologies, diverse location distributions, and size differences, thereby significantly improving the efficiency of feature fusion and task performance.

[0056] The following section describes the specific implementation process of this method, and its structure is as follows: Figure 1 As shown, it consists of four parts: a bottom-up multi-scale feature extraction module, a lateral connection module, a multi-scale fusion module, and a classification module. Specifically:

[0057] I. Bottom-up multi-scale feature extraction module

[0058] This section borrows the design approach of FPN, using ResNet50 as the backbone network for bottom-up multi-scale feature extraction. A 224*224 tuberculosis X-ray color image is input into the network, and the size is reduced to 112*112 through the Conv1 layer (7*7 convolutional kernels and max pooling), increasing the number of channels to 64. Subsequently, feature maps are extracted layer by layer from Conv2 to Conv5, with the resolution gradually decreasing and the number of channels gradually increasing. These feature maps are C2 (56×56×256), C3 (28×28×512), C4 (14×14×1024), and C5 (7×7×2048). Feature maps at different layer numbers capture features at different scales. Lower layer numbers focus more on detailed information, while higher layer numbers focus more on semantic information.

[0059] II. Horizontal Connection Module

[0060] The role of lateral connections is to align the channels of feature maps at different scales. Since the multi-scale features extracted from the bottom up have different numbers of channels, consistency in the number of channels needs to be ensured during feature fusion. Therefore, lateral connections use 1×1 convolutions to adjust the number of channels in each layer of feature maps to a uniform 256, thus ensuring consistency between feature maps of different scales during the fusion process. After adjusting the number of channels, feature maps C3, C4, and C5 are transformed into F3, F4, and F5, respectively.

[0061] III. Multi-scale Feature Fusion Module

[0062] The fusion process is generally divided into two steps. First, the F3 and F4 features are fused to obtain new F4 features. Then, the new F4 features are fused with F5 features to obtain the final features. Specifically: (1) The fusion process of F3 and F4 features is as follows: First, F3 is downsampled to reduce its size so that it matches the spatial dimension of F4 (becoming 14×14×256). Then, F3 and F4 are fused based on cross-attention (cross-attention will be introduced in detail in a later section). The output result is added to the original F4 features to obtain new F4 features. This addition operation is inspired by the design idea of ​​ResNet, which can alleviate the gradient vanishing problem. If the features learned after the cross-attention mechanism are not good enough, the network can also degenerate to use the original F4 features. (2) Feature fusion of F4 and F5: The feature fusion process of F4 and F5 is similar to the feature fusion operation of F3 and F4, so it will not be described again.

[0063] Among them: (1) The spatial cross-attention fusion steps are as follows:

[0064] Low-level features F i After downsampling, its size and high-level feature F i+1 Consistency, i.e., R H*W*C Where C is the number of channels, H is the height, and W is the width. Figure 2 F was shown in the middle. i+1 and F i For ease of demonstration, the feature fusion process... Figure 2 The values ​​of H and W are both set to 14. First, F... i+1 and F i Shape converted to R N*C Where N = H * W, and N represents N local features. Then the high-level features F i+1 The vector Q is obtained after a linear transformation (multiplied by a trainable matrix). i+1 Low-level features F i After the same operation, two vectors K are obtained. i and V i Q i+1 ,K i Vi ∈R N*C Q i+1 It was through F i+1 Obtained through a linear transformation, it therefore contains information about high-level features. Similarly, K... i and V i It also includes information about low-level features. Then Q... i+1 With K i After multiplying by the transpose and applying softmax, the correlation matrix can be obtained. It reflects the correlation between N local features at the lower level and N local features at the higher level. The calculation formula is shown in Formula 1:

[0065]

[0066] This correlation matrix With V i Multiplying them together yields the characteristics of spatial cross-attention. Its formula is

[0067]

[0068] in It needs to be transformed into R H*W*C .

[0069] By calculating the similarity and dependencies between low-level detailed local features and high-level semantic local features, we can effectively filter out detailed features that are more relevant to the semantic information. This operation allows the model to more accurately integrate contextual information and optimize the interaction between features, enabling low-level details to be reasonably strengthened or suppressed under semantic guidance, ultimately forming a more in-depth and hierarchical feature representation. This three-dimensional information fusion approach helps improve the model's ability to understand and analyze complex scenes, resulting in more comprehensive and accurate feature representations.

[0070] (2) The steps for channel cross-attention fusion are as follows:

[0071] (2.1) Global Information Aggregation

[0072] Low-level features F i After downsampling, its size and high-level feature F i+1 Consistency, i.e., R C*H*W Where C is the number of channels, H is the height, and W is the width. Figure 3 F was shown in the middle. i+1 and F i For ease of demonstration, the feature fusion process... Figure 3 In this context, H and W are both set to 14, and C is set to 256. To obtain global context information, F... i+1 and Fi Average pooling was performed to obtain g respectively i+1 and g i ,

[0073]

[0074] Where p and q represent the pixels in the p-th row and q-th column. At this point, g... i+1 and g i The size is R C .

[0075] (2.2) Nonlinear mapping

[0076] g is processed using fully connected layers FC1 and FC2, the ReLU activation function, and the Sigmoid activation function. i+1 and g i Perform nonlinear mapping.

[0077] g i =Sigmoid(FC2(ReLU(FC1(g)) i ))))

[0078] g i+1 =Sigmoid(FC2(ReLU(FC1(g)) i+1 ))))

[0079] FC1 reduces the dimensionality of the features to R. C / 8 To introduce non-linearity, the ReLU activation function FC2 restores the dimension to the original R. C The Sigmoid function normalizes the weights.

[0080] (2.3) Calculation of attention weights

[0081] g i+1 transpose and g i Matrix multiplication is performed between the channels, and finally a softmax layer is applied to obtain the channel attention map. It can reflect the correlation between channels, where:

[0082]

[0083] (2.4) Reweighting features

[0084] Multiplying the obtained attention weights by the original features, the features after channel cross-attention are:

[0085]

[0086] in, Finally, transform it into R. H*W*C .

[0087] Channel features represent information extracted by different filters. The importance of each channel varies depending on the task. By calculating the dependencies between low-level and high-level channels, contextual information can be integrated more accurately, and the interactions between features can be optimized. This effectively filters out low-level channels that are more relevant to high-level semantic information, allowing the model to dynamically allocate channel weights according to task requirements. Therefore, channel cross-attention can improve the model's understanding and dynamic adjustment of relationships between channels, thus enabling better fusion of multi-scale features.

[0088] (2.5) Spatial and channel attention fusion

[0089] Ultimately, new high-level features Equal to the original F i+1 Spatial cross-attention features and spatial cross-attention features The sum, its formula is as follows:

[0090]

[0091] IV. Classification Module

[0092] The final classification module receives the fused features, maps the features to classification results through a fully connected layer or classifier (such as Softmax or Sigmoid), and outputs a judgment on whether the patient has tuberculosis (TB or Normal).

[0093] Example 2

[0094] like Figure 2 and Figure 3 As shown, this embodiment also provides a tuberculosis detection system based on a multi-scale feature fusion algorithm, including:

[0095] The bottom-up multi-scale feature extraction module borrows the design method of FPN. In the process of extracting multi-scale features from the bottom up, ResNet50 is used as the backbone network. The feature maps of different layers capture features of different scales. The lower the layer, the more attention is paid to detailed information, while the higher the layer, the more attention is paid to semantic information.

[0096] The horizontal connection module ensures the consistency of the number of channels during feature fusion through horizontal connections. After adjusting the number of channels, the C3, C4, and C5 feature maps are converted into F3, F4, and F5.

[0097] The multi-scale feature fusion module consists of two parts: the first part is to fuse F3 and F4 features to obtain a new F4 feature, and the second part is to fuse the new F4 feature with F5 features to obtain the final feature.

[0098] The classification module receives the fused features, maps the features to classification results through a fully connected layer or classifier, and outputs a judgment on whether the patient has tuberculosis.

[0099] In addition, this embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1-5.

[0100] In addition, this embodiment also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: when the processor executes the program, it implements the method of any one of claims 1-5.

[0101] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A tuberculosis detection method based on a multi-scale feature fusion algorithm, characterized in that... Includes the following steps: S1. Input the tuberculosis X-ray color image into the network and extract the feature map; S2. Align channels of feature maps at different scales by lateral connection; S3. Fuse the F3 and F4 features to obtain a new F4 feature, and then fuse the new F4 feature with the F5 feature to obtain the final fused feature. S4. Map the features to the classification results and output the judgment of whether the patient has tuberculosis.

2. The tuberculosis detection method based on a multi-scale feature fusion algorithm according to claim 1, characterized in that... In step S1, a 224*224 tuberculosis X-ray color image is input into the network using ResNet50 as the backbone network. The image size is reduced to 112*112 and the number of channels is increased to 64 through the Conv1 layer (7*7 convolutional kernels and max pooling). The Conv1 layer contains 7*7 convolutional kernels and max pooling. Subsequently, feature maps are extracted layer by layer from Conv2 to Conv5, with the resolution gradually decreasing and the number of channels gradually increasing. The feature maps include C2 with a size of 56×56×256, C3 with a size of 28×28×512, C4 with a size of 14×14×1024, and C5 with a size of 7×7×2048.

3. The tuberculosis detection method based on a multi-scale feature fusion algorithm according to claim 1, characterized in that... In S2, the lateral connections adjust the number of channels in each layer's feature map to a uniform 256 through 1×1 convolution.

4. The tuberculosis detection method based on a multi-scale feature fusion algorithm according to claim 1, characterized in that... The specific steps of S3 include: S3.1 and F3 reduce their size by downsampling, making their spatial dimensions 14×14×256 and matching the spatial dimensions of F4. F3 and F4 are fused by cross-attention. The output of F3 is added to the original F4 features to obtain new F4 features. S3.2 and F4 reduce their size by downsampling, making their spatial dimensions 7×7×128 and matching the spatial dimensions of F5. F4 and F5 are then fused with cross-attention. The output of F4 is added to the original F5 features to obtain new F5 features.

5. The tuberculosis detection method based on a multi-scale feature fusion algorithm according to claim 4, characterized in that... The specific steps of the cross-attention fusion method include: (1) Spatial cross-attention fusion: low-level feature F i After downsampling, its size and high-level feature F i+1 Both are identical, with a size of R. H*W*C Where C is the number of channels, H is the height, and W is the width; F i+1 and F i Shape converted to R N*C ,in: N = H * W In the formula, N represents N local features; Then add high-level features F i+1 Multiplying by a trainable matrix yields the vector Q. i+1 Low-level features F i Multiplying by a trainable matrix yields two vectors K. i and V i ,in: Q i+1 ,K i ,V i ∈R N*C In the formula, Q i+1 It was through F i+1 K contains information about high-level characteristics. i and V i It contains information about low-level features; Q i+1 With K i After multiplying by the transpose and applying softmax, the correlation matrix can be obtained. in: Will With V i Multiplication yields the characteristics of spatial cross attention. in: In the formula, Transform it into R H*W*C ; (2) Channel cross-attention fusion: low-level feature F i After downsampling, its size and high-level feature F i+1 Consistent, its size is R H*W*C In the formula, C is the number of channels, H is the height, and W is the width; F i+1 and F i Average pooling was performed to obtain g respectively i+1 and g i ,in: In the formula, p and q represent the pixels in the p-th row and q-th column, respectively, where g i+1 and g i The size is R C ; g is processed using fully connected layers FC1 and FC2, the ReLU activation function, and the Sigmoid activation function. i+1 and g i Perform a nonlinear mapping, where: g i =Sigmoid(FC2(ReLU(FC1(g i )))) g i+1 =Sigmoid(FC2(ReLU(FC1(g i+1 )))) In the formula, FC1 reduces the dimensionality of the features to R. C / 8 To introduce non-linearity, the ReLU activation function FC2 restores the dimension to the original R. C The Sigmoid function normalizes the weights. g i+1 transpose and g i Matrix multiplication is performed between the channels, and finally a softmax layer is applied to obtain channel attention. in: Multiplying the obtained attention weights by the original features, the features after channel cross-attention are: In the formula, Transform it into R H*W*C ; (3) Fusion of spatial attention and channel attention: a new high-level feature Equal to the original F i+1 Spatial cross-attention features and spatial cross-attention features The sum of, where:

6. A tuberculosis detection system based on a multi-scale feature fusion algorithm, characterized in that... include: The bottom-up multi-scale feature extraction module draws on the design method of FPN. In the process of extracting multi-scale features from the bottom up, ResNet50 is used as the backbone network. The feature maps of different layers capture features of different scales. The lower the layer, the more attention is paid to detailed information, while the higher the layer, the more attention is paid to semantic information. The horizontal connection module ensures the consistency of the number of channels during feature fusion through horizontal connection. After adjusting the number of channels, the C3, C4, and C5 feature maps are converted into F3, F4, and F5. The multi-scale feature fusion module has two parts in its fusion process: the first part is to fuse F3 and F4 features to obtain a new F4 feature, and the second part is to fuse the new F4 feature with F5 features to obtain the final feature. The classification module receives the fused features, maps the features to classification results through a fully connected layer or classifier, and outputs a judgment on whether the patient has tuberculosis.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1-5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the method described in any one of claims 1-5.

Images