A chest radiograph lesion detection method based on enhanced feature extraction and fusion

Abstract

The present invention belongs to the field of image processing technology and provides a chest X-ray lesion detection method based on enhanced feature extraction and fusion, comprising: inputting a chest X-ray into a target detection model to obtain a lesion detection result; marking the lesion detection result in the chest X-ray; the target detection model comprises: a backbone network: used to extract N-1 feature maps of different scales from the chest X-ray, including a reference feature map and a non-reference feature map; a neck network: performing size transformation on the non-reference feature map; obtaining a spliced ​​feature map, fusing the spliced ​​feature map to obtain a fused feature map and distributing it to obtain a hierarchical distribution feature map; fusing the fused features of a preset distribution level and the feature map of level N to obtain multiple detection features; a head output layer: obtaining a lesion detection result based on the multiple detection features; the present invention reduces information loss, can obtain more detailed features, and at the same time captures the overall structural information of the chest X-ray, can adapt to targets of different sizes, and improves the accuracy of lesion target detection.

Description

Technical Field

[0001] The invention relates to the field of image processing technology, and in particular to a chest X-ray lesion detection method based on enhanced feature extraction and fusion. Background Art

[0002] CXR (Chest X-Ray) is a chest X-ray or chest film or chest plain film. It is an image formed by X-rays penetrating chest tissue, which can help doctors evaluate structures such as the heart, lungs, and bones. Figure 1 An example of a chest X-ray is shown in FIG. 1 . As can be seen, a chest X-ray is a high-resolution overlapping image, which is characterized by mutual occlusion between tissues and organs and a high proportion of small target lesions.

[0003] The YOLO family of algorithms is commonly used for lesion detection in medical images such as CT scans and demonstrates high detection efficiency. The YOLO family of algorithms typically consists of a backbone network, a neck network, and a head output layer. The backbone network extracts multi-scale features from medical images. The neck network fuses these multi-scale features to enhance detection of objects of varying sizes. The head output layer generates the final detection result based on the output features of the neck network, including the object category, location (bounding box), and confidence level.

[0004] In the YOLO series of algorithms, the neck structure generally uses the Feature Pyramid Network (FPN) and its improved version to achieve multi-level feature fusion. However, these information fusion methods have an obvious flaw: when cross-layer fusion information is required, the traditional FPN-like structure cannot transmit information losslessly, which hinders the YOLO network from better information fusion. Specifically, Figure 2 As shown in the figure, in the feature fusion structure of the traditional FPN, the P3, P4, and P5 layers are arranged from top to bottom. When the P3 layer wants to use the P4 layer information, it can directly access and fuse the P4 layer information. Figure 2The "Fuse" module stands for fusion. If the P3 layer wants to utilize information from the P5 layer, it must fuse the information from the P4 and P5 layers and then fuse it with the information from the P3 layer to indirectly obtain the P5 layer information. This method of transmission may result in a large amount of information loss during the calculation process. Information exchange between layers can only exchange information selected in the intermediate process. Unselected information will be discarded during the transmission process, resulting in the information of a certain layer only being able to partially utilize adjacent layers, while weakening the help provided by other layers. Therefore, when the YOLO series of algorithms are used for lesion identification and localization in chest radiographs, the above-mentioned defects will cause the loss of a large number of subtle features in the chest radiographs, resulting in low accuracy in small target detection. Although related technologies directly use high-resolution feature maps to add detection heads to improve the detection performance of small targets, the high-resolution feature maps require a large number of pixels to process each target, which greatly increases the computational complexity of the model. Summary of the Invention

[0005] The present application aims to at least solve the technical problems existing in the prior art and provide a chest X-ray lesion detection method based on enhanced feature extraction and fusion.

[0006] In a first aspect, the present application provides a chest X-ray lesion detection method based on enhanced feature extraction and fusion, comprising: obtaining a chest X-ray; inputting the chest X-ray into a trained target detection model to obtain a lesion detection result; marking the lesion detection result in the chest X-ray, and outputting the marked chest X-ray; the target detection model comprises: a backbone network: comprising N levels of feature extraction networks connected in sequence, the N levels of feature extraction networks respectively extracting feature maps of different scales of the chest X-ray; inputting N-1 feature maps obtained by the feature extraction networks of levels 2 to N into a neck network; wherein the level index of the feature extraction network is 1 to N, the N-1 feature maps include 1 reference feature map and N-2 non-reference feature maps, the level index of the reference feature map is greater than 2 and less than N, and N is a positive number greater than or equal to 3 Integer; Neck network: with the size of the reference feature map as a reference, the non-reference feature map is resized; the reference feature map and the resized non-reference feature map are spliced ​​in the channel dimension to obtain a spliced ​​feature map; the spliced ​​feature map is fused to obtain a fused feature map; the fused feature map is distributed to obtain a hierarchical distribution feature map of one or more preset distribution levels; the hierarchical distribution feature maps of one or more preset distribution levels are inversely resized and fused with the feature maps corresponding to the preset distribution levels to obtain fused features of one or more preset distribution levels, where the level index of the preset distribution level is greater than 2 and less than N; multiple detection features are obtained by fusing the fused features of one or more preset distribution levels with the feature map of level N; Head output layer: lesion detection results are obtained based on multiple detection features.

[0007] In a second aspect, the present application provides a chest X-ray lesion detection device based on enhanced feature extraction and fusion, which is used to implement the chest X-ray lesion detection method based on enhanced feature extraction and fusion provided in the first aspect of the present invention, including: a chest X-ray acquisition module, which acquires a chest X-ray; a detection module, which inputs the chest X-ray into a trained target detection model to obtain a lesion detection result; a marking output module, which marks the lesion detection result in the chest X-ray and outputs the marked chest X-ray; the target detection model includes: a backbone network: including N levels of feature extraction networks connected in sequence, and the N levels of feature extraction networks respectively extract feature maps of different scales of the chest X-ray; the N-1 feature maps obtained by the feature extraction networks of levels 2 to level N are input into the neck network; wherein the level index of the feature extraction network is 1 to N, and the N-1 feature maps include 1 reference feature map and N-2 non-reference feature maps, reference The hierarchical index of the reference feature map is greater than 2 and less than N, where N is a positive integer greater than or equal to 3; the neck network: with the size of the reference feature map as a reference, the non-reference feature map is resized; the reference feature map and the resized non-reference feature map are spliced ​​in the channel dimension to obtain a spliced ​​feature map; the spliced ​​feature map is fused to obtain a fused feature map; the fused feature map is distributed to obtain hierarchical distribution feature maps of one or more preset distribution levels; the hierarchical distribution feature maps of one or more preset distribution levels are inversely resized and fused with the feature maps corresponding to the preset distribution levels to obtain fused features of one or more preset distribution levels, where the hierarchical index of the preset distribution level is greater than 2 and less than N; multiple detection features are obtained by fusing the fused features of one or more preset distribution levels and the feature map of level N; the head output layer: lesion detection results are obtained based on multiple detection features.

[0008] In a third aspect, the present application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the chest X-ray lesion detection method based on enhanced feature extraction and fusion as provided in the first aspect of the present invention.

[0009] Beneficial technical effects of the present invention:

[0010] (1) The present invention introduces N-1 levels of feature maps from the backbone network to the neck network. In the neck network, a feature map of a level between level 2 and level N is used as a reference feature map. The size of the reference feature map is used as a reference to resize the non-reference feature map so that the size of the non-reference feature map is the same as that of the reference feature map, thereby completing the alignment of N-1 feature maps. This alignment method can reduce the information loss of the feature maps during the alignment process.

[0011] (2) The reference feature map and the non-reference feature map after size transformation are spliced ​​in the channel dimension to obtain a spliced ​​feature map, the spliced ​​feature map is fused to obtain a fused feature map, the fused feature map is distributed to obtain a hierarchical distribution feature map of more than one preset distribution level, the fused features of more than one preset distribution level and the feature map of level N are fused to obtain multiple detection features, and the head output layer obtains the lesion detection result based on the multiple detection features. It can be seen that the present application introduces the high-resolution level 2 feature map into the neck network for feature fusion, so that the hierarchical distribution feature map has more chest X-ray details and obtains richer semantic information;

[0012] (3) The feature map of level N has a lower resolution but a larger receptive field, which helps the target detection model to more effectively capture the overall structural information of the chest X-ray. Fusion of the feature map with feature maps of other levels in the neck network helps the target detection model better adapt to targets of different sizes and improve detection performance and generalization ability.

[0013] (4) Introducing the feature map of level N into the head output layer: Due to its lower resolution, fewer pixels are required to process each target, thus maintaining the inherent efficiency enjoyed by the YOLO series model and improving target recognition accuracy without significantly increasing the computational complexity;

[0014] Therefore, through the combination of the above-mentioned technical means, the target detection model provided by this application improves the target detection accuracy, especially the small target detection accuracy, without significantly increasing the computational complexity. The lesion detection method provided by this application helps to reduce the number of missed diagnoses and misdiagnoses, and provide clinicians with a more comprehensive and reliable diagnostic basis. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 1 is a flow chart of a chest X-ray lesion detection method based on enhanced feature extraction and fusion in a preferred embodiment of the present invention;

[0016] Figure 2 This is a schematic diagram of the network structure of a target detection model in a preferred embodiment of the present invention;

[0017] Figure 3 This is a schematic diagram of the network structure of the G-CA module in a preferred embodiment of the present invention;

[0018] Figure 4 It is a structural diagram of an electronic device in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0020] In the description of the present invention, it should be understood that the terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention.

[0021] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal communication between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0022] The execution subject of the chest X-ray lesion detection method based on enhanced feature extraction and fusion provided by the present invention includes but is not limited to at least one of the electronic devices such as the server, the terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the chest X-ray lesion detection method based on enhanced feature extraction and fusion can be executed by software or hardware installed on the terminal device or the server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and big data and artificial intelligence platforms.

[0023] The present invention provides a chest X-ray lesion detection method based on enhanced feature extraction and fusion. In a preferred embodiment, see Figure 1 As shown, chest radiograph lesion detection methods include:

[0024] Step S1: Obtain a chest X-ray.

[0025] For example, the chest X-ray to be tested can be obtained from a chest X-ray device, such as a digital radiography system (DR), a computerized radiography system (CR), a mobile X-ray machine (bedside DR), etc., via a network interface or a data cable. Alternatively, the chest X-ray to be tested can be read from a memory or database storing chest X-rays.

[0026] Step S2: Input the chest X-ray into the trained target detection model to obtain lesion detection results.

[0027] Illustratively, the lesion detection results include target bounding box coordinates, confidence scores, and category labels for multiple lesion targets. The lesion targets are not limited to cysts, nodules, effusions, etc.

[0028] Step S3: Mark the lesion detection result on the chest X-ray and output the marked chest X-ray.

[0029] For example, Figure 2 In the output image, the bounding box is marked in the chest X-ray according to the target bounding box coordinates in the lesion detection result, and the confidence score and category label are displayed in the bounding box.

[0030] In this embodiment, preferably, see Figure 2 , the target detection model includes:

[0031] Backbone network: includes N levels of feature extraction networks connected in sequence, each of which extracts feature maps of different scales of the chest X-ray. The N-1 feature maps obtained by the feature extraction networks at levels 2 to N are input into the neck network. The level index of the feature extraction network is 1 to N, and the N-1 feature maps include 1 reference feature map and N-2 non-reference feature maps. The level index of the reference feature map is greater than 2 and less than N, where N is a positive integer greater than or equal to 3.

[0032] Neck network: With the size of the reference feature map as a reference, the non-reference feature map is resized; the reference feature map and the resized non-reference feature map are spliced ​​in the channel dimension to obtain a spliced ​​feature map; the spliced ​​feature map is fused to obtain a fused feature map; the fused feature map is distributed to obtain a hierarchical distribution feature map of one or more preset distribution levels; the hierarchical distribution feature maps of one or more preset distribution levels are inversely resized and fused with the feature maps corresponding to the preset distribution levels to obtain fused features of one or more preset distribution levels, where the level index of the preset distribution level is greater than 2 and less than N; multiple detection features are obtained by fusing the fused features of one or more preset distribution levels with the feature map of level N.

[0033] Head output layer: obtains lesion detection results based on multiple detection features.

[0034] In this embodiment, the backbone network is used to extract lesion target features from chest X-rays, obtaining feature maps of different scales. The neck network is used to fuse feature maps of different scales to obtain richer semantic information and more accurate target location information. The head output layer is used to generate the final lesion detection results (including target bounding box coordinates, confidence scores, and category labels). The head output layer adopts a decoupled detection head structure, where each detection head is divided into two branches, one for predicting the category and the other for predicting the location information.

[0035] In this embodiment, the feature extraction networks of N levels each include a downsampling convolution to obtain feature maps of N different scales. Only the N-1 feature maps obtained by the feature extraction networks of levels 2 to N in the backbone network are extracted for subsequent neck network fusion processing. Among the N-1 feature maps extracted, excluding the feature maps of levels 2 and N, any feature map can be selected as a reference feature map, and the N-2 feature maps other than the reference feature map in the N-1 feature maps can be used in whole or in part as non-reference feature maps. Preferably, the reference feature map is a feature map at an intermediate level among the N-1 feature maps, which is conducive to reducing information loss in feature map alignment.

[0036] In one example, if Figure 2 As shown, N is 6, and the feature maps B2, B3, B4, B5 and B6 obtained by the feature extraction network of level 2 to level 6 in the backbone network are input into the neck network, where the i′th feature map N represents batch size; Represents the number of channels, and the feature map size is R Bi =W×H, where W represents width, H represents height, and i′∈[2,6]. The feature map B4 at level 4 is selected as the reference feature map, and the feature maps B2, B3, B5, and B6 are all used as non-reference feature maps. Figure 2 In

[15] , the chest radiograph input to the backbone network is 1280×1280×3 in size, the feature map B2 of level 2 is 320×320×128 in size, the feature map B3 of level 3 is 160×160×256 in size, the feature map B4 of level 4 is 80×80×512 in size, the feature map B5 of level 5 is 40×40×768 in size, and the feature map B6 of level 6 is 20×20×1024 in size.

[0037] In this embodiment, the neck network includes two stages of processing.

[0038] In the first stage of processing: with the size of the reference feature map as a reference, the non-reference feature map is resized so that the sizes of N-1 feature maps are the same and are all the same as the size of the reference feature map; the reference feature map and the resized non-reference feature map are spliced ​​in the channel dimension to obtain a spliced ​​feature map; the spliced ​​feature map is fused to obtain a fused feature map; the fused feature map is distributed to obtain hierarchical distribution feature maps of one or more preset distribution levels; the hierarchical distribution feature maps of one or more preset distribution levels are inversely resized and fused with the feature maps corresponding to the preset distribution levels to obtain fused features of one or more preset distribution levels, and the level index of the preset distribution level is greater than 2 and less than N; the size of the hierarchical distribution feature map of the preset distribution level after inverse resizing is the same as the size of the feature map corresponding to the preset distribution level, so as to facilitate fusion. The number of preset distribution levels is greater than or equal to 2 and less than or equal to N-3. Preferably, the preset distribution level includes the level where the reference feature map is located, and at least one level upstream and / or at least one level downstream of the level where the reference feature map is located. In the above example, the prediction distribution levels include level 3, level 4, and level 5. The large-scale feature maps are fused through the first stage processing.

[0039] Second stage processing: reference Figure 2 , which is a top-down feature aggregation, fuses the fusion features of more than one preset distribution level and the feature map of level N to obtain multiple detection features. Preferably, the fusion features of all preset distribution levels and the feature map of level N are fused, and the feature map of level 2 is discarded for fusion. This can avoid the traditional method of directly using the feature map B2 to add the P2 detection head in order to improve the detection performance of small targets. Because the feature map B2 has a high resolution and the number of pixels required to process each target is large, the traditional method greatly increases the model calculation complexity. Therefore, this application only introduces the feature map B2 of level 2 into the fusion part, and does not generate the hierarchical distribution feature map of level 2. It can obtain high-resolution details and retain more subtle features, which helps to improve the detection accuracy while not significantly increasing the inference time. The small-sized feature maps are fused through the second stage processing. In the above example, the fusion features of levels 3, 4 and 5 and the feature map of level 6 are fused to obtain the detection features of 4 levels from level 3 to level 6. The detection features of the 4 levels are respectively input into the detection heads of the corresponding levels in the head output layer.

[0040] In the above example, feature maps B2, B3, B4, B5, and B6 were selected for fusion. The higher-resolution feature map B2 was specifically selected to improve small object detection performance. High-resolution feature maps retain more image details and provide accurate positioning information, but lack rich semantic information. Fusion with global contextual information improves the model's understanding capabilities and is crucial for improving small object detection performance.

[0041] In this embodiment, through the first stage processing and the second stage processing, the detection capability of the target detection model for targets of different sizes in the chest X-ray is improved and the network computing overhead is reduced.

[0042] In a preferred embodiment, it is beneficial to extract the detailed features of the chest X-ray. Figure 2 , the feature extraction network of layer 1 includes downsampling convolution; the feature extraction network of layer N includes downsampling convolution, C2F module and SPPF module connected in sequence; the feature extraction network of any layer between layer 1 and layer N includes downsampling convolution and C2F module connected in sequence.

[0043] In this implementation, both the C2F module and the SPPF module are existing modules in the YOLOv8 network. The C2F module (Cross Stage Partial Fusion) enables better feature extraction, stronger gradient propagation, and higher computational efficiency. The SPPF module (Spatial Pyramid Pooling Fast) efficiently extracts multi-scale features, making it suitable for real-time detection tasks.

[0044] In a preferred embodiment, in order to make the target detection model pay more attention to the abnormal area in the chest X-ray CXR and reduce the interference of background features, the present application introduces an improved global coordinate attention mechanism module (G-CA module). Specifically, in the backbone network, the feature extraction network of at least one level from level 3 to level N-1 is connected to the feature extraction network of the next level through the G-CA module, and / or, the feature extraction network of level N also includes a G-CA module located between the C2F module and the SPPF module. In the above example, referring to Figure 2 The feature extraction network of layer 3 is connected to the feature extraction network of layer 4 through the G-CA module, and the feature extraction network of layer 4 is connected to the feature extraction network of layer 5 through the G-CA module.

[0045] Among them, reference Figure 3 , G-CA module includes:

[0046] The first branch is used to obtain the global channel attention weight g of the input feature map of the G-CA module. The global channel attention weight of the cth channel is available as g c express.

[0047] The second branch is used to obtain the height directional attention weight g of the input feature map of the G-CA module h and width-wise attention weight g w , the height and width direction attention weights of the c-th channel can be respectively and express.

[0048] The multiplication weighted operation unit is used to multiply the input feature map of the G-CA module by the global channel attention weight, the height direction attention weight and the width direction attention weight element by element to obtain the G-CA module output feature y. The feature map y of the cth channel output by the G-CA module c The eigenvalue y in row i and column j c The eigenvalue of (i,j) can be expressed as:

[0049]

[0050] Among them, x c (i, j) represents the cth channel x in the input feature map of the G-CA module c The eigenvalue of the feature graph point (i, j), Indicates the height direction attention weight of the cth channel The weight value of the i-th row in , Represents the attention weight in the width direction of the cth channel The weight value of the jth column in .

[0051] In this embodiment, refer to the attached Figure 3 , preferably, the first branch includes:

[0052] The compression unit compresses the height and width information of each channel of the input feature map of the G-CA module into channel feature values, and the channel feature values ​​of all channels form the feature vector z.

[0053] In one example, the input feature map of the G-CA module is represented as a tensor The output feature of the G-CA module is a transformed tensor of the same size as X The compression unit compresses the spatial information (i.e., height and width) of each channel into a single value (i.e., channel eigenvalue), thereby obtaining a feature vector z containing the global information of all channels:

[0054]

[0055] The first branch convolution unit performs convolution processing on the feature vector to obtain the first branch feature map F. The convolution kernel size of the first branch convolution unit is 1×1.

[0056] The first branch activation function is processed. The activation function is applied to the first branch feature map F to obtain the weight of each channel (i.e., predict the importance of each channel). The weights of all channels constitute the global channel attention weight g. The activation function σ is preferably, but not limited to, a Sigmoid activation function.

[0057] g = σ(F(z)).

[0058] In this embodiment, refer to the attached Figure 3 , preferably, the second branch includes:

[0059] The global average pooling unit performs global average pooling on the input feature map of the G-CA module in the height direction and width direction respectively to obtain the height direction pooling feature z h and width-wise pooling feature z w , so the output of the cth channel at height h can be expressed as:

[0060]

[0061] Similarly, the output of the cth channel at width w can be expressed as:

[0062]

[0063] Among them, W and H represent the width and height of the input feature map of the G-CA module respectively, and x c (h, j) represents the feature map x of the cth channel of the input feature map X of the G-CA module c The eigenvalue at height h (hth row) and jth column, x c (i,w) represents the feature map x of the cth channel of the input feature map X of the G-CA module c The eigenvalue of row i with width w (column w).

[0064] The second branch splicing unit splices the height direction pooling feature and the width direction pooling feature to obtain the splicing pooling feature. The height direction pooling feature z in both the width and height directions of the global receptive field is obtained. h and width-wise pooling feature z w Splicing together to get the splicing pooling feature [z h ,z w ].

[0065] The second branch convolution unit performs convolution processing on the spliced ​​pooling features to obtain the second branch feature map. h ,z w ]The input shared convolution kernel is a 1×1 convolution module, and its dimension is reduced to the original C / r to obtain the second branch feature map.

[0066] The distribution unit performs batch normalization (obtaining feature map F1) and activation function processing on the second branch feature map, obtaining a feature map f of the form 1×(W+H)×C / r. The activated feature map f is distributed into height feature submaps and width feature submaps. The activation function is preferably a Sigmoid activation function.

[0067] The height direction attention weight acquisition unit performs convolution processing (convolution kernel is 1×1) and activation processing (such as Sigmoid activation function) on the height feature sub-graph in sequence to obtain the height direction attention weight g h After the high-level feature sub-graph is convolved, the feature graph f with the same number of channels as the original one is obtained. h .

[0068] The width direction attention weight acquisition unit performs convolution processing (convolution kernel is 1×1) and activation processing (such as Sigmoid activation function) on the width feature sub-graph in sequence to obtain the width direction attention weight g w After the width feature sub-graph is convolved, the feature graph f with the same number of channels as the original one is obtained. w .

[0069] The above process can be described as:

[0070]

[0071] f=σ(F1([z h ,z w ]))

[0072] g h =σ(F h (f h ))

[0073] g w =σ(F w (f w )).

[0074] In this implementation, by generating a global information embedding, direction-aware, and position-sensitive attention map, the model can more accurately identify and locate targets of interest, improving its understanding of spatial structure in CXR anomaly detection. In the G-CA module, two one-dimensional features along the image's height and width are used to encode position information, while a global attention block is used in parallel to extract contextual features. This allows the model to obtain both spatial position information and global contextual feature information for regions of interest, thereby accurately locating distinguishable areas within the image.

[0075] In a preferred embodiment, referring to Figure 2 As shown in Figure 1, in the first stage of the neck network processing, the size of the reference feature map is used as a reference to transform the size of the non-reference feature map, including:

[0076] For non-reference feature maps larger than the reference feature map, the resizing process involves convolving the non-reference feature map with a first resizing convolution based on a preset step size. The preset step sizes for resizing non-reference feature maps at different levels vary, with those at lower levels being smaller than those at higher levels. The kernel size for the first resizing convolution is 3×3.

[0077] For non-reference feature maps that are smaller than the reference feature map, the resizing process is as follows: bilinear interpolation and a second resizing convolution are performed on the non-reference feature map. The convolution kernel size of the second resizing convolution is 1×1.

[0078] In the above example, refer to Figure 2 The resizing process for the non-reference feature maps is as follows: using a 3×3 convolution kernel with strides of 2 and 4, respectively, to resize the non-reference feature maps B2 (320×320) and B3 (160×160) to the same size as the reference feature map B4 (80×80). Next, for the non-reference feature maps B5 (40×40) and B6 (20×20), bilinear interpolation is used in conjunction with a 1×1 convolution operation to bring them to the same size as the reference feature map B4. After these steps, the feature maps of the five levels are unified in width and height. Because chest radiographs are overlapping images and small target abnormalities account for a large proportion, the resizing process described above can reduce information loss during the algorithm's information transmission process, retain more subtle features, and help improve lesion detection accuracy.

[0079] In a preferred embodiment, in the first stage of processing of the neck network, please refer to Figure 2 As shown, the hierarchical distribution feature maps of more than one preset distribution level are inversely transformed in size, including:

[0080] For a preset distribution level whose feature map size is larger than the reference feature map size, the inverse size transformation process for the hierarchical distribution feature map of the preset distribution level is: bilinear interpolation and second size transformation convolution. The convolution kernel size of the second size transformation convolution is 1×1. In the above example, the preset distribution level whose feature map size is larger than the reference feature map size is level 3.

[0081] For a preset distribution layer where the feature map size is smaller than the reference feature map size, the inverse size transformation process for the hierarchical distribution feature map of the preset distribution layer is as follows: the hierarchical distribution feature map of the preset distribution layer is convolved using a first size transformation convolution based on a preset step size. For a preset distribution layer where the feature map size is smaller than the reference feature map size, the preset step sizes are different for different preset distribution layers, with higher levels having larger preset step sizes. No inverse size transformation is performed on the hierarchical distribution feature map of the preset distribution layer where the reference feature map is located.

[0082] In a preferred embodiment, in the first stage of processing of the neck network, please refer to Figure 2 The concatenated feature maps are fused to obtain a fused feature map, including sequentially performing convolution, batch normalization (BN), and activation function processing on the concatenated feature maps. The activation function in the activation function processing is preferably, but not limited to, a ReUL activation function. The convolution kernel in the convolution processing is preferably, but not limited to, 1×1.

[0083] In this embodiment, although the first size transformation convolution or bilinear interpolation + second size transformation convolution is used to change the size of the feature map in the size transformation, which reduces the information, the number of parameters will increase. Therefore, in order to avoid introducing too many parameters, in the splicing feature map fusion process, there is no need to re-parameterize it. Only one 1×1 convolution is used, combined with batch normalization processing BN and activation function processing, which can also effectively fuse features.

[0084] In the above example, after the non-reference feature map is resized, the five levels of feature maps after resizing are spliced ​​in the channel dimension, thereby completing the alignment of the feature maps and obtaining the spliced ​​feature map F align , for the concatenated feature map F align Perform fusion processing to obtain the fusion feature map F fuse , for the fusion feature map F fuse Perform distribution processing (Spl it operation) to obtain the hierarchical distribution feature maps of three preset distribution levels from layer 3 to layer 5, which are represented by F fuse_P3 、F fuse_P4 and F fuse_P5 Since the size of the distributed hierarchical feature maps is consistent with the reference feature map B4 of level 4, the hierarchical distribution feature map F of level 3 needs to be fuse_P3 Perform bilinear interpolation with 1×1 convolution operation to enlarge it to 160×160, and distribute the feature map F to the level 5 layer fuse_P5 A 3×3 convolution operation with a step size of 2 is used to reduce the image to 40×40, and finally fused with the corresponding level 3, level 4 and level 5 feature maps (B3, B4 and B5) to obtain the level 3, level 4 and level 5 fused features P3, P4 and P5 respectively.

[0085] In the above example, the first stage of the neck network processing can be expressed as follows:

[0086] F align =Concat([Conv s＝4 (B2),Conv s＝2 (B3),B4,

[0087] Conv 1×1 (Billnear(B5),Conv 1×1 (Billnear(B6))])

[0088] F fuse =ReUL(BN(Conv 1×1 (F align ))

[0089] F fuse_P3 ,F fuse_P4 ,F fuse_P5 =Split(F fuse )

[0090]

[0091] In this embodiment, in the second stage processing of the neck network, a top-down path aggregation module is used to fuse the fusion features of more than one preset distribution level and the feature map of level N to obtain multiple detection features.

[0092] Please refer to Figure 2 ,In the above example, the path aggregation module includes N-2 aggregation sub-networks, that is, 4 aggregation sub-networks, which correspond one-to-one to the N-2 detection heads in the head output layer and also correspond one-to-one to layers 3 to N in the backbone network.

[0093] The aggregation subnetwork corresponding to level 3 includes a cascaded C2F module and an aggregation convolution, where the output features of the C2F module are the detection features of level 3 and are input into the detection head Detect-P3.

[0094] The aggregation subnetwork corresponding to level 4 includes a cascaded first C2F module, a connection unit, a second C2F module and an aggregation convolution, wherein the input end of the first C2F module is input with the fusion feature P4 of level 4, and the connection unit is used to connect the output features of the aggregation subnetwork corresponding to level 3 and the output features of the first C2F module to obtain the connection features, and use the connection features as the detection features of level 4, and input the detection features of level 4 into the detection head Detect-P4.

[0095] The aggregation subnetwork corresponding to level 5 includes a cascaded third C2F module, a connection unit, a fourth C2F module and an aggregation convolution, wherein the input end of the third C2F module is input with the fusion feature P5 of level 5, and the connection unit is used to connect the output feature of the aggregation subnetwork corresponding to level 4 and the output feature of the third C2F module to obtain the connection feature, and input the connection feature into the fourth C2F module. The output feature of the fourth C2F module is used as the detection feature P5 of level 5, and the detection feature P5 of level 5 is input into the detection head Detect-P5.

[0096] The aggregation subnetwork corresponding to level 6 includes a cascaded connection unit and a fifth C2F module, wherein the connection unit is used to connect the feature map of level 6 and the features output by the aggregation subnetwork corresponding to level 5, and the output features of the fifth C2F module are used as the detection features P6 of level 6, and the detection features P6 of level 6 are input into the detection head Detect-P6.

[0097] In this implementation, the neck network achieves high-quality fusion of features at all levels in the first stage of processing. This uses convolution operations to resize the feature maps, which increases the computational effort somewhat. However, in the second stage, to reduce model computational overhead, a top-down path aggregation module is employed. Specifically, this module begins with high-resolution feature maps and propagates information downward layer by layer through downsampling operations. At each level, the high-resolution feature maps are laterally connected and concatenated with the low-resolution feature maps.

[0098] The neck network's two-stage processing effectively reduces information loss during transmission while ensuring the high performance of the YOLO family of algorithms. This design strategy ultimately filters and retains more critical and useful feature information for the head detection branch, significantly improving the accuracy of chest X-ray abnormality detection and reducing the false positive rate.

[0099] The training process of the target detection model in this application includes:

[0100] Step 1: Construct a chest X-ray sample set. Each chest X-ray sample in the chest X-ray sample set includes a chest X-ray and the corresponding annotation data of each chest X-ray. The annotation data includes the bounding box coordinates, category labels and confidence scores of one or more targets in the chest X-ray. The chest X-ray sample set is divided into a training set, a test set and a validation set.

[0101] Step 2: Build the network structure of the target detection model.

[0102] In step 3, the object detection model is iteratively trained using the training set. During each training session, the object detection model outputs the lesion detection results of the chest radiographs in the training samples. A loss function is calculated based on the lesion detection results and the labeled data of the training samples. Based on the loss function, the network parameters of the object detection model are updated using gradient descent until the training stop condition is met. The training stop condition is when the number of training cycles reaches the preset maximum number of training cycles or when the loss function converges numerically. The loss function includes bounding box localization loss, confidence loss, and classification loss. The classification loss can be the existing cross-entropy loss.

[0103] In step 4, the target detection model is tested and verified using the test set and validation set after the training stop condition is met. When the test and validation pass, the final target detection model is obtained. If the test and validation fail, return to step 3.

[0104] The present invention also discloses a chest X-ray lesion detection device based on enhanced feature extraction and fusion, which is used to implement the above-mentioned chest X-ray lesion detection method based on enhanced feature extraction and fusion, comprising:

[0105] A chest X-ray acquisition module is used to acquire chest X-rays;

[0106] The detection module inputs the chest X-ray into the trained target detection model to obtain the lesion detection results; the marking output module marks the lesion detection results on the chest X-ray and outputs the marked chest X-ray;

[0107] The target detection model includes:

[0108] Backbone network: includes N levels of feature extraction networks connected in sequence, each of which extracts feature maps of different scales of the chest X-ray. The N-1 feature maps obtained by the feature extraction networks at levels 2 to N are input into the neck network. The level index of the feature extraction network is 1 to N, and the N-1 feature maps include 1 reference feature map and N-2 non-reference feature maps. The level index of the reference feature map is greater than 2 and less than N, where N is a positive integer greater than or equal to 3.

[0109] Neck network: With the size of the reference feature map as a reference, the non-reference feature map is resized; the reference feature map and the resized non-reference feature map are spliced ​​in the channel dimension to obtain a spliced ​​feature map; the spliced ​​feature map is fused to obtain a fused feature map; the fused feature map is distributed to obtain a hierarchical distribution feature map of one or more preset distribution levels; the hierarchical distribution feature maps of one or more preset distribution levels are inversely resized and fused with the feature maps corresponding to the preset distribution levels to obtain fused features of one or more preset distribution levels, where the level index of the preset distribution level is greater than 2 and less than N; multiple detection features are obtained by fusing the fused features of one or more preset distribution levels with the feature map of the Nth level;

[0110] Head output layer: obtains lesion detection results based on multiple detection features.

[0111] In this embodiment, the chest X-ray acquisition module, detection module and marking output module correspond one-to-one to steps S1, S2 and S3 in the above-mentioned chest X-ray lesion detection method based on enhanced feature extraction and fusion, and the target detection model is consistent with the target detection model in the above-mentioned chest X-ray lesion detection method based on enhanced feature extraction and fusion, which will not be repeated here.

[0112] The present invention also discloses an electronic device. In one embodiment, the electronic device includes at least one processor; and a memory connected to the at least one processor; wherein,

[0113] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the chest X-ray lesion detection method based on enhanced feature extraction and fusion provided by the present invention.

[0114] like Figure 4 FIG2 is a schematic diagram of the structure of an electronic device for a method for detecting chest X-ray lesions based on enhanced feature extraction and fusion, according to one embodiment of the present invention. The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and executable on the processor 10, such as a program for detecting chest X-ray lesions based on enhanced feature extraction and fusion.

[0115] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing the programs or modules stored in the memory 11 (for example, executing a chest X-ray lesion detection method based on enhanced feature extraction and fusion, etc.), as well as calling the data stored in the memory 11, to perform various functions of the electronic device and process data.

[0116] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the electronic device. Furthermore, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of a chest X-ray lesion detection method program based on enhanced feature extraction and fusion, but can also be used to temporarily store data that has been output or is to be output.

[0117] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0118] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, for displaying information processed in the electronic device and for displaying a visual user interface.

[0119] Figure 4 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 4The structure shown does not limit the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0120] For example, although not shown, the electronic device may further include a power source (such as a battery) for powering various components. Preferably, the power source may be logically connected to at least one processor 10 via a power management device, thereby implementing functions such as charge management, discharge management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0121] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0122] Furthermore, if the module / unit integrated into the electronic device is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0123] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "example," "specific example," "one implementation," "a preferred implementation," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0124] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A chest X-ray lesion detection method based on enhanced feature extraction and fusion, characterized in that: include: Obtain a chest x-ray; Input the chest X-ray into the trained target detection model to obtain the lesion detection results; Marking the lesion detection results on the chest X-ray and outputting the marked chest X-ray; The target detection model includes: Backbone network: includes N levels of feature extraction networks connected in sequence, each of which extracts feature maps of different scales of the chest X-ray. The N-1 feature maps obtained by the feature extraction networks at levels 2 to N are input into the neck network. The level index of the feature extraction network is 1 to N, and the N-1 feature maps include 1 reference feature map and N-2 non-reference feature maps. The level index of the reference feature map is greater than 2 and less than N, where N is a positive integer greater than or equal to 3. Neck network: With the size of the reference feature map as a reference, the non-reference feature map is resized; the reference feature map and the resized non-reference feature map are spliced ​​in the channel dimension to obtain a spliced ​​feature map; the spliced ​​feature map is fused to obtain a fused feature map; the fused feature map is distributed to obtain a hierarchical distribution feature map of one or more preset distribution levels; the hierarchical distribution feature maps of one or more preset distribution levels are inversely resized and fused with the feature maps corresponding to the preset distribution levels to obtain fused features of one or more preset distribution levels, where the level index of the preset distribution level is greater than 2 and less than N; multiple detection features are obtained by fusing the fused features of one or more preset distribution levels with the feature map of level N; Head output layer: obtains lesion detection results based on multiple detection features.

2. The chest X-ray lesion detection method based on enhanced feature extraction and fusion according to claim 1, characterized in that: The step of resizing the non-reference feature map based on the size of the reference feature map includes: For a non-reference feature map whose size is larger than that of the reference feature map, the size transformation process is as follows: convolving the non-reference feature map with a first size transformation convolution based on a preset step size; For a non-reference feature map whose size is smaller than that of the reference feature map, the size transformation process is: bilinear interpolation processing and second size transformation convolution processing are performed on the non-reference feature map in sequence.

3. The chest X-ray lesion detection method based on enhanced feature extraction and fusion according to claim 2, characterized in that: The hierarchical distribution feature maps of the one or more preset distribution levels are subjected to inverse size transformation, including: For a preset distribution level whose feature map size is larger than the reference feature map size, the inverse size transformation process of the hierarchical distribution feature map of the preset distribution level is: performing bilinear interpolation processing and second size transformation convolution processing in sequence; For a preset distribution level whose feature map size is smaller than the reference feature map size, the inverse size transformation process of the hierarchical distribution feature map of the preset distribution level is as follows: performing convolution processing on the hierarchical distribution feature map of the preset distribution level using a first size transformation convolution based on a preset step size; No inverse size transformation is performed on the hierarchical distribution feature map of the preset distribution layer where the reference feature map is located.

4. A chest X-ray lesion detection method based on enhanced feature extraction and fusion as described in claim 1, 2 or 3, characterized in that: The fusing process of the spliced ​​feature maps to obtain the fused feature maps includes: sequentially performing convolution processing, batch normalization processing, and activation function processing on the spliced ​​feature maps.

5. The chest X-ray lesion detection method based on enhanced feature extraction and fusion according to claim 4, characterized in that: The feature extraction network at level 1 includes downsampling convolutions; The feature extraction network of level N includes downsampling convolution, C2F module and SPPF module connected in sequence; The feature extraction network at any level between level 1 and level N consists of downsampling convolution and C2F modules connected sequentially.

6. The chest X-ray lesion detection method based on enhanced feature extraction and fusion according to claim 5, characterized in that: The feature extraction network of at least one of levels 3 to level N-1 is connected to the feature extraction network of the next level through the G-CA module, and / or the feature extraction network of level N further includes a G-CA module located between the C2F module and the SPPF module; Among them, the G-CA module includes: The first branch is used to obtain the global channel attention weight of the input feature map of the G-CA module; The second branch is used to obtain the height-direction attention weight and width-direction attention weight of the input feature map of the G-CA module; The multiplication weighted operation unit is used to perform element-wise multiplication of the input feature map of the G-CA module with the global channel attention weight, height direction attention weight, and width direction attention weight.

7. The chest X-ray lesion detection method based on enhanced feature extraction and fusion according to claim 6, characterized in that: The first branch includes: The compression unit compresses the height and width information of each channel of the input feature map of the G-CA module into channel feature values, and the channel feature values ​​of all channels form a feature vector; A first branch convolution unit performs convolution processing on the feature vector to obtain a first branch feature map; The first branch activation function is processed, and the activation function is performed on the first branch feature map to obtain the weight of each channel. The weights of all channels constitute the global channel attention weight.

8. A chest X-ray lesion detection method based on enhanced feature extraction and fusion according to claim 6 or 7, characterized in that: The second branch includes: The global average pooling unit performs global average pooling on the input feature map of the G-CA module in the height direction and width direction respectively to obtain the height direction pooling feature and the width direction pooling feature; The second branch splicing unit splices the height-direction pooling features and the width-direction pooling features to obtain the splicing pooling features; The second branch convolution unit performs convolution processing on the spliced ​​pooling features to obtain the second branch feature map; The distribution unit performs batch normalization and activation function processing on the second branch feature map in sequence, and distributes the feature map after activation function processing into height feature submaps and width feature submaps; The height direction attention weight acquisition unit performs convolution and activation processing on the height feature sub-graph in sequence to obtain the height direction attention weight; The width-wise attention weight acquisition unit performs convolution and activation processing on the width feature sub-graph in sequence to obtain the width-wise attention weight.

9. A chest X-ray lesion detection device based on enhanced feature extraction and fusion, used to implement the chest X-ray lesion detection method based on enhanced feature extraction and fusion as described in any of claims 1-8, characterized in that: include: A chest X-ray acquisition module is used to acquire chest X-rays; The detection module inputs the chest X-ray into the trained target detection model to obtain the lesion detection results; The marking output module marks the lesion detection results on the chest X-ray and outputs the marked chest X-ray; The target detection model includes: Backbone network: includes N levels of feature extraction networks connected in sequence, each of which extracts feature maps of different scales of the chest X-ray. The N-1 feature maps obtained by the feature extraction networks at levels 2 to N are input into the neck network. The level index of the feature extraction network is 1 to N, and the N-1 feature maps include 1 reference feature map and N-2 non-reference feature maps. The level index of the reference feature map is greater than 2 and less than N, where N is a positive integer greater than or equal to 3. Neck network: With the size of the reference feature map as a reference, the non-reference feature map is resized; the reference feature map and the resized non-reference feature map are spliced ​​in the channel dimension to obtain a spliced ​​feature map; the spliced ​​feature map is fused to obtain a fused feature map; the fused feature map is distributed to obtain a hierarchical distribution feature map of one or more preset distribution levels; the hierarchical distribution feature maps of one or more preset distribution levels are inversely resized and fused with the feature maps corresponding to the preset distribution levels to obtain fused features of one or more preset distribution levels, where the level index of the preset distribution level is greater than 2 and less than N; multiple detection features are obtained by fusing the fused features of one or more preset distribution levels with the feature map of level N; Head output layer: obtains lesion detection results based on multiple detection features.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, A memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform a chest X-ray lesion detection method based on enhanced feature extraction and fusion as described in any one of claims 1 to 8.

Images