A chest multi-class disease detection method based on a lightweight neural network model

Abstract

The application discloses a chest multi-class disease detection method based on a light neural network model and belongs to the technical field of the cross of biomedicine and information technology. The Mobile Vit framework is improved through the designed FSConv module, N-EffS module and HFC module, a Mobile Vit-mix model is obtained, deep feature extraction can be realized while the network is light, local features and global features are fused, the MobileViT-Mix can better capture information in the image, and light and accurate detection is achieved; through the simple operation process, the user can obtain the detection result in a short time, helps the user to know the chest health condition in time, prevents and discovers the potential disease risk in advance, and greatly reduces the detection cost.

Description

Technical Field

[0001] This invention relates to the field of biomedical and information technology intersection, specifically to a method for detecting multiple chest diseases based on a lightweight neural network model. Background Technology

[0002] Traditional deep learning inference processes require significant computational resources, including high-performance computers, substantial storage, and bandwidth. Some researchers have developed convolutional neural network (CNN) models using chest X-ray images to detect lung diseases, achieving better results than baseline models. Others have proposed a confidence-based fusion method to combine MobileNetV2 and VGG16, achieving better classification accuracy on two publicly available datasets. Still others have used a VGG16-based architecture fine-tuned with additional custom layers, proving highly effective for classification. Still others have utilized a pre-trained VGG19 model with three CNN blocks for feature extraction and fully connected networks, demonstrating superior performance compared to existing work. However, this high cost and complex detection and classification processes significantly hinder the widespread adoption of smart healthcare, especially in the medical field. Furthermore, large-scale deep learning models for healthcare require the target population to travel to designated locations for testing, consuming considerable time and effort in awaiting results.

[0003] However, some challenges remain. The models mentioned above are all purely lightweight convolutional models and have not been integrated with the currently popular Transformer architecture. Therefore, developing models that maintain low computational and memory requirements while balancing performance and accuracy is a pressing issue. Firstly, lightweight models significantly reduce the number of model parameters by slightly reducing accuracy. Accuracy is crucial for diagnosing and treating patients in the medical field; therefore, it is necessary to reduce model complexity while maintaining accuracy. This is a further area for research and improvement needed for deploying lightweight models on mobile devices. Furthermore, mobile devices have limited computing power. Although current mobile devices' CPUs, GPUs, and NPUs provide powerful computing capabilities, they are still insufficient for AI models. Therefore, AI models need to be made even lighter to facilitate direct deployment on mobile devices.

[0004] By using lightweight neural network models to replace traditional deep learning models, not only can information mining of key features in medical image data be achieved, but it can also be deployed on mobile devices with a smaller number of parameters and less model complexity. This can greatly improve the efficiency of patients' medical treatment and enable timely detection and treatment of diseases even with limited resources. Summary of the Invention

[0005] The technical problem to be solved by this invention is: how to solve the problem of low efficiency in hospital treatment due to the large computing power of medical equipment in the existing technology. The invention provides a method for detecting multiple chest diseases based on a lightweight neural network model. It uses a module based on depthwise separable convolution to focus on and extract features of abnormal chest regions, and performs attention alignment according to their category labels. This makes the features of chest diseases correspond better with the labels, so as to achieve lightweight and accurate detection. It is mainly implemented in primary medical services such as rural areas.

[0006] The present invention solves the above-mentioned technical problems through the following technical solution, and the present invention includes the following steps:

[0007] S1: Model Building

[0008] Based on the basic architecture of the Mobile Vit framework, the Mobile Vit-mix model is constructed and trained by combining the designed FSConv module, N-EffS module and HFC module.

[0009] S2: Compilation Model Framework

[0010] The trained Mobile Vit-mix model is compiled into a model framework that can be deployed on a server;

[0011] S3: Deployed on a server

[0012] Deploy the compiled Mobile Vit-mix model from step S2 to the server;

[0013] S4: Chest Disease Detection and Classification

[0014] The patient uploads the chest X-ray medical images to the mobile device, which then transmits the images to the server via the network. The server uses the Mobile Vit-mix model to detect and classify chest diseases in the chest X-ray images and sends the results back to the mobile device.

[0015] Furthermore, in step S1, the Mobile Vit framework includes layer1, layer2, layer3, layer4, and layer5. In the Mobile Vit-mix model, the three repeated MV2 modules in layer2 are replaced with FSConv modules, and the Mobile Vit Blocks in layers3, 4, and 5 are replaced with Mobile Vit MixBlocks. The local feature extraction part of the Mobile Vit Mix Block is an N-EffS module, and the fusion part is an HFC module.

[0016] Furthermore, the FSConv module is used as an input encoding layer to divide the input image into a series of image blocks, and maps each image block to a fixed-dimensional vector representation through linear transformation, and performs feature extraction on the input image; the N-EffS module is used to extract local features of the image; the HFC module uses improved efficient residual connections to perform feature fusion operations.

[0017] Furthermore, the specific processing procedure of the FSConv module is as follows:

[0018] S101: The input feature map is processed by a 3×3 DW layer to obtain image features. These image features are then concatenated with the input feature map, as shown in the following expression:

[0019] F' = DW(I1)

[0020] Where I1 represents the image features output through layer1, i.e., the input feature map, and F' represents the image features obtained through processing by a 3×3 DW layer;

[0021] S102: Use two branches to combine hierarchical information. In the first branch, use the Squeeze and expand layers from the Fire module to extract local information, as shown in the following expression:

[0022] F1' = Fire(F')

[0023] Where F1' represents the feature obtained through the Fire module;

[0024] In the second branch, the group convolution module is used to divide the number of channels into groups, and the channel rearrangement module is used to shuffle the number of channels to obtain information exchange and feature fusion between different channels. The expression is as follows:

[0025] F2' = GC(F')

[0026] Where F2' represents the feature obtained through the second branch, and GC represents the feature obtained through the group convolution module and the channel rearrangement module;

[0027] S103: Perform ConCat fusion on the features obtained from the two branches, as shown in the following expression:

[0028] F”=F1'+F2'

[0029] Wherein, F” represents the feature obtained after fusion;

[0030] S104: Then use a 1×1 pointwise convolution to restore the number of channels;

[0031] S105: The input feature map and the output after 1×1 pointwise convolution are connected by a truncated branch to promote feature fusion at different levels and multi-scale representation. The expression is as follows:

[0032] I2 = I1 + F”

[0033] Where I2 represents the image features after passing through layer 2.

[0034] Furthermore, in step S102, the Fire module includes a Squeeze layer and an expand layer. The Squeeze convolutional layer with a 1×1 kernel achieves parameter sharing between channels and inputs it into the expand layer. The expand layer mixes 1×1 and 3×3 convolutional kernels, which can extract features of different scales simultaneously.

[0035] Furthermore, in step S103, in the group convolution module, the input feature map and convolution kernel are divided into g groups, and each group performs convolution operation independently; in the channel rearrangement module, the multiple channels of the group are reshaped into an m-row n-column matrix, and then transposed to obtain a new matrix. Then, it is flattened into a one-dimensional vector, which can promote information exchange and feature fusion between different channels and mix the information of different channels together.

[0036] Furthermore, the specific processing procedure of the N-EffS module is as follows:

[0037] S111: Three branches are used to form three paths; in the first branch, the standard convolution is decomposed into a 3×3 depthwise convolution and two 1×1 pointwise convolutions. The depthwise convolution processes spatial information in the length and width directions, and the pointwise convolution processes information across the channel directions. The calculation process is as follows:

[0038] F3'1 = PDP(I2)

[0039] Wherein, PDP is a combination of depthwise convolution and pointwise convolution, and F3'1 is the image feature obtained through the first branch;

[0040] In the second branch, 3×3 average pooling and 1×1 convolution are used for processing, as shown in the following expression:

[0041] F3'2=AG(I2)

[0042] Wherein, AG is a combined convolution formed by 3×3 average pooling and 1×1 convolution, and F3'2 is the image feature obtained through the second branch;

[0043] In the third branch, a 1×1 convolution is used directly, as shown in the following expression:

[0044] F3'3=Conv(I2)

[0045] Where Conv represents a 1×1 convolution, and F3'3 represents the image features obtained through the third branch;

[0046] S112: Perform ConCat fusion on the image features obtained from the three branches, as shown in the following expression:

[0047] F3”=F3'1+F3'2+F3'3

[0048] Among them, F3” represents the image features obtained by fusing the three branches;

[0049] S113: The feature maps fused by ConCat are rearranged through the channel rearrangement module, so that the channels in each group are mixed with the channels of other groups, realizing cross-group information exchange. The expression is as follows:

[0050] I3' = CS(F3”)

[0051] Where CS represents the channel rearrangement operation, and I'3 represents the image features after passing through the N-EffS module.

[0052] Furthermore, the specific processing procedure of the HFC module is as follows:

[0053] S121: Input the features processed by layer n into the N-EffS module, where n is 2, 3 or 4;

[0054] S122: The output I'3 of the N-EffS module is used as a new branch and fused with the output after global feature extraction using ConCat. The expression for the global feature extraction process is as follows:

[0055]

[0056] Where Q, K, and V represent query, key, and value features, respectively, d k Indicates the dimension of the embedding;

[0057] The expression for fusing the output I'3 of the N-EffS module with the ConCat fusion process of the output extracted from global features is as follows:

[0058] I3”1=Conv(GR(I3'))+I3'

[0059] Where I3”1 represents the result of combining local and global features, and GR represents the global features.

[0060] S123: In layer n, the input features are processed through the inverse residual structure consisting of PW layer, DW layer, and PW layer, as shown in the following expression:

[0061] I3”2=PDW(I 3MV )

[0062] Wherein, PDW represents the inverted residual structure composed of PW layer, DW layer, and PW layer, I 3MV The features in layer n after passing through the MV2 block;

[0063] S124: Perform ConCat fusion on I3”1 and I3”2 to obtain the output feature I3 of the layer after layer n, as shown in the following expression:

[0064] I3 = concat(I3”1,I3”2).

[0065] Furthermore, in step S2, the trained Mobile Vit-mix model is converted from a PyTorch model to the ONNX format, and then the Mobile Vit-mix model is compiled into a model framework that can be deployed on a server.

[0066] Compared with existing technologies, this invention has the following advantages: This method for detecting multiple chest diseases based on a lightweight neural network model improves the Mobile Vit framework by designing the FSConv, N-EffS, and HFC modules, resulting in the Mobile Vit-mix model. This model can achieve deep feature extraction while maintaining the network's lightweight nature. By fusing local and global features, MobileViT-Mix can better capture information in images, achieving lightweight and accurate detection. Furthermore, this invention's method allows users to obtain detection results quickly through a simple operation process, helping them understand their chest health status, prevent and detect potential disease risks early, and significantly reduce detection costs. Attached Figure Description

[0067] Figure 1 This is the overall framework diagram of the Mobile Vit-mix model in Embodiment 1 of the present invention;

[0068] Figure 2(a) is a schematic diagram of the group convolution module in Embodiment 1 of the present invention;

[0069] Figure 2(b) is a schematic diagram of the channel rearrangement module in Embodiment 1 of the present invention;

[0070] Figure 3 This is a schematic diagram of the Fire module in Embodiment 1 of the present invention;

[0071] Figure 4 This is a schematic diagram of the FSConv module in Embodiment 1 of the present invention;

[0072] Figure 5 This is a schematic diagram of the N-EffS module in Embodiment 1 of the present invention;

[0073] Figure 6 This is a diagram of the Vision Transformer framework in Embodiment 1 of the present invention;

[0074] Figure 7 This is an architecture diagram of the chest multi-disease detection method based on a lightweight neural network model in Embodiment 1 of the present invention;

[0075] Figure 8 This is a schematic diagram of the overall process of the method for detecting multiple chest diseases based on a lightweight neural network model in Embodiment 1 of the present invention;

[0076] Figure 9 This is a schematic diagram illustrating the principle of the chest multi-disease detection method based on a lightweight neural network model in Embodiment 1 of the present invention;

[0077] Figure 10a This is the mobile software user login interface corresponding to the chest multi-disease detection method based on a lightweight neural network model in Embodiment 2 of the present invention;

[0078] Figure 10b This is the user information input interface of the mobile software corresponding to the chest multi-disease detection method based on a lightweight neural network model in Embodiment 2 of the present invention;

[0079] Figure 10c This is the user manual interface of the mobile software corresponding to the chest multi-disease detection method based on a lightweight neural network model in Embodiment 2 of the present invention;

[0080] Figure 10d This is the mobile software result viewing interface corresponding to the chest multi-disease detection method based on a lightweight neural network model in Embodiment 2 of the present invention. Detailed Implementation

[0081] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0082] Example 1

[0083] This embodiment provides a technical solution: a method for detecting multiple chest diseases based on a lightweight neural network model, comprising the following steps:

[0084] Step 1: Model Building

[0085] Based on the basic architecture of the Mobile Vit framework, the Mobile Vit-mix model is constructed and trained by combining the designed FSConv module, N-EffS module and HFC module.

[0086] In this step, the overall framework of the Mobile Vit-mix model is as follows: Figure 1 As shown, specifically, it consists of three types of modules: FSConv (compressed dilated convolution and channel re-following) module, N-EffS (novel efficient channel re-following convolution) module, and HFC (efficient fusion connection) module. Figure 1 As shown, Mobile Vit-mix adopts the basic structure of the Mobile Vit framework. First, the three-layer repeated MV2 (Inverted Residual Block) structure in layer 2 is replaced with the FSConv module designed in this invention; second, the local feature extraction part in the Mobile Vit Block in layers 3, 4, and 5 is replaced with the N-EffS module; finally, the Fusion part in the Mobile Vit Block is replaced with the HFC module, which can effectively reduce the number of parameters.

[0087] More specifically, the FSConv module mainly serves as the input encoding layer, dividing the input image into a series of image patches. Through linear transformation, each image patch is mapped to a fixed-dimensional vector representation, and features are extracted from the input image to obtain a low-level feature representation of each image patch.

[0088] More specifically, the N-EffS module extracts local features from the image, introducing more local feature information, enabling the model to better perceive fine-grained features in the input image.

[0089] More specifically, the HFC module uses improved, efficient residual connections to perform feature fusion operations, which helps optimize deeper layers of the architecture.

[0090] In the Mobile Vit-mix model, the number of output feature map channels in each layer is out1 to out5, where out1 = 16, out2 = 32, out3 = 48, out4 = 64, and out5 = 80. The Mobile Vit-mix design reduces the number of repeated stackings in the second layer compared to Mobile Vit, replacing it with a custom-designed module (FSConv module), thus reducing parameters and computational complexity. It integrates two lightweight core architectures that currently offer excellent classification performance. Furthermore, it employs a more lightweight and efficient local feature extraction module (N-EffS module) in the local feature extraction part. The fusion part (HFC module) uses an improved, highly efficient residual connection to fuse the original input, the features extracted from local features, and the features extracted from global features.

[0091] The design of Mobile Vit-mix differs from other works in that: (1) it employs more skip connections, (2) it consists of a shallower network, and (3) it integrates current high-quality lightweight modules. These design considerations reduce the number of parameters and computational cost in the network. To maintain training accuracy and classification performance, the following network performance enhancements were implemented while developing the network modules: more complex connection methods were used to add more feature extraction modules, enhancing the ability to extract image features; input features, global features, and inputs from other locations were fused to achieve multi-scale information fusion and enhance information interaction.

[0092] The following provides a more detailed explanation of the FSConv module, N-EffS module, and HFC module.

[0093] FSConv module

[0094] In this embodiment, the structures of the group convolution module and the channel rearrangement module (used for channel shuffle operation) are shown in Figures 2(a) and (b). First, group convolution can effectively reduce the network size, making the network more lightweight. The number of parameters and computational cost of group convolution compared to ordinary convolution are as follows:

[0095] Ordinary convolution:

[0096]

[0097] Grouped convolution:

[0098]

[0099] From the above formula, it can be seen that in ordinary convolution, each convolutional kernel performs calculations at every position of the input feature map. Therefore, the number of parameters and the computational cost are directly proportional to the number and size of the convolutional kernels. Let the number of input channels be M, each channel correspond to one convolution, each convolutional kernel be K×K in size, the feature map size be H×W, and the number of output channels be N. In grouped convolution, the input feature map and the convolutional kernel are divided into g groups, and each group performs the convolution operation independently. This means that the number of parameters and the computational cost of each group are only 1 / g of those of ordinary convolution. Therefore, the number of parameters and the computational cost of ordinary convolution are g times that of grouped convolution, that is, the number of parameters and the computational cost of ordinary convolution are multiples of the number of groups. Therefore, grouped convolution can reduce the number of parameters and the computational cost compared to ordinary convolution.

[0100] Secondly, the Channel Shuffle process can be viewed as reshaping the grouped channels into an m x n matrix, then performing a transpose operation to obtain a new matrix, and finally flattening it into a one-dimensional vector. This operation promotes information exchange and feature fusion between different channels, mixing information from different channels together, thereby increasing the interaction between features and the transmission of information across channels, which helps to extract richer and more expressive features. Furthermore, by rearranging the channels, redundant parameters and computational operations can be combined, reducing the number of parameters and computational load in the model, thus improving the model's efficiency.

[0101] like Figure 3 The diagram shows the structure of the Fire module. The main idea is to decompose a convolutional layer into a Squeeze layer and an expand layer. A Squeeze convolutional layer (1×1 kernel) enables parameter sharing between channels, which is then input into an expand layer. The expand layer mixes 1×1 and 3×3 kernels, allowing for the simultaneous extraction of features at different scales. This multi-scale feature representation helps the model perceive targets and structures of different sizes, improving its expressive and generalization abilities. Both layers are equipped with ReLU activation layers, significantly reducing the computational cost of the convolutional layers and achieving a very compact model structure. The squeeze layer contains S 1×1 kernels. The expand layer contains S1 and S2 1×1 kernels and 3×3 kernels, where S < (S1 + S2).

[0102] By leveraging the advantages and characteristics of the three modules mentioned above, innovations and improvements were made to obtain the FSConv module, such as... Figure 4As shown, a 3×3 deep convolutional (DW) layer is used at the beginning to connect it to the input feature map. It has a large receptive field and can capture a wider range of contextual information, thereby extracting richer and more expressive local features from the input data, as shown in Equation (3):

[0103] F'=DW(I1) (3)

[0104] Where I1 represents the image features output through layer 1, i.e., the input feature map, and F' represents the image features after passing through a 3×3 depthwise convolutional layer (DW).

[0105] Previous excellent networks have demonstrated the importance of semantic context, and multi-scale information fusion is a suitable method. Therefore, we use a two-branch structure to jointly combine hierarchical information. To reduce computational cost, we use the Squeeze and expand layers from the Fire module in the first branch to extract local information, as shown in Equation (4):

[0106] F1' = Fire(F') (4)

[0107] Here, F1' represents the feature obtained through the Fire module.

[0108] Furthermore, in the second branch, the group convolution module is used to divide the number of channels into groups, and the number of channels is shuffled by the channel rearrangement module to obtain information exchange and feature fusion between different channels, as shown in the following equation (5):

[0109] F2'=GC(F') (5)

[0110] Where F2' represents the feature obtained through the second branch, and GC represents the feature obtained through the group convolution module and the channel rearrangement module.

[0111] We perform ConCat fusion on the two branches, and then use 1×1 pointwise convolution to recover the number of channels as shown in Equation (6):

[0112] F”=F1'+F2' (6)

[0113] Finally, we perform truncated branch connections on the input and the output after the 1×1 pointwise convolution to promote feature fusion and multi-scale representation at different levels, and to effectively increase the depth and complexity of the network without introducing too many additional parameters and computational costs, as shown in Equation 7:

[0114] I2=I1+F” (7)

[0115] Where I2 represents the image features after passing through layer 2.

[0116] N-EffS module

[0117] Numerous studies have shown that cascading can improve network performance and speed to some extent. Cascading increases network depth and complexity, thereby enhancing its representational capabilities. Deeper networks can learn more abstract, higher-level feature representations, helping to improve the model's understanding and discrimination of complex data. Furthermore, introducing mechanisms such as skip connections or residual connections promotes information flow and gradient propagation, mitigating the vanishing gradient problem and accelerating training while improving model convergence. However, cascading can also increase computational complexity and the number of parameters, leading to larger and more time-consuming models. Therefore, convolutions with fewer parameters or lightweight convolutions are often used in each pathway.

[0118] The structure of the N-EffS module proposed in this invention is as follows: Figure 5 As shown in the figure, the N-EffS module is a three-path system composed of lightweight convolutions. The first branch adopts an inverse residual structure based on depthwise separable convolutions, which decomposes the standard convolution into a 3×3 depthwise convolution and two 1×1 pointwise convolutions. The depthwise convolution processes spatial information in the length and width directions, and the pointwise convolutions process information across channels. According to formulas (8) and (9), the number of input channels is M and the size is D. F ×D F The number of output channels is N, and the kernel size is D. k ×D k Its structure can reduce the number of parameters and computational load in network transmission, and its representation process is Equation 10.

[0119]

[0120] F3'1=PDP(I2) (10)

[0121] Wherein, PDP is a combination of depthwise convolution and pointwise convolution, and F3'1 is the image feature obtained through the first branch.

[0122] The second branch simplifies the compression process and reduces computation by using 3×3 average pooling and 1×1 convolution. The process is as follows:

[0123] F3'2=AG(I2) (11)

[0124] Where AG is a combined convolution formed by 3×3 average pooling and 1×1 convolution, and F3'2 is the image feature obtained through the second branch.

[0125] The third branch directly uses a 1×1 convolution, the process is as follows:

[0126] F3'3=Conv(I2) (12)

[0127] F3'3 represents the image features obtained through the third branch.

[0128] Then, merge the three branches using ConCat:

[0129] F3”=F3'1+F3'2+F3'3 (13)

[0130] The N-EffS module is a series of modules integrated with lightweight convolutions and cascaded operations, used to extract low-level features and applied to the local feature extraction locations in this invention. N-EffS runs in layer 3 and every subsequent stage, and these three branches together constitute the architecture of N-EffS. The configuration of N-EffS balances the channels of the input and output feature maps, avoids group convolutions, and reduces computational cost.

[0131] Finally, the structure output from the three branches is fused using ConCat, and then the input feature maps are rearranged using Channel Shuffle, so that the channels in each group are mixed with the channels in other groups, thereby realizing cross-group information exchange and effectively reducing the increase in the number of parameters and computational complexity caused by multiple branches, and improving the performance of the model to a certain extent, as shown in Equation 14.

[0132] I3'=CS(F3”) (14)

[0133] Where CS represents the channel rearrangement operation, and I3' represents the image features after passing through the N-EffS module.

[0134] HFC module

[0135] Because the basic network adopts the Mobile Vit architecture, it is a lightweight, general-purpose vision Transformer for mobile devices. Mobile Vit provides a different perspective for global information processing using Transformers. The global feature extraction part uses VIT (Vision Transformer), employing the Transformer network architecture to further extract image features.

[0136] like Figure 6As shown, the Vision Transformer (ViT) framework includes Channel Attention and Spatial Attention. Channel Attention typically uses global pooling operations (such as average pooling or max pooling) to aggregate each channel, resulting in a channel-level representation. Then, attention weights are calculated for each channel through one or more fully connected or convolutional layers. These weights can adjust the feature map representation based on the importance of the channel. Spatial Attention typically uses a multi-head self-attention mechanism to model the spatial dimension. Each attention head can learn different spatial relationships and generate corresponding attention weights. These two attention mechanisms complement each other, helping the ViT model better understand the content in the image.

[0137] However, the biggest problem with Transformer is its high computational cost, because the amount of information in an image is far greater than that in text in NLP. The self-attention operation leads to a complexity of N squared, which is unacceptable for lightweight networks. Therefore, it is important to reduce the computational cost.

[0138] Generally, the width of shallow layers in a network is smaller than that of deep layers. Deeper layers can collect more channel information, making each channel more significant. Therefore, in Mobile Vit, the HFC module is used for efficient fusion and connection operations in the local and global feature extraction and fusion parts to achieve deep feature extraction. First, the features from layer 2 are input into the local feature extraction part (N-EffS module). The output I3' of the N-EffS module is then used as a new branch and concatenated with the output after global feature extraction using ConCat. The global feature extraction process is as follows:

[0139]

[0140] Where Q, K, and V represent query, key, and value features, respectively, d k Indicates the dimension of the embedding.

[0141] The process of fusing the output I3' of the N-EffS module with the output after global feature extraction using ConCat is as follows:

[0142] I3"1=Conv(GR(I3'))+I3' (16)

[0143] Where I3”1 represents the result of combining local and global features, and GR represents global features.

[0144] The features after passing through the MV2 block in layer 2 are then input into the inverse residual structure formed by PW, DW, and PW lightweight convolutional layers, as shown in equation (17):

[0145] I3”2=PDW(I 3MV (17)

[0146] Among them, I 3MV These are the features in layer 2 after passing through the MV2 block.

[0147] After obtaining the result I3”2 after passing through PW, DW, and PW lightweight convolutional layers, I3”1 and I3”2 are fused to obtain the final output features. The process is shown in the following equation (18), specifically as follows: Figure 1 The MobileViT-Mix block is shown in the figure.

[0148] I3=concat(I3”1,I3”2) (18)

[0149] This design allows for deep feature extraction while maintaining a lightweight network. By fusing local and global features, MobileViT-Mix can better capture information from images.

[0150] The calculation process for Layer 4 and Layer 5 is the same as that for Layer 3; please refer to the above explanation.

[0151] Step 2: Compile the model framework

[0152] The trained Mobile Vit-mix model is converted from a PyTorch model to the ONNX format, and then the Mobile Vit-mix model is compiled into a model framework that can be deployed on a server.

[0153] In this step, after successfully training the Mobile Vit-mix model network using a large amount of data (pre-collected hospital chest X-ray medical image data and a related database), the model is converted from a PyTorch model to ONNX format (Open Neural Network Exchange), and then the ONNX format is converted to models supported by ncnn, MACE, MNN, and TNN. ONNX format is a standard for representing deep learning models, enabling model transfer between different frameworks.

[0154] It should be noted that ONNX is used to convert PyTorch to ncnn, MACE, MNN, or TNN, and the mobile inference framework library is compiled according to iOS or Android. To further improve the inference speed of the model, the inference framework can be used to fuse convolutional and activation layers to reduce the I / O overhead between network layers; float32 can also be quantized with int8, which theoretically can reduce the model size by 4 times and improve the inference speed by 2 to 3 times without reducing the model's accuracy.

[0155] Step 3: Deploy on the server

[0156] Deploy the compiled Mobile Vit-mix model from step two to the server;

[0157] Step Four: Chest Disease Detection and Classification

[0158] The patient uploads the chest X-ray medical images to the mobile device, which then transmits the images to the server via the network. The server uses the Mobile Vit-mix model to detect and classify chest diseases in the chest X-ray images and sends the results back to the mobile device.

[0159] Example 2

[0160] This embodiment provides users with dedicated mobile software for the chest multi-disease detection method described in Embodiment 1. Using this software, real-time detection and classification of chest X-ray medical images uploaded by patients can be performed. This allows patients to obtain results of their chest health status more conveniently and quickly. The specific process is as follows:

[0161] Data transmission

[0162] After professional chest X-rays are taken at the hospital, the qualified images are transmitted to the server via 5G network after being judged by the hospital staff and tested by the software, so that the server can detect and classify them later.

[0163] Server identification

[0164] The server receives over 90% of the data uploaded by hospitals, which can be used by a lightweight neural network algorithm (Mobile Vit-mix model) deployed on the server to detect and classify various chest diseases, resulting in relatively fast and accurate classifications. The remaining unsuitable images need to be returned to senior physicians for further review to ensure accurate detection and classification of the patient's images.

[0165] Output Results

[0166] The chest X-ray data taken by the hospital is detected and classified by a lightweight neural network algorithm on the server. Then, using the mobile network, the accurate results are returned to the patient's mobile device so that the patient can view the chest X-ray results in real time without having to go to the hospital, thus enabling timely prevention and treatment of the disease.

[0167] In summary, the lightweight neural network-based method for detecting multiple chest diseases described above improves the MobileVit framework by designing the FSConv, N-EffS, and HFC modules, resulting in the MobileVit-mix model. This model maintains the network's lightweight nature while achieving deep feature extraction. By fusing local and global features, MobileViT-Mix can better capture information from images, achieving lightweight and accurate detection. The method, through its simple operation, allows users to obtain detection results quickly, helping them understand their chest health status, prevent and detect potential disease risks early, and significantly reduce detection costs.

[0168] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for detecting multiple chest diseases based on a lightweight neural network model, characterized in that, Includes the following steps: S1: Model Building Based on the basic architecture of the Mobile Vit framework, the Mobile Vit-mix model is constructed and trained by combining the designed FSConv module, N-EffS module and HFC module. S2: Compilation Model Framework The trained Mobile Vit-mix model is compiled into a model framework that can be deployed on a server; S3: Deployed on a server Deploy the compiled Mobile Vit-mix model from step S2 to the server; S4: Chest Disease Detection and Classification The patient uploads the chest X-ray medical images to the mobile device, which then transmits the chest X-ray medical images to the server via the network. The server uses the Mobile Vit-mix model to detect and classify chest diseases in the chest X-ray medical images and sends the results back to the mobile device. In step S1, the Mobile Vit framework includes layer1, layer2, layer3, layer4 and layer5. In the Mobile Vit-mix model, the three repeated MV2 modules in layer2 are replaced with FSConv modules, and the Mobile Vit Blocks in layers3, 4 and 5 are replaced with Mobile Vit Mix Blocks. The local feature extraction part of the Mobile VitMix Block is an N-EffS module, and the fusion part is an HFC module. The specific processing procedure of the FSConv module is as follows: S101: Input feature map passed The image features are obtained after processing by the DW layer, and the expression is as follows: ； in, This represents the image features output through layer 1, i.e., the input feature map. Indicates passage The DW layer processes the image features, and then the image features are concatenated with the input feature map. S102: Use two branches to combine hierarchical information. In the first branch, use the Squeeze and expand layers from the Fire module to extract local information, as shown in the following expression: ； in, This represents the features obtained through the Fire module; In the second branch, the group convolution module is used to divide the number of channels into groups, and the channel rearrangement module is used to shuffle the number of channels to obtain information exchange and feature fusion between different channels. The expression is as follows: ； in, This represents the features obtained through the second branch. This indicates the use of group convolution modules and channel rearrangement modules; S103: Perform ConCat fusion on the features obtained from the two branches, as shown in the following expression: ； in, The features obtained after fusion; S104: Then use The number of channels recovered by pointwise convolution; S105: Input feature map and The output after pointwise convolution is truncated into branches to promote feature fusion at different levels and multi-scale representation, as shown in the following expression: ； in, This represents the image features after passing through layer 2; The specific processing procedure of the N-EffS module is as follows: S111: Three branches are used to form three pathways; in the first branch, the standard convolution is decomposed into one... Depth convolution and two Pointwise convolution and depthwise convolution process each handle spatial information in the length and width directions, while pointwise convolution processes information across channel directions. The calculation process is as follows: ； in, This is a combination of depthwise convolution and pointwise convolution. The image features obtained through the first branch; In the second branch, using Average pooling and The convolution is processed as follows: ； in, for Average pooling and Combination convolutions formed by convolutions, The image features obtained through the second branch; In the third branch, we directly adopt... The convolution is processed as follows: ； in, express convolution, Image features obtained through the third branch; S112: Perform ConCat fusion on the image features obtained from the three branches, as shown in the following expression: ； in, The image features obtained by fusing the three branches; S113: The feature maps fused by ConCat are rearranged through the channel rearrangement module, so that the channels in each group are mixed with the channels of other groups, realizing cross-group information exchange. The expression is as follows: ； in, This is a channel rearrangement operation. The image features are those obtained after passing through the N-EffS module; The specific processing procedure of the HFC module is as follows: S121: Input the features processed by layer n into the N-EffS module, where n is 2, 3 or 4; S122: Convert the output of the N-EffS module As a new branch, it is fused with the output after global feature extraction using ConCat, where the global feature extraction process is expressed as follows: ； in, , and Indicates query, key, and value characteristics. Indicates the dimension of the embedding; Output of N-EffS module The expression for the ConCat fusion process with the output after global feature extraction is as follows: ； in, This represents the result of combining local and global features. Represents global features; S123: In layer n, the input features are processed through the inverse residual structure consisting of PW layer, DW layer, and PW layer, as shown in the following expression: ； in, This represents the inverted residual structure composed of PW layer, DW layer, and PW layer. The features in layer n after passing through the MV2 block; S124: Yes and Perform ConCat fusion to obtain the output features of the next layer after layer n. The expression is as follows: 。 2. The method for detecting multiple chest diseases based on a lightweight neural network model according to claim 1, characterized in that, The FSConv module is used as the input encoding layer, which divides the input image into a series of image blocks, maps each image block to a fixed-dimensional vector representation through linear transformation, and extracts features from the input image; the N-EffS module is used to extract local features from the image; the HFC module uses improved efficient residual connections to perform feature fusion operations.

3. The method for detecting multiple chest diseases based on a lightweight neural network model according to claim 1, characterized in that, In step S102, the Fire module includes a Squeeze layer and an expand layer, through... The Squeeze convolutional layer enables parameter sharing between channels, which is then input into the expand layer, where the parameters are mixed. and The convolutional kernel can extract features at different scales simultaneously.

4. The method for detecting multiple chest diseases based on a lightweight neural network model according to claim 3, characterized in that, In step S102, in the group convolution module, the input feature map and convolution kernel are divided into g groups, and each group performs convolution operation independently; in the channel rearrangement module, the multiple channels of the group are reshaped into an m-row n-column matrix, and then transposed to obtain a new matrix. Then, it is flattened into a one-dimensional vector, which can promote information exchange and feature fusion between different channels and mix the information of different channels together.

5. The method for detecting multiple chest diseases based on a lightweight neural network model according to claim 1, characterized in that, In step S2, the trained Mobile Vit-mix model is converted from a PyTorch model to the ONNX format, and then the Mobile Vit-mix model is compiled into a model framework that can be deployed on a server.

Images