A Transformer-based segmentation method and related equipment for chest X-ray lesions of high altitude pulmonary edema

By using a Transformer-based U-shaped fully convolutional neural network segmentation model, combined with the feature extraction and lesion segmentation techniques of Swin Transformer and DenseNet, the problem of low segmentation accuracy of lesions in high-altitude pulmonary edema was solved, achieving more accurate lesion identification and diagnosis.

CN117218135BActive Publication Date: 2024-11-19BEIJING SHIJITAN HOSPITAL CAPITAL MEDICAL UNIVERSITY +3
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311191474.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-09-15
Publication Date
2024-11-19
Estimated Expiration
2043-09-15

AI Technical Summary

Technical Problem

Existing methods for segmenting pulmonary edema lesions in high-altitude areas suffer from low segmentation accuracy and an inability to accurately identify lesion regions. Traditional convolutional neural network models ignore the correlation between different spatial locations.

Method used

A Transformer-based method for segmenting lesions on chest X-rays of high-altitude pulmonary edema was adopted. By combining the local and global multi-level window attention mechanism of Swing Transformer and the dense connections of DenseNet, a U-shaped fully convolutional neural network segmentation model was constructed. Feature extraction and lesion segmentation were performed through Skip Connection layers.

Benefits of technology

It improves the segmentation accuracy and recognition capability of lesion areas, better captures the weight of information from different locations, enhances feature restoration capability, and provides more accurate diagnostic results and treatment suggestions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117218135B_ABST
    Figure CN117218135B_ABST
Patent Text Reader

Abstract

The present invention provides a Transformer-based segmentation method and related equipment for chest X-ray lesions of high altitude pulmonary edema, which are applied in the field of data processing technology. The present application obtains training samples and target images; preprocesses the training samples to generate labeled training samples; constructs a U-shaped fully convolutional neural network segmentation model; trains the neural network segmentation model with the labeled training samples to generate a trained neural network segmentation model; inputs the target image into the trained segmentation model for processing, and outputs the target case; and adds the target image to the training sample set for updating the neural network segmentation model. The use of the Transformer network to extract features and segment lesions from chest X-ray images can effectively solve the problems of low segmentation accuracy and inability to accurately identify lesion areas in the prior art.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a Transformer-based segmentation method for chest X-ray lesions of high altitude pulmonary edema and related equipment. Background Art

[0002] High-altitude pulmonary edema (HAPE) is a common type of altitude sickness. It is a disease caused by abnormal adaptive response of people in the hypoxic environment of the plateau. In the hypoxic environment of the plateau, the body's adaptive response can lead to symptoms such as pulmonary microcirculation disorder, increased alveolar permeability, and increased pulmonary capillary permeability, thereby causing pulmonary edema, which is called high-altitude pulmonary edema. Early diagnosis and treatment of HAPE are very important, because if not treated in time, it will cause the patient's condition to worsen, and even endanger life in severe cases. At present, the medical community mainly uses medical imaging technology to diagnose and treat HAPE. However, traditional medical imaging technology often requires doctors to perform manual segmentation and diagnosis, which is not only time-consuming and labor-intensive, but also prone to subjective errors, affecting the diagnosis and treatment effects.

[0003] Therefore, an automated HAPE lesion segmentation method based on computer vision technology has emerged, which can effectively solve problems such as manual segmentation and subjective errors and improve the diagnosis and treatment effects. The current automated HAPE lesion segmentation method is mainly based on convolutional neural network methods, which achieve automated segmentation of HAPE lesions through deep learning and image segmentation of medical images. However, traditional convolutional neural network models often ignore the correlation between different spatial positions during training.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present application is to provide a Transformer-based segmentation method and related equipment for chest X-ray lesions of high altitude pulmonary edema. The Transformer network is used to extract features and segment lesions from chest X-ray images, which can effectively solve the problems of low segmentation accuracy and inability to accurately identify lesion areas in the prior art. In addition, Swin Transformer introduces a local and global multi-level window attention mechanism, which can better capture the weights of information at different locations and assign different weights to lesion areas and non-lesion areas. DenseNet establishes dense connections between different convolutional layers, which can better utilize the feature information of previous layers and improve the ability to restore features.

[0006] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present invention.

[0007] According to one aspect of the present application, a Transformer-based segmentation method for high-altitude pulmonary edema chest X-ray lesions is provided, including obtaining training samples and target images; preprocessing the training samples to generate labeled training samples; constructing a U-shaped fully convolutional neural network segmentation model, wherein the U-shaped fully convolutional neural network segmentation model includes an upsampling model and a downsampling model, the upsampling model and the downsampling model are connected via a Skip Connection layer, the downsampling model is a Swin Transformer deep learning model, and the upsampling model is a DenseNet convolutional neural network model; the neural network segmentation model is trained with the labeled training samples to generate a trained neural network segmentation model; the target image is input into the trained segmentation model for processing, and a target case is output, wherein the target case includes the segmentation result of the high-altitude pulmonary edema lesion; the target image is added to the training sample set for updating of the neural network segmentation model.

[0008] In one embodiment of the present application, the upsampling model and the downsampling model are connected through a SkipConnection connection layer, including: the output end of the Swin Transformer deep learning model is connected to the input end of the DenseNet convolutional neural network model through the Skip Connection connection layer and the bottom feature layer; the output end of the DenseNet convolutional neural network model is size-enlarged by upsampling to generate preprocessed model features; the preprocessed model features are connected to the Skip Connection connection layer.

[0009] In one embodiment of the present application, the preprocessing of the training samples to generate labeled training samples includes: performing data enhancement processing on the training samples to generate enhanced training samples; processing the enhanced training samples based on preset division rules to generate lesion areas and non-lesion areas.

[0010] In one embodiment of the present application, the preprocessing of the training samples to generate labeled training samples also includes: performing normalization preprocessing on the training samples to generate training samples of target brightness; obtaining preset classification rules, wherein the preset classification rules include segmentation data of several different lesion types; and processing the training samples of the target brightness based on the preset classification rules to generate classified training samples.

[0011] In one embodiment of the present application, the target image is input into the trained segmentation model for processing and the target case is output, including: gray-scaling the target image to generate a target color image; normalizing the target color image to generate a color image of a preset size; slicing the color image to sequentially generate three consecutive slice images; processing the slice image to generate a three-channel image; and inputting the three-channel image into the trained segmentation model for processing to output the target case.

[0012] In one embodiment of the present application, the slicing process of the color image to sequentially generate three continuous slice images includes: processing the color image based on a preset color division rule to generate a first color image containing different classification numbers; dividing the first color image based on the classification number to generate a number of initial slice images; preprocessing the initial slice image based on a target segmentation model to generate a preprocessed slice image; processing the preprocessed slice image based on a preset bilinear interpolation to generate a number of slice images of preset sizes.

[0013] In one embodiment of the present application, the slicing processing of the color image to sequentially generate three continuous slice images also includes: obtaining a preset size value to process the color image to generate a panoramic image of a preset size; performing multi-layer slicing processing on the panoramic image to generate multiple slice image groups of different depth levels; processing the slice image group based on time sequence or attribute information to generate several slice images.

[0014] Another aspect of the present application is a Transformer-based segmentation device for high-altitude pulmonary edema chest X-ray lesions, characterized in that the device includes: an acquisition module for acquiring training samples and a target image; a processing module for preprocessing the training samples to generate labeled training samples; constructing a U-shaped fully convolutional neural network segmentation model, wherein the U-shaped fully convolutional neural network segmentation model includes an upsampling model and a downsampling model, the upsampling model and the downsampling model are connected through a Skip Connection layer, the downsampling model is a Swin Transformer deep learning model, and the upsampling model is a DenseNet convolutional neural network model; the neural network segmentation model is trained with the labeled training samples to generate a trained neural network segmentation model; the target image is input into the trained segmentation model for processing, and a target case is output, wherein the target case includes the segmentation result of the high-altitude pulmonary edema lesion; the target image is added to the training sample set for updating of the neural network segmentation model.

[0015] According to another aspect of the present application, an electronic device is characterized in that it includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the above-mentioned Transformer-based high altitude pulmonary edema chest X-ray lesion segmentation method by executing the executable instructions.

[0016] According to another aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned Transformer-based high-altitude pulmonary edema chest X-ray lesion segmentation method is implemented.

[0017] According to another aspect of the present application, a computer program product is provided, including a computer program, characterized in that when the computer program is executed by a processor, the above-mentioned Transformer-based high altitude pulmonary edema chest X-ray lesion segmentation method is implemented.

[0018] The present application provides a Transformer-based segmentation method for high-altitude pulmonary edema chest X-ray lesions, which includes obtaining training samples and target images; preprocessing the training samples to generate labeled training samples; constructing a U-shaped fully convolutional neural network segmentation model, wherein the U-shaped fully convolutional neural network segmentation model includes an upsampling model and a downsampling model, the upsampling model and the downsampling model are connected via a Skip Connection layer, the downsampling model is a Swin Transformer deep learning model, and the upsampling model is a DenseNet convolutional neural network model; the neural network segmentation model is trained with the labeled training samples to generate a trained neural network segmentation model; the target image is input into the trained segmentation model for processing, and a target case is output, wherein the target case includes the segmentation result of the high-altitude pulmonary edema lesions; the target image is added to the training sample set for updating of the neural network segmentation model. The Transformer network is used to extract features and segment lesions from chest X-ray images, which can effectively solve the problems of low segmentation accuracy and inability to accurately identify lesion areas in existing technologies. In addition, Swin Transformer introduces a local and global multi-level window attention mechanism, which can better capture the weights of information at different locations and assign different weights to lesion areas and non-lesion areas. DenseNet establishes dense connections between different convolutional layers, which can better utilize the feature information of previous layers and improve the ability to restore features.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a method for segmenting high altitude pulmonary edema chest X-ray lesions based on Transformer provided in one embodiment of the present application is shown;

[0021] Figure 2 A schematic structural diagram of a Transformer-based segmentation device for high altitude pulmonary edema chest X-ray lesions provided in one embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram of a storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0025] Combine the following Figure 1 To describe the segmentation method of high altitude pulmonary edema chest radiograph lesions based on Transformer according to the exemplary embodiment of the present application. It should be noted that the following application scenarios are only shown to facilitate understanding of the spirit and principle of the present application, and the embodiments of the present application are not limited in this regard. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0026] In one embodiment, the present application also proposes a Transformer-based segmentation method for high altitude pulmonary edema chest X-ray lesions. Figure 1 The flowchart of a method for segmenting chest radiographs of HAPE lesions based on Transformer according to an embodiment of the present application is schematically shown. Figure 1 As shown, the method is applied to a server, comprising:

[0027] S101, obtaining training samples and target images.

[0028] In one implementation, chest X-ray image data for training, verification and testing are obtained as training samples, and chest X-ray image data of a patient to be tested are obtained as target images.

[0029] S102: pre-process the training samples to generate labeled training samples.

[0030] In one embodiment, data enhancement processing is performed on the training samples to generate enhanced training samples, and the enhanced training samples are processed based on preset division rules to generate lesion areas and non-lesion areas. Data enhancement is an indispensable part of deep learning training. The present invention adopts the data enhancement method of Cutout, which performs independent and repeated uniformly distributed sampling on the resolution of the input image, and then selects different scales with equal probability, and finally performs cropping or random transformation. This method erases some key point information and even brings some confusing information, forcing the model to infer the erased information and identify confusing information through context, which greatly improves the robustness of the model and allows the model to have very good generalization ability on unknown data sets.

[0031] In another embodiment, the chest X-ray image data is subjected to noise reduction processing, artificial and natural noise in the image that hinders the determination of the lesion location is removed, and the noise-reduced image is deformed and rotated to expand the data set. In addition, each ultrasound grayscale image in the data set is padded and standardized respectively, and the training set is augmented, wherein padding refers to filling fixed pixel values ​​around the image so that the size of the image output after the convolutional neural network training is consistent with the size of the original image, and standardization refers to subtracting the mean of each dimension of the data and dividing it by the standard deviation. The data augmentation method includes image translation, rotation or affine transformation.

[0032] In another embodiment, the training samples are normalized and preprocessed to generate training samples of target brightness. The normalized preprocessing specifically includes: (1) obtaining a three-dimensional image grayscale matrix from the original sequence image; (2) normalizing the pixel grayscale values ​​of the three-dimensional image grayscale matrix between 0 and 1; (3) normalizing the three-dimensional pixel spacing of the image normalized in step 2 to a preset value. Normalized preprocessing of chest X-ray image data can also adjust the image illumination and contrast to avoid the influence of large differences in image illumination and contrast due to different equipment. Obtain a preset classification rule, wherein the preset classification rule includes segmentation data of several different lesion types, and the training samples of target brightness are processed based on the preset classification rule to generate classified training samples. By collecting the segmentation and classification results of the same lesion on multiple CT images containing lesions, the expert-annotated segmentation results and the expert-annotated classification results are obtained.

[0033] S103, construct a U-shaped fully convolutional neural network segmentation model.

[0034] In one embodiment, the U-type fully convolutional neural network segmentation model includes an upsampling model and a downsampling model. The upsampling model and the downsampling model are connected through a Skip Connection layer. The downsampling model is a SwinTransformer deep learning model, and the upsampling model is a DenseNet convolutional neural network model. Among them, the SwinTransformer deep learning model performs feature extraction as a downsampling, and the DenseNet convolutional neural network model performs feature restoration as an upsampling. For the U-type network, each downsampling will obtain a feature map, and the size will be reduced once, which is counted as a scale feature. The low level may focus on large-scale features such as the overall contour of the image, and the high level may focus on the fine-grained features of the image. For example, the low level may focus on the overall contour of the chest, and the high level may focus on the lesion area, especially the area where the lesion and normal tissue meet.

[0035] In another embodiment, the output of the Swin Transformer deep learning model is connected to the input of the DenseNet convolutional neural network model through the SkipConnection connection layer and the bottom feature layer. The output of the DenseNet convolutional neural network model is upsampled to generate preprocessed model features, and the preprocessed model features are connected to the Skip Connection connection layer to form a complete segmentation result. Through the design of the above technical features and connection relationships, this model can more efficiently and accurately segment high altitude pulmonary edema lesions in lung CT images. Among them, the bottom feature contains high-level semantic information, and the bottom feature is a feature map with a large number of channels and a small size. It is the deepest feature representation in the encoder. The bottom feature layer is the same as the bottom feature layer of the original U-Net, consisting of 2 3x3 conv convolutions and ReLU. In the decoder, the bottom feature is connected to the features in the upsampling path to help retain more detail information. Through the connection operation, the bottom feature can pass fine-grained information to the decoder, thereby achieving more accurate segmentation.

[0036] S104, training the neural network segmentation model using the labeled training samples to generate a trained neural network segmentation model.

[0037] In one embodiment, the segmentation model is trained using manually annotated lesion images. During training, the doctor selects a portion of patients as a training set and annotates the HAPE lesions in the CT images, setting the lesion area to 1 and the non-lesion area to 0.

[0038] In another embodiment, the segmentation model is trained using manually annotated lesion images to obtain a semantic segmentation pre-trained model, the learning rate of the model training is adjusted, and the semantic segmentation pre-trained model is continued to be trained using a training set with semantic image content to obtain a neural network segmentation model with semantic recognition function.

[0039] S105, inputting the target image into the trained segmentation model for processing, and outputting a target case, wherein the target case includes a segmentation result of a high-altitude pulmonary edema lesion.

[0040] In one implementation, the target image is grayed to generate a target color image, wherein the target image includes the actual condition of the customer. The target color image is normalized to generate a color image of a preset size, the color image is sliced ​​to sequentially generate three consecutive slice images, the slice images are processed to generate a three-channel image, the three-channel image is input into the trained segmentation model for processing, and the target case is output.

[0041] S106, adding the target image to the training sample set for updating the neural network segmentation model.

[0042] In one implementation, a target image of the user's actual condition is added to a training sample set, thereby continuously updating the training sample set, so that the neural network segmentation model can be continuously updated, thereby improving the recognition accuracy of the neural network segmentation model.

[0043] This application obtains training samples and target images, pre-processes the training samples, generates labeled training samples, and constructs a U-shaped full convolutional neural network segmentation model, wherein the U-shaped full convolutional neural network segmentation model includes an upsampling model and a downsampling model, the upsampling model and the downsampling model are connected through a Skip Connection connection layer, the downsampling model is a Swin Transformer deep learning model, and the upsampling model is a DenseNet convolutional neural network model. The labeled training samples are used to train the neural network segmentation model to generate a trained neural network segmentation model, the target image is input into the trained segmentation model for processing, and the target case is output, wherein the target case includes the segmentation result of the high-altitude pulmonary edema lesion, and the target image is added to the training sample set for updating the neural network segmentation model. Improving the accuracy and precision of high-altitude pulmonary edema chest X-ray lesion segmentation can better identify the lesion area, provide clinicians with more accurate diagnosis results and treatment suggestions, thereby improving medical standards and treatment effects. At the same time, the Transformer-based network has strong adaptability and generalization capabilities, can adapt to different types of chest X-ray images, and has good application prospects.

[0044] Optionally, in another embodiment of the method of the present application, the slicing process of the color image to sequentially generate three continuous slice images includes:

[0045] Processing the color image based on a preset color classification rule to generate a first color image containing different classification numbers;

[0046] Dividing the first color image based on the classification number to generate a plurality of initial slice images;

[0047] Preprocessing the initial slice image based on the target segmentation model to generate a preprocessed slice image;

[0048] The preprocessed slice images are processed based on a preset bilinear interpolation to generate a plurality of slice images of preset sizes.

[0049] In one embodiment, the color image is annotated and divided according to the color depth of the target image. All slices are preprocessed, all preprocessed slices are input into the optimal segmentation model and the segmentation results are obtained, the segmentation results are bilinearly interpolated and upsampled to restore to the original size, and the segmentation results of all slices of the chest three-dimensional CT scan image to be segmented are restored to the original size and merged into three-dimensional data according to the position relationship, and three consecutive CT slices are combined into a three-channel image, and sent to the trained segmentation model, so that the segmentation results of the high-altitude pulmonary edema lesions can be quickly obtained. The present invention combines three consecutive CT slices into a 3-channel image, mainly based on the ImageNet pre-training weights. The pre-training weights require an image of 3 channels, so the three consecutive slices are combined into a 3-channel image.

[0050] Optionally, in another embodiment of the method of the present application, the slicing process of the color image to sequentially generate three continuous slice images further includes:

[0051] Obtaining a preset size value to process the color image and generate a panoramic image of a preset size;

[0052] Performing multi-layer slicing processing on the panoramic image to generate a plurality of slice image groups at different depth levels;

[0053] The slice image group is processed based on time sequence or attribute information to generate a plurality of slice images.

[0054] In one embodiment, corresponding slice images can be generated in sequence according to the order of slices or the type of slices. The color image is sliced ​​to obtain multiple slice images of the initial depth level, and a slice image group of the initial depth level is generated based on the multiple slice images of the initial depth level. The multiple slice images of the current depth level are sliced ​​to obtain multiple slice images of the next depth level, and a slice image group of the next depth level is generated based on the multiple slice images of the next depth level. Repeat the previous step until the depth level reaches a preset first threshold, stop the slicing process, and obtain multiple slice image groups of different depth levels.

[0055] By applying the above technical solution, training samples and target images are obtained, data enhancement processing is performed on the training samples to generate enhanced training samples, the enhanced training samples are processed based on preset division rules to generate lesion areas and non-lesion areas, the training samples are normalized and preprocessed, and training samples of target brightness are generated. A preset classification rule is obtained, wherein the preset classification rule includes segmentation data of several different lesion types, and the training samples of the target brightness are processed based on the preset classification rule to generate classified training samples, and a U-type full convolutional neural network segmentation model is constructed, wherein the U-type full convolutional neural network segmentation model includes an upsampling model and a downsampling model, and the output end of the Swin Transformer deep learning model is connected to the input end of the DenseNet convolutional neural network model through the Skip Connection connection layer and the bottom feature layer, and the output end of the DenseNet convolutional neural network model is size-enlarged by upsampling to generate preprocessed model features, and the preprocessed model features are connected to the Skip Connection connection layer, the downsampling model is the Swin Transformer deep learning model, and the upsampling model is the DenseNet convolutional neural network model, and the neural network segmentation model is trained with the labeled training samples to generate a trained neural network segmentation model.

[0056] In addition, the target image is grayed to generate a target color image, the target color image is normalized to generate a color image of a preset size, the color image is processed based on a preset color division rule to generate a first color image containing different classification numbers, the first color image is divided based on the classification number to generate a number of initial slice images, the initial slice image is preprocessed based on the target segmentation model to generate a preprocessed slice image, the preprocessed slice image is processed based on a preset bilinear interpolation to generate a number of slice images of a preset size, the color image is processed by obtaining a preset size value to generate a panoramic image of a preset size, the panoramic image is multi-layered to generate a plurality of slice image groups of different depth levels, the slice image group is processed based on time sequence or attribute information to generate a number of slice images, the slice image is processed to generate a three-channel image, the three-channel image is input into the trained segmentation model for processing, and the target case is output, wherein the target case includes the segmentation result of the high-altitude pulmonary edema lesion, and the target image is added to the training sample set for updating the neural network segmentation model. The Transformer network is used to extract features and segment lesions from chest X-ray images, which can effectively solve the problems of low segmentation accuracy and inability to accurately identify lesion areas in existing technologies. In addition, Swin Transformer introduces a local and global multi-level window attention mechanism, which can better capture the weights of information at different locations and assign different weights to lesion areas and non-lesion areas. DenseNet establishes dense connections between different convolutional layers, which can better utilize the feature information of previous layers and improve the ability to restore features.

[0057] In one embodiment, if Figure 2 As shown, the present application also provides a Transformer-based segmentation device for chest X-ray lesions of high altitude pulmonary edema, comprising:

[0058] An acquisition module 201 is used to acquire training samples and target images;

[0059] The processing module 202 is used to preprocess the training samples to generate labeled training samples; construct a U-shaped fully convolutional neural network segmentation model, wherein the U-shaped fully convolutional neural network segmentation model includes an upsampling model and a downsampling model, the upsampling model and the downsampling model are connected through a Skip Connection layer, the downsampling model is a Swin Transformer deep learning model, and the upsampling model is a DenseNet convolutional neural network model; train the neural network segmentation model with the labeled training samples to generate a trained neural network segmentation model; input the target image into the trained segmentation model for processing, and output a target case, wherein the target case includes a segmentation result of a high-altitude pulmonary edema lesion; and add the target image to the training sample set for updating the neural network segmentation model.

[0060] This application obtains training samples and target images, pre-processes the training samples, generates labeled training samples, and constructs a U-shaped full convolutional neural network segmentation model, wherein the U-shaped full convolutional neural network segmentation model includes an upsampling model and a downsampling model, the upsampling model and the downsampling model are connected through a Skip Connection connection layer, the downsampling model is a Swin Transformer deep learning model, and the upsampling model is a DenseNet convolutional neural network model. The labeled training samples are used to train the neural network segmentation model to generate a trained neural network segmentation model, the target image is input into the trained segmentation model for processing, and the target case is output, wherein the target case includes the segmentation result of the high-altitude pulmonary edema lesion, and the target image is added to the training sample set for updating the neural network segmentation model. Improving the accuracy and precision of high-altitude pulmonary edema chest X-ray lesion segmentation can better identify the lesion area, provide clinicians with more accurate diagnosis results and treatment suggestions, thereby improving medical standards and treatment effects. At the same time, the Transformer-based network has strong adaptability and generalization capabilities, can adapt to different types of chest X-ray images, and has good application prospects.

[0061] In another embodiment of the present application, the processing module 202 is configured to connect the upsampling model and the downsampling model through a Skip Connection layer, including: the output end of the Swin Transformer deep learning model is connected to the input end of the DenseNet convolutional neural network model through the Skip Connection layer and the bottom feature layer; the output end of the DenseNet convolutional neural network model is resized by upsampling to generate preprocessed model features; the preprocessed model features are connected to the Skip Connection layer.

[0062] In another embodiment of the present application, the processing module 202 is configured to pre-process the training samples to generate labeled training samples, including: performing data enhancement processing on the training samples to generate enhanced training samples; processing the enhanced training samples based on preset division rules to generate lesion areas and non-lesion areas.

[0063] In another embodiment of the present application, the processing module 202 is configured to preprocess the training samples to generate labeled training samples, and also includes: performing normalization preprocessing on the training samples to generate training samples of target brightness; obtaining preset classification rules, wherein the preset classification rules include segmentation data of several different lesion types; processing the training samples of the target brightness based on the preset classification rules to generate classified training samples.

[0064] In another embodiment of the present application, the processing module 202 is configured to input the target image into the trained segmentation model for processing and output the target case, including: gray-scaling the target image to generate a target color image; normalizing the target color image to generate a color image of a preset size; slicing the color image to sequentially generate three consecutive slice images; processing the slice image to generate a three-channel image; and inputting the three-channel image into the trained segmentation model for processing to output the target case.

[0065] In another embodiment of the present application, the processing module 202 is configured to perform slicing processing on the color image to generate three consecutive slice images in sequence, including: processing the color image based on a preset color division rule to generate a first color image containing different classification numbers; dividing the first color image based on the classification number to generate a number of initial slice images; preprocessing the initial slice image based on the target segmentation model to generate a preprocessed slice image; processing the preprocessed slice image based on a preset bilinear interpolation to generate a number of slice images of preset sizes.

[0066] In another embodiment of the present application, the processing module 202 is configured to slice the color image to generate three consecutive slice images in sequence, and also includes: obtaining a preset size value to process the color image to generate a panoramic image of a preset size; performing multi-layer slicing on the panoramic image to generate multiple slice image groups of different depth levels; processing the slice image group based on time sequence or attribute information to generate several slice images.

[0067] The present application obtains training samples and target images, performs data enhancement processing on the training samples, generates enhanced training samples, processes the enhanced training samples based on preset division rules, generates lesion areas and non-lesion areas, performs normalization preprocessing on the training samples, and generates training samples of target brightness. Obtain preset classification rules, wherein the preset classification rules include segmentation data of several different lesion types, process the training samples of target brightness based on the preset classification rules, generate classified training samples, and construct a U-shaped full convolutional neural network segmentation model, wherein the U-shaped full convolutional neural network segmentation model includes an upsampling model and a downsampling model, the output end of the Swin Transformer deep learning model is connected to the input end of the DenseNet convolutional neural network model through the Skip Connection connection layer and the bottom feature layer, the output end of the DenseNet convolutional neural network model is size-enlarged by upsampling, and preprocessed model features are generated, the preprocessed model features are connected to the Skip Connection connection layer, the downsampling model is the Swin Transformer deep learning model, the upsampling model is the DenseNet convolutional neural network model, the labeled training samples are used to train the neural network segmentation model, and a trained neural network segmentation model is generated.

[0068] In addition, the target image is grayed to generate a target color image, the target color image is normalized to generate a color image of a preset size, the color image is processed based on a preset color division rule to generate a first color image containing different classification numbers, the first color image is divided based on the classification number to generate a number of initial slice images, the initial slice image is preprocessed based on the target segmentation model to generate a preprocessed slice image, the preprocessed slice image is processed based on a preset bilinear interpolation to generate a number of slice images of a preset size, the color image is processed by obtaining a preset size value to generate a panoramic image of a preset size, the panoramic image is multi-layered to generate a plurality of slice image groups of different depth levels, the slice image group is processed based on time sequence or attribute information to generate a number of slice images, the slice image is processed to generate a three-channel image, the three-channel image is input into the trained segmentation model for processing, and the target case is output, wherein the target case includes the segmentation result of the high-altitude pulmonary edema lesion, and the target image is added to the training sample set for updating the neural network segmentation model. The Transformer network is used to extract features and segment lesions from chest X-ray images, which can effectively solve the problems of low segmentation accuracy and inability to accurately identify lesion areas in existing technologies. In addition, Swin Transformer introduces a local and global multi-level window attention mechanism, which can better capture the weights of information at different locations and assign different weights to lesion areas and non-lesion areas. DenseNet establishes dense connections between different convolutional layers, which can better utilize the feature information of previous layers and improve the ability to restore features.

[0069] The present application embodiment provides an electronic device, such as Figure 3 As shown, it includes a processor 300, a memory 301, a bus 302 and a communication interface 303, and the processor 300, the communication interface 303 and the memory 301 are connected via the bus 302; the memory 301 stores a computer program that can be run on the processor 300, and when the processor 300 runs the computer program, it executes the Transformer-based high-altitude pulmonary edema chest X-ray lesion segmentation method provided in any of the aforementioned embodiments of the present application.

[0070] The memory 301 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 303 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0071] The bus 302 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 301 is used to store a program, and the processor 300 executes the program after receiving an execution instruction. The Transformer-based high altitude pulmonary edema chest radiograph lesion segmentation method disclosed in any implementation of the aforementioned embodiment of the present application may be applied to the processor 300, or implemented by the processor 300.

[0072] The processor 300 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 300. The above processor 300 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor to be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 301, and the processor 300 reads the information in the memory 301 and completes the steps of the above method in combination with its hardware.

[0073] The electronic device provided in the above-mentioned embodiments of the present application and the Transformer-based high altitude pulmonary edema chest X-ray lesion segmentation method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0074] The present application embodiment provides a computer-readable storage medium, such as Figure 4 As shown, the computer-readable storage medium stores 401 a computer program, and when the computer program is read and executed by the processor 402, the aforementioned Transformer-based chest X-ray lesion segmentation method for high altitude pulmonary edema is implemented.

[0075] The technical solution of the embodiment of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling an electronic device (which may be an air conditioner, a refrigeration device, a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0076] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the Transformer-based high altitude pulmonary edema chest X-ray lesion segmentation method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0077] An embodiment of the present application provides a computer program product, including a computer program, wherein the computer program is executed by a processor to implement the method described above.

[0078] The computer program product provided in the above-mentioned embodiments of the present application and the Transformer-based high altitude pulmonary edema chest X-ray lesion segmentation method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0079] It should be noted that, in this application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or still includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0080] Each embodiment in the present application is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the Transformer-based high altitude pulmonary edema chest X-ray lesion segmentation method, electronic device, electronic device, and readable storage medium embodiment, since they are basically similar to the above-mentioned Transformer-based high altitude pulmonary edema chest X-ray lesion segmentation method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the above-mentioned Transformer-based high altitude pulmonary edema chest X-ray lesion segmentation method embodiment.

[0081] Although the present application is disclosed as above, the present application is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application. Therefore, the protection scope of the present application shall be subject to the scope defined by the claims.

Claims

1. A Transformer-based segmentation method for chest X-ray lesions of high altitude pulmonary edema, characterized in that: include: Obtain training samples and target images; Preprocessing the training samples to generate labeled training samples; Constructing a U-shaped fully convolutional neural network segmentation model, wherein the U-shaped fully convolutional neural network segmentation model includes an upsampling model and a downsampling model, the upsampling model and the downsampling model are connected through a Skip Connection layer, the downsampling model is a Swin Transformer deep learning model, and the upsampling model is a DenseNet convolutional neural network model; Training the neural network segmentation model with the labeled training samples to generate a trained neural network segmentation model; Inputting the target image into the trained segmentation model for processing, and outputting a target case, wherein the target case includes a segmentation result of a high-altitude pulmonary edema lesion; The step of inputting the target image into the trained segmentation model for processing and outputting the target case comprises: grayscale the target image to generate a target color image; Performing image normalization processing on the target color image to generate a color image of a preset size; Slicing the color image to generate three consecutive slice images in sequence; Processing the slice image to generate a three-channel image; Inputting the three-channel image into the trained segmentation model for processing, and outputting a target case; The slicing process is performed on the color image to sequentially generate three continuous slice images, including: Processing the color image based on a preset color classification rule to generate a first color image containing different classification numbers; Dividing the first color image based on the classification number to generate a plurality of initial slice images; Preprocessing the initial slice image based on the target segmentation model to generate a preprocessed slice image; Processing the preprocessed slice image based on a preset bilinear interpolation to generate a plurality of slice images of preset sizes; Adding the target image to the training sample set for updating the neural network segmentation model; The up-sampling model and the down-sampling model are connected via a Skip Connection layer, including: The output end of the Swin Transformer deep learning model is connected to the input end of the DenseNet convolutional neural network model through the Skip Connection layer and the bottom feature layer; The output end of the DenseNet convolutional neural network model is upsampled to generate preprocessed model features; The preprocessed model features are connected to the Skip Connection layer; The step of slicing the color image to sequentially generate three continuous slice images further includes: Obtaining a preset size value to process the color image and generate a panoramic image of a preset size; Performing multi-layer slicing processing on the panoramic image to generate a plurality of slice image groups at different depth levels; The slice image group is processed based on time sequence or attribute information to generate a plurality of slice images.

2. The method according to claim 1, characterized in that: The preprocessing of the training samples to generate labeled training samples includes: Performing data enhancement processing on the training samples to generate enhanced training samples; The enhanced training samples are processed based on preset division rules to generate lesion areas and non-lesion areas.

3. The method according to claim 1, characterized in that The preprocessing of the training samples to generate labeled training samples also includes: Performing normalization preprocessing on the training samples to generate training samples of target brightness; Acquiring a preset classification rule, wherein the preset classification rule includes segmentation data of several different lesion types; The training samples of the target brightness are processed based on the preset classification rules to generate classified training samples.

4. A Transformer-based segmentation device for chest X-ray lesions of high altitude pulmonary edema, characterized in that: The device comprises: An acquisition module, used to acquire training samples and target images; A processing module is used to preprocess the training samples to generate labeled training samples; construct a U-shaped full convolutional neural network segmentation model, wherein the U-shaped full convolutional neural network segmentation model includes an upsampling model and a downsampling model, the upsampling model and the downsampling model are connected through a Skip Connection connection layer, the downsampling model is a Swin Transformer deep learning model, and the upsampling model is a DenseNet convolutional neural network model; train the neural network segmentation model with the labeled training samples to generate a trained neural network segmentation model; input the target image into the trained segmentation model for processing, and output a target case, wherein the target case includes a segmentation result of a high-altitude pulmonary edema lesion; the inputting the target image into the trained segmentation model for processing, and outputting the target case, includes: graying the target image to generate a target color image; normalizing the target color image to generate a color image of a preset size; slicing the color image to generate three consecutive slice images in sequence; processing the slice image to generate a target case; into a three-channel image; inputting the three-channel image into the trained segmentation model for processing and outputting the target case; slicing the color image to generate three continuous slice images in sequence, including: processing the color image based on a preset color division rule to generate a first color image containing different classification numbers; dividing the first color image based on the classification number to generate a number of initial slice images; preprocessing the initial slice image based on the target segmentation model to generate a preprocessed slice image; processing the preprocessed slice image based on a preset bilinear interpolation to generate a number of slice images of preset sizes; adding the target image to the training sample set for updating the neural network segmentation model; the upsampling model and the downsampling model are connected by Skip The method comprises: connecting the output end of the Swin Transformer deep learning model to the input end of the DenseNet convolutional neural network model through the Skip Connection connection layer and the bottom feature layer; the output end of the DenseNet convolutional neural network model is size-enlarged by upsampling to generate preprocessed model features; the preprocessed model features are connected to the Skip Connection connection layer; the color image is sliced ​​to generate three consecutive slice images in sequence, and further comprises: obtaining a preset size value to process the color image to generate a panoramic image of a preset size; performing multi-layer slicing on the panoramic image to generate multiple slice image groups of different depth levels; and processing the slice image group based on time sequence or attribute information to generate a plurality of slice images.

5. An electronic device, characterized in that: include: processor; and a memory for storing executable instructions for the processor; The processor is configured to execute the Transformer-based high altitude pulmonary edema chest X-ray lesion segmentation method according to any one of claims 1 to 3 by executing the executable instructions.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for segmenting high altitude pulmonary edema chest X-ray lesions based on Transformer according to any one of claims 1 to 3 is implemented.