Method and system for automatically analyzing chest respiratory muscle volume based on deep learning network

Abstract

The invention discloses a method and a system for automatically analyzing chest respiratory muscle volume based on a deep learning network, and belongs to the field of medical images. The method for automatically analyzing the chest respiratory muscle volume based on the deep learning network comprises the steps that a CT image and a chest respiratory muscle image manually marked by a doctor are collected, and the CT image and the image manually marked by the doctor are preprocessed to obtain a standard sketching graph; performing data enhancement operation on the standard sketching picture; constructing a U-Net model, and training the constructed U-Net model by using the standard sketching graph after data enhancement; and automatically sketching the chest respiratory muscle in the medical image by using the trained U-Net model to obtain the contour of the chest respiratory muscle. A U-Net network model is constructed, an efficient, reliable and easy-to-implement solution is provided for three-dimensional medical image segmentation, the method has wide application prospects and clinical values, time consumed in clinical work is greatly shortened, respiratory muscle related information is helped to be obtained clinically and rapidly, treatment of a patient is guided, and prognosis of the patient is improved.

Description

Technical Field

[0001] The present invention belongs to the field of medical imaging, and specifically relates to a method and system for automatically analyzing chest respiratory muscle volume based on a deep learning network. Background Art

[0002] The loss of muscle mass in critically ill hospital patients is associated with severe consequences, such as prolonged mechanical ventilation, immobilization, and higher mortality, which greatly affects the patient's recovery. Therefore, monitoring muscle mass, especially its reduction, should be done with a rapid and accurate method. Patients in the intensive care unit are often mechanically ventilated due to their condition, which may have a greater impact on the patient's chest respiratory muscles and is associated with poorer patient outcomes.

[0003] In the prior art, muscle mass can be assessed by various imaging methods, such as computed tomography, magnetic resonance imaging, ultrasound, dual-energy X-ray absorptiometry, etc. However, magnetic resonance imaging is time-consuming and expensive; dual-energy X-ray absorptiometry has radiation, and the results are affected by tissue hydration; ultrasound examination is subjective and may lead to biased measurement results; computed tomography (CT) is fast and can quantify fat-free muscle mass in patients with severe fluid retention; however, the above methods use manual delineation, which takes a lot of time and is not conducive to the development of clinical work. Summary of the invention

[0004] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a method and system for automatically analyzing the volume of chest respiratory muscles based on a deep learning network, thereby solving the problems in the prior art.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] The method for automatically analyzing chest respiratory muscle volume based on a deep learning network comprises the following steps:

[0007] Collecting CT images and chest respiratory muscle images manually annotated by doctors, and preprocessing the CT images and the images manually annotated by doctors to obtain standard outlines, which are used as identification label sets;

[0008] Perform data augmentation on standard sketches;

[0009] Construct a U-Net model and train the constructed U-Net model using the data-enhanced standard sketch map;

[0010] The trained U-Net model is used to automatically outline the chest respiratory muscles in medical images to obtain the contours of the chest respiratory muscles.

[0011] Furthermore, the process of preprocessing the CT images and the images manually annotated by doctors to obtain standard delineation images is as follows: images of qualified quality are screened out, and during the data processing, the window levels of all DICOM files are reset to the range of -190 to 150 HU.

[0012] Furthermore, data augmentation operations are performed on the standard sketch including random XY flipping, 90 degree rotation, ±10 degree rotation around the XY axis, and random cropping.

[0013] Further, the U-Net model includes an encoder, a decoder and an output layer;

[0014] The encoder includes four convolution blocks, each of which is provided with two 3D convolution layers and one 3D maximum pooling layer; each 3D convolution layer uses two 3×3×3 convolution kernels, the 3D convolution layer is followed by BN and ReLU activation functions, and each convolution block is downsampled by a 2×2×2 3D maximum pooling layer;

[0015] The decoder also includes four convolution blocks, each of which includes a 3D upsampling layer and two 3D convolution layers, and the upsampling operation is performed through a 2×2×2 3D upsampling layer; after upsampling, each 3D convolution layer uses two 3×3×3 convolution kernels and is also equipped with BN and ReLU activation functions;

[0016] The output layer uses a 3D convolutional layer to map the output of the last convolutional block of the decoder to the number of target categories.

[0017] Furthermore, in the encoder, at the start stage of the second convolution block, a maximum pooling layer of 4×4×4 size is introduced to extract features of the input data.

[0018] Furthermore, the output of each convolution block in the encoder is transmitted to the corresponding decoder part through a skip connection, and the decoder is spliced ​​by the upsampled feature map with the skip connection from the corresponding convolution block of the encoder.

[0019] The system for automatically analyzing the volume of chest respiratory muscles based on deep learning network includes:

[0020] Image acquisition module: collects CT images and chest respiratory muscle images manually annotated by doctors, and pre-processes the CT images and images manually annotated by doctors to obtain standard outline images, which are used as identification label sets;

[0021] Image enhancement processing module: perform data enhancement operations on standard sketch images;

[0022] Model building and training module: build a U-Net model and train it using the standard delineation graph after data enhancement;

[0023] And, the image delineation module: uses the trained U-Net model to automatically delineate the chest respiratory muscles in the medical image to obtain the contour of the chest respiratory muscles.

[0024] A computer storage medium stores a readable program, which can execute the above-mentioned method of automatically analyzing the volume of chest respiratory muscles based on a deep learning network when the program is run.

[0025] An electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0026] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned method for automatically analyzing the volume of chest respiratory muscles based on a deep learning network.

[0027] A computer program product includes computer instructions, wherein the computer instructions instruct a computing device to execute operations corresponding to the above-mentioned method for automatically analyzing chest respiratory muscle volume based on a deep learning network.

[0028] Beneficial effects of the present invention:

[0029] By constructing a U-Net network model, the present invention provides an efficient, reliable and easy-to-implement solution for three-dimensional medical image segmentation, which has broad application prospects and clinical value, greatly reduces the time spent in clinical work, helps clinicians quickly obtain respiratory muscle-related information, guides patient treatment, and improves patient prognosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 It is a flow chart of a method for automatically analyzing chest respiratory muscle volume based on a deep learning network of the present invention;

[0032] Figure 2 It is a schematic diagram of the structure of the deep learning network model of the present invention;

[0033] Figure 3 This is a three-dimensional image of the chest respiratory muscles drawn by the deep learning network model of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] Example 1

[0036] The method for automatically analyzing chest respiratory muscle volume based on a deep learning network comprises the following steps:

[0037] S1, collecting CT images and chest respiratory muscle images manually annotated by doctors, and preprocessing the CT images and the images manually annotated by doctors to obtain standard outlines, which are used as identification label sets;

[0038] The steps of preprocessing the CT images and the chest respiratory muscle images manually annotated by doctors include:

[0039] S11, filter out images of qualified quality

[0040] S12, during the data processing, the window levels of all DICOM files are reset to the range of -190 to 150 HU to ensure that the information required for accurate images can be obtained within this range.

[0041] S2, data enhancement operation is performed on the standard sketch map;

[0042] Data augmentation operations include random XY flipping, 90-degree rotation, ±10-degree rotation around the XY axis, and random cropping.

[0043] Among them, random XY flipping refers to randomly flipping the image horizontally or vertically to generate samples in different directions. This operation can improve the model's ability to recognize objects in different directions.

[0044] 90 degree rotation: Rotating the image 90 degrees clockwise or counterclockwise can increase the diversity of the data and help the model generalize better.

[0045] ±10 degree rotation around the XY axis:

[0046] (1) Calculate the rotation matrix: Calculate the rotation matrix R based on the rotation angle (±10 degrees). The following formula can be used:

[0047] R=[[cos(angle),-sin(angle)],[sin(angle),cos(angle)]]

[0048] (2) Apply the rotation matrix: For each pixel in the image, transform its coordinates (x, y) according to the above rotation matrix:

[0049] new_x=x*R[0][0]+y*R[0][1]

[0050] new_y=x*R[1][0]+y*R[1][1];

[0051] Random cropping: If the rotated coordinates exceed the range of the original image, they can be handled by padding (using pixel values ​​outside the boundary) or cropping (removing pixels outside the boundary).

[0052] S3, constructing a U-Net model, and using the data-enhanced standard outline image to train the constructed U-Net model, to obtain a chest respiratory muscle model corresponding to the standard outline image (i.e., the trained U-Net model);

[0053] like Figure 2 As shown, the U-Net model includes an encoder, a decoder, and an output layer;

[0054] 1) Encoder

[0055] The encoder consists of four convolution blocks, each of which contains two 3D convolution layers and one 3D max pooling layer. Each 3D convolution layer uses two 3×3×3 convolution kernels to ensure full extraction of local features. The 3D convolution layer is followed by batch normalization (BN) and ReLU activation functions to enhance the training stability and nonlinear expression ability of the model. Each convolution block is downsampled by a 2×2×2 3D max pooling layer with a step size of 2 to reduce the spatial dimension of the feature map while retaining the most important feature information. In addition, at the beginning of the second convolution block, a 4×4×4 max pooling layer is introduced to perform deeper feature extraction on the input data. By selecting the maximum value in the local area, the salient features in the image are captured, providing a more abstract and compact feature representation for subsequent processing.

[0056] In addition, the output of each convolutional block is passed to the corresponding decoder part through a skip connection, laying the foundation for subsequent feature fusion and detail reconstruction.

[0057] 2) Decoder

[0058] The decoder also includes four convolution blocks, each of which includes a 3D upsampling layer and two 3D convolution layers; the size of the 3D upsampling layer is 2×2×2, and the upsampling operation is performed through the 2×2×2 3D upsampling layer to increase the spatial dimension of the feature map and restore the detail information that may be lost during the downsampling process. After upsampling, each 3D convolution layer uses two 3×3×3 convolution kernels and is also equipped with BN and ReLU activation functions to further refine the feature representation.

[0059] In addition, the decoder achieves comprehensive utilization of cross-level features by splicing the upsampled feature maps with the jump connections from the corresponding convolutional blocks of the encoder; this fusion strategy not only enhances the model's ability to extract multi-scale features of input data, but also significantly improves the model's ability to represent complex structures and detailed information.

[0060] 3) Output layer

[0061] The output layer uses a 3D convolutional layer, which maps the output of the last convolutional block of the decoder to the number of target categories. Specifically, the number of output channels is reduced to a dimension that matches the number of target categories through a 1×1×1 convolution kernel, and the Softmax activation function is used to output the probability that each voxel belongs to a different category. In the present invention, the output layer can accurately distinguish different tissue types such as background, muscle, and fat.

[0062] When building the 3D U-Net+ model, a 4×4×4 max pooling layer is designed at the beginning of the second convolutional block of the encoder, immediately following the first convolutional layer. The introduction of this max pooling layer is intended to effectively extract deep features from the input data, capturing the most significant and important information in the image by selecting the maximum value in each local area. This operation not only reduces the dimension of the data, but also helps the network focus on the most discriminative features, providing a more compact and abstract feature representation for subsequent network layers.

[0063] Subsequently, the feature map of the second convolution block of the decoder after processing by the maximum pooling layer is fused with the feature map of the fourth convolution block. This feature fusion strategy combines feature information from different convolution blocks and different levels to achieve comprehensive utilization of cross-level features. This integration not only enhances the model's ability to capture multi-scale features of input data, but also improves the model's ability to represent complex structures and detailed information.

[0064] S4, using the trained U-Net model to automatically outline the chest respiratory muscles in the medical image to obtain the contours of the chest respiratory muscles.

[0065] It should be noted that the method for automatically analyzing chest respiratory muscle volume based on deep learning network proposed in the present invention cannot be used directly to diagnose patients' diseases. It only provides some reference value for clinical judgment of patients' condition changes. It also needs to be combined with other indicators, such as patients' laboratory indicators, imaging examination results and clinical symptoms, etc., to provide some suggestions for patients' diagnosis and treatment.

[0066] Based on similar inventive concepts, an embodiment of the present invention also provides a computer storage medium storing a readable program, which, when the program is running, can execute the method of automatically analyzing the volume of chest respiratory muscles based on a deep learning network.

[0067] Based on similar inventive concepts, an embodiment of the present invention provides an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0068] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the method for automatically analyzing chest respiratory muscle volume based on a deep learning network.

[0069] Based on similar inventive concepts, an embodiment of the present invention also provides a computer program product, including computer instructions, which instruct a computing device to execute operations corresponding to the method for automatically analyzing chest respiratory muscle volume based on a deep learning network.

[0070] Example 2

[0071] In order to verify the effectiveness of the method for automatically analyzing the volume of chest respiratory muscles based on a deep learning network in Example 1, a specific example is proposed in this embodiment to verify the above method. The specific steps are as follows:

[0072] S1, collects CT image data and chest respiratory muscle images manually annotated by doctors. The chest CT images are derived from 100 cases of DICOM format data provided by Zhongda Hospital affiliated to Southeast University. Doctors use ITK-SNAP software to semi-automatically draw respiratory muscle images. The respiratory muscles delineated are the 3rd to 8th intercostal muscles and serratus anterior muscles, and the threshold range used is -20-120HU; and preprocessing is performed to obtain a standard delineation image.

[0073] S2, data enhancement processing of the standard delineation image;

[0074] S3, using the standard delineation diagram to train the U-Net model, and using the trained U-Net model to automatically delineate the chest respiratory muscles in the medical image to obtain the contours of the chest respiratory muscles, such as Figure 3 As shown;

[0075] Figure 3 It can be seen that the deep learning model can segment the chest respiratory muscles well, and the segmented chest respiratory muscles range from the 3rd to 8th intercostal muscles and the serratus anterior muscle. This model can help clinicians quickly obtain information related to the patient's respiratory muscles, provide information for clinical diagnosis and treatment, and combine with other clinical examination results to help patients recover and judge their prognosis.

[0076] Example 3

[0077] In this embodiment, a system for automatically analyzing chest respiratory muscle volume based on a deep learning network is proposed, comprising:

[0078] Image acquisition module: collects CT images and chest respiratory muscle images manually annotated by doctors, and pre-processes the CT images and images manually annotated by doctors to obtain standard outline images, which are used as identification label sets;

[0079] Image enhancement processing module: perform data enhancement operations on standard sketch images;

[0080] Model building and training module: build a U-Net model and train it using the standard delineation graph after data enhancement;

[0081] And, the image delineation module: uses the trained U-Net model to automatically delineate the chest respiratory muscles in the medical image to obtain the contour of the chest respiratory muscles.

[0082] The method of the present invention may be implemented in hardware, firmware, or as software or computer code that may be stored in a recording medium (such as a CDROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded over a network and will be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that a computer, processor, microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, processor, or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown herein.

[0083] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A method for automatically analyzing chest respiratory muscle volume based on a deep learning network, characterized in that: The following steps are involved: Collecting CT images and chest respiratory muscle images manually annotated by doctors, and preprocessing the CT images and the images manually annotated by doctors to obtain standard outlines, which are used as identification label sets; Perform data augmentation on standard sketches; Construct a U-Net model and train the constructed U-Net model using the data-enhanced standard sketch map; The trained U-Net model is used to automatically outline the chest respiratory muscles in medical images to obtain the contours of the chest respiratory muscles.

2. The method for automatically analyzing chest respiratory muscle volume based on a deep learning network according to claim 1, characterized in that: The process of preprocessing the CT images and the images manually annotated by doctors to obtain standard delineation images is as follows: images of qualified quality are screened out, and during the data processing, the window levels of all DICOM files are reset to the range of -190 to 150 HU.

3. The method for automatically analyzing chest respiratory muscle volume based on deep learning network according to claim 1, characterized in that: Data augmentation operations on standard sketches include random XY flipping, 90-degree rotation, ±10-degree rotation around the XY axis, and random cropping.

4. The method for automatically analyzing chest respiratory muscle volume based on deep learning network according to claim 1, characterized in that: The U-Net model includes an encoder, a decoder and an output layer; The encoder includes four convolution blocks, each of which is provided with two 3D convolution layers and one 3D maximum pooling layer; each 3D convolution layer uses two 3×3×3 convolution kernels, the 3D convolution layer is followed by BN and ReLU activation functions, and each convolution block is downsampled by a 2×2×2 3D maximum pooling layer; The decoder also includes four convolution blocks, each convolution block includes a 3D upsampling layer and two 3D convolution layers, and an upsampling operation is performed through a 2×2×2 3D upsampling layer; After upsampling, each 3D convolutional layer uses two 3×3×3 convolution kernels and is also equipped with BN and ReLU activation functions; The output layer uses a 3D convolutional layer to map the output of the last convolutional block of the decoder to the number of target categories.

5. The method for automatically analyzing chest respiratory muscle volume based on deep learning network according to claim 4, characterized in that: In the encoder, at the beginning stage of the second convolutional block, a maximum pooling layer of 4×4×4 size is introduced to extract features of the input data.

6. The method for automatically analyzing chest respiratory muscle volume based on deep learning network according to claim 4, characterized in that: The output of each convolution block in the encoder is passed to the corresponding decoder part through a skip connection, and the decoder is spliced ​​by the upsampled feature map with the skip connection from the corresponding convolution block of the encoder.

7. A system for automatically analyzing chest respiratory muscle volume based on a deep learning network, characterized in that: include: Image acquisition module: collects CT images and chest respiratory muscle images manually annotated by doctors, and pre-processes the CT images and images manually annotated by doctors to obtain standard outline images, which are used as identification label sets; Image enhancement processing module: perform data enhancement operations on standard sketch images; Model building and training module: build a U-Net model and train it using the standard delineation graph after data enhancement; And, the image delineation module: uses the trained U-Net model to automatically delineate the chest respiratory muscles in the medical image to obtain the contour of the chest respiratory muscles.

8. A computer storage medium storing a readable program, characterized in that: When the program is running, it can execute the method for automatically analyzing the volume of chest respiratory muscles based on a deep learning network as described in any one of claims 1 to 6.

9. An electronic device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the method for automatically analyzing chest respiratory muscle volume based on a deep learning network as described in any one of claims 1-6.

10. A computer program product comprising computer instructions, characterized in that: The computer instructions instruct the computing device to perform operations corresponding to the method for automatically analyzing the volume of chest respiratory muscles based on a deep learning network as described in any one of claims 1-6.

Images