A method for constructing a segmentation network model for small bowel CT image recognition and a recognition method.

Abstract

This invention provides a method for constructing a small bowel CT image recognition and segmentation network model and a recognition method. The construction method includes the following steps: acquiring several small bowel CT images showing lesions and constructing an image dataset, dividing the dataset into a training set and a test set proportionally; enhancing the image dataset in the above steps using dynamic stochastic resonance enhancement; constructing a U-Net network model and training the network model using the enhanced training set obtained in the above steps; testing the network model using the enhanced test set obtained in the above steps, and completing the model construction. This invention proposes a method for enhancing CT images based on dynamic stochastic resonance, which can effectively enhance useful signals in the original image. Since medical image processing is mainly used to assist doctors in diagnosis, the input is an image, and the output is also an image that is easier for doctors to use. This invention establishes a U-shaped neural network that meets this requirement.

Description

Technical Field

[0001] This invention belongs to the field of small bowel CT image recognition technology, and particularly relates to a method for constructing a small bowel CT image recognition segmentation network model and a recognition method. Background Technology

[0002] With industrial development and changes in diet, living environment, and lifestyle, the number of people suffering from gastrointestinal diseases has increased significantly. These diseases can occur in any part of the small intestine and lead to serious health problems, decreased quality of life, and even death. Due to the diversity and irregularity of clinical manifestations and the lack of specific and reliable diagnostic methods, the preoperative diagnostic rate of small bowel lesions has remained low. Small intestinal lymphoma (SIL) is one of the most common malignant tumors of the small intestine, originating from the lamina propria and submucosal lymphoid tissue, and is usually treated with surgical resection. Inflammatory bowel disease (IBD) is a common chronic digestive system disease that can cause infections or ulcers in the small and large intestines. However, small intestinal non-lymphoma (SINL) diseases share characteristics with inflammatory bowel disease, such as intestinal bacterial diseases and viral infectious diseases. Currently, barium meals and endoscopy are commonly used diagnostic methods for small bowel diseases. A full digestive tract barium meal examination is helpful for localization diagnosis but cannot observe the relationship between suspicious sites and surrounding structures. Endoscopic and histopathological examinations can obtain lesion tissue and perform biopsies for further diagnosis of specific diseases. However, similar to a small bowel barium meal, endoscopy does not show the thickness of lesions or extraintestinal lesions. Patients are often reluctant to undergo these procedures due to the painful and invasive nature of the surgical process and the potential for complications.

[0003] The rapid development of medical imaging technologies such as CT and MRI has provided more effective diagnostic methods for small bowel lesions. In particular, CT enterography can clearly show the location, number, and characteristics of tumors, as well as their relationship to the intestine, blood vessels, and invaded adjacent structures. Small bowel lymphomas tend to grow from cavities, are relatively large, and appear as a clearly round or oval shape on CT images. In inflammatory bowel disease, abrupt changes in the lesion will result in a clear boundary with the normal intestine, while in later stages, due to extensive fibrosis, the intestinal lumen will become irregular. Non-lymphomatous diseases of the small intestine caused by tuberculous caseous necrosis, such as intestinal tuberculosis, primarily affect the ileocecal region of the intestine and are characterized by spastic contractions and an indeterminate intestinal shape.

[0004] Because these lesions often present with no obvious symptoms in their early stages, they are difficult to detect. Patients are frequently diagnosed at an advanced stage, resulting in a relatively low 5-year survival rate. Therefore, early diagnosis and treatment are crucial for improving the treatment outcomes of small bowel diseases.

[0005] In recent years, computer-aided diagnostic technology has been widely applied in the field of medical imaging. Automatic recognition and segmentation provide doctors with reliable and effective auxiliary diagnostic tools, significantly reducing their workload and effectively minimizing problems such as misdiagnosis and missed diagnosis caused by human error, thus saving patients more treatment time. Therefore, research on intelligent and automated medical image recognition and segmentation of small bowel lesions is of great significance. To better solve the problem of disease recognition in medical images, deep learning, as an effective method of artificial intelligence, has become a hot topic in medical image research. Common deep learning networks include FCN, SegNet, and U-Net.

[0006] In 2015, deep learning made groundbreaking progress in medical image segmentation, particularly with the proposal of a U-shaped network structure, including an encoder, bottleneck module, and decoder. This U-Net, with its advantages of contextual information combination, fast training speed, and low data requirements, met the technical requirements of medical image segmentation and was widely applied in medical image analysis. In 2018, Roy et al. added squeeze excitation (SE) modules to the standard U-Net. Experiments showed good segmentation performance, but computational complexity also increased. Ozgun-Cicek et al. proposed a 3D U-Net structure based on a 3D convolutional network, which can improve segmentation accuracy. However, due to the large amount of memory required for computation, a complete CT image needs to be segmented into a series of 3D blocks for input, leading to the loss of some global information. Although significant research has been achieved in image segmentation of vital organs such as the heart, kidneys, and lungs, progress in CT image recognition and segmentation of small bowel lesions has been minimal due to the difficulty in distinguishing the characteristics of certain small bowel lesions, such as inflammatory bowel disease and small bowel non-lymphoma, and the relatively large range of images requiring exploration and processing.

[0007] Furthermore, neural network models used for intelligent image segmentation and recognition require a large amount of labeled data, which needs to be judged and labeled by professional doctors. Because CT images are created by using X-rays to obtain tomographic images of relevant parts of the body, the resulting images are grayscale images composed of pixels with different gray levels. Additionally, the location of the lesion in the small intestine and the stage of the lesion can cause the gray level of the lesion area to be similar to that of nearby normal tissue pixels. All of these factors pose difficulties for doctors in diagnosis and in preparing labeled data. Currently, CT-assisted diagnosis is mainly performed manually by doctors, which is a time-consuming and labor-intensive task due to the limitations of doctors' experience and the complexity of the lesions. Summary of the Invention

[0008] To address the problems of unclear lesion areas in small bowel CT images, easy missed diagnoses, and difficulties in preparing label data for automatic lesion identification systems, this invention proposes a method for automatically identifying and segmenting small bowel lesion areas based on dynamic stochastic resonance-enhanced CT images and then utilizing a U-net network with an embedded attention mechanism. Dynamic stochastic resonance can enhance useful signals in small bowel images, especially lesion tissue, thus facilitating faster and more accurate sample labeling by physicians. Based on the U-Net architecture, an embedded attention mechanism (AM) module enables the network to focus on specific target features of the CT image and suppress irrelevant features. Simultaneously, the addition of a batch normalization (BN) module effectively alleviates the problems of slow convergence speed or gradient explosion during training. The first aspect of this invention provides a method for constructing a small bowel CT image recognition and segmentation network model, including the following steps:

[0009] Step 1: Obtain several CT images of the small intestine showing lesions and build an image dataset. Divide the dataset into a training set and a test set according to the proportions.

[0010] Step 2: Enhance the image dataset from Step 1 using dynamic stochastic resonance enhancement; the iterative formula for dynamic stochastic resonance is:

[0011] x(n+1)=x(n)+Δt[(ax(n)-bx 3 (n))+Input] (1)

[0012] Where x(n) represents the result of the image after n iterations of enhancement, n represents the number of iterations of dynamic random resonance, a and b are two parameters in the bistable system, Δt represents the step size of the iteration, and Input=f(t)+ξ(t) represents the noise driving force, i.e. the original image data;

[0013] Step 3: Build the U-Net network model and train the network model using the enhanced training set obtained in Step 2;

[0014] Step 4: Use the enhanced test set obtained in Step 2 to test the network model and complete the model building.

[0015] In one possible design, the specific process of enhancing CT images through dynamic stochastic resonance in step 2 is as follows:

[0016] S21, scan the small bowel CT images in the dataset line by line, and scan the images into a one-dimensional sequence x(n);

[0017] S22, set Δt and n to a fixed value, and set a = 0; take a step size of 0.001, let b traverse [0,20]. Each time b changes, substitute x(n) in S21 into formula (1) for iterative processing to obtain the enhanced x(n+1). Calculate and record the Pearson correlation coefficient between x(n) and x(n+1):

[0018]

[0019] Where r is the Pearson correlation coefficient. Let x(n) and x(n+1) represent the mean values ​​of x(n) and x(n+1), respectively.

[0020] S23, take a step size of 0.001, let a traverse [0,20]. Each time a changes, substitute x(n) in S21 into formula (1) for iterative processing to obtain the enhanced x(n+1). Calculate and record the Pearson correlation coefficient between x(n) and x(n+1).

[0021] S24, take the x(n+1) corresponding to the highest value of Pearson correlation coefficient recorded in S22 and S23 as the result of dynamic stochastic resonance enhancement;

[0022] S25: Reconstruct the enhanced one-dimensional x(n+1) into an enhanced two-dimensional small bowel CT image.

[0023] In one possible design, the U-Net network model built in step 3 consists of a compressed channel on the left and an expanded channel on the right. The compressed channel is a typical convolutional neural network structure, which repeatedly uses two convolutional layers and one max pooling layer. After each pooling operation, the dimension of the feature map doubles. The expanded channel first performs one deconvolution operation to halve the dimension of the feature map, then concatenates the feature maps cropped from the corresponding compressed channel to form a new feature map that is twice the size. Then, two convolutional layers are used for feature extraction, and this structure is repeated. In the final output layer, two convolutional layers are used to map the 16-dimensional feature map into a 2-dimensional output map.

[0024] In one possible design, the U-Net network model embeds an attention module at the bottom layer of the U-shaped network structure, i.e., before the first upper convolutional layer. The specific structure of the attention module includes: a 1×1 convolutional layer, a random dropout layer, a spatial max pooling layer, an inter-class averaging layer, and a scaling layer. The 1×1 convolutional layer is used to compress the data, the random dropout layer is used to discard unimportant features, the spatial max pooling layer is used to restore the data to a three-dimensional structure, the inter-class averaging layer is used to find the relationship between features of different classes, and the scaling layer is used to adjust the weights of the input features based on the inter-class relationship.

[0025] In one possible design, a batch normalization module is added to each convolutional layer of the U-Net network model. The specific steps for batch normalization are as follows:

[0026] S1, calculate the mean and variance of the input data;

[0027] S2, based on the mean and variance obtained from S1, normalizes the input data;

[0028] S3. Multiply the normalized data by a scaling factor and add a translation factor to obtain the batch normalization results of the data. The scaling factor and translation factor can be obtained through network training.

[0029] In one possible design, the specific process of testing the network model in step 4 is as follows:

[0030] S41, Traverse the CT images to be tested in the test set and normalize them;

[0031] S42, load the optimal model saved during training into the U-Net network model, and then use the U-Net network with the model parameters to predict the normalized test images in S41 one by one.

[0032] S43, Based on the segmentation results predicted by the network model, different colors are assigned to the lesion areas to distinguish different lesion conditions;

[0033] S44, Based on the segmentation prediction results of the network model and the lesion annotation images made by the doctor, calculate the IoU value of each image predicted by the network, specifically:

[0034] S441, iterate through each image with lesion annotations;

[0035] S442, Read the network prediction result image corresponding to the labeled image;

[0036] S443, calculate the range of intersection between the small bowel lesion labeled region T and the network model segmentation result region P, |T∩P|, where T represents the small bowel lesion labeled region and P is the segmentation result obtained by the model;

[0037] S444, calculate the range of the merged region |T∪P| between the small bowel lesion labeled region T and the network model segmentation result region P;

[0038] S445, thus calculating the IoU index value:

[0039] A second aspect of the present invention also provides a method for recognizing small bowel CT images, comprising the following steps:

[0040] First, the original CT image of the small intestine is acquired; then, the original CT image of the small intestine is enhanced by dynamic random resonance enhancement; the enhanced CT image of the small intestine is input into the recognition and segmentation network model constructed by the construction method described in the first aspect; finally, the CT image of the small intestine after processing by the network model is acquired.

[0041] A third aspect of the present invention also provides a device for small bowel CT image recognition and segmentation, the device comprising at least one processor and at least one memory coupled together; the memory stores a program of a recognition and segmentation network model constructed by the construction method described in the first aspect; when the processor executes the program stored in the memory, it can realize the recognition and segmentation of small bowel CT images.

[0042] A fourth aspect of the present invention also provides a computer-readable storage medium storing a computer program or instructions for a recognition and segmentation network model constructed by the construction method described in the first aspect, wherein the program or instructions, when executed by a processor, can realize the recognition and segmentation of small bowel CT images.

[0043] Compared with existing technologies, this invention provides a method for constructing a segmentation network model for small bowel CT images and a recognition method, which greatly improves the segmentation and recognition effect and produces the following beneficial effects:

[0044] 1. To address the problem of unclear lesion areas in small bowel CT images, this invention proposes a method based on dynamic stochastic resonance enhancement of CT images, which can effectively enhance useful signals (especially lesion tissue areas) in the original image and improve contrast.

[0045] 2. Since medical image processing is mainly used to assist doctors in diagnosis, the input is an image and the output is also an image that is easier for doctors to use. This invention establishes a U-shaped neural network that meets this requirement.

[0046] 3. In order to improve the network training efficiency and recognition accuracy, this invention embeds an attention module into the U-Net network to enable the network to focus on specific target features of CT images and suppress irrelevant features. At the same time, a batch normalization module is added to alleviate the problems of slow convergence speed or gradient explosion during training. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the method for constructing a segmentation network model according to the present invention.

[0048] Figure 2 This is a schematic diagram of a bistable double-potential-well stochastic resonance, represented by U(t).

[0049] Figure 3This is a schematic diagram comparing the original and enhanced CT images of the small intestine of the same patient.

[0050] Figure 4 This is a schematic diagram of the U-Net network structure used for segmentation and recognition of small bowel CT images.

[0051] Figure 5 A schematic diagram of the structure for adding an attention module to a U-Net network.

[0052] Figure 6 A schematic diagram of the U-Net network structure for embedding the attention mechanism and batch normalization module.

[0053] Figure 7 This image shows an enhanced CT scan of the small intestine and the segmentation and recognition results of a patient with small bowel lymphoma.

[0054] Figure 8 Image of the small intestine after CT enhancement and segmentation recognition results for a patient with inflammatory bowel disease.

[0055] Figure 9 This image shows an enhanced CT scan of the small intestine and the segmentation and recognition results of a patient with non-lymphoma of the small intestine.

[0056] Figure 10 This is a simplified block diagram of the device structure for small bowel CT image recognition and segmentation according to the present invention. Detailed Implementation

[0057] The invention will be further described below with reference to specific embodiments.

[0058] Example 1:

[0059] like Figure 1 As shown, this invention provides a method for constructing a small bowel CT image recognition and segmentation network model, mainly including the following steps:

[0060] Step 1: Obtain several CT images of the small intestine showing lesions and build an image dataset. Divide the dataset into a training set and a test set according to the proportions.

[0061] Step 2: Enhance the image dataset from Step 1 using dynamic stochastic resonance enhancement; the iterative formula for dynamic stochastic resonance is:

[0062] x(n+1)=x(n)+Δt[(ax(n)-bx 3 (n))+Input]

[0063] Where x(n) represents the result of the image after n iterations of enhancement, n represents the number of iterations of dynamic random resonance, a and b are two parameters in the bistable system, Δt represents the step size of the iteration, and Input=f(t)+ξ(t) represents the noise driving force, i.e. the original image data;

[0064] Step 3: Build the U-Net network model and train the network model using the enhanced training set obtained in Step 2;

[0065] Step 4: Use the enhanced test set obtained in Step 2 to test the network model and complete the model building.

[0066] 1. Obtain the dataset.

[0067] The CT image dataset used in the experiment came from the Affiliated Hospital of Qingdao University and was accurately labeled by professional and experienced radiologists. The dataset includes CT images of 140 patients with small bowel diseases, including 30 cases of small bowel lymphoma, 50 cases of inflammatory bowel disease, and 60 cases of non-lymphoma small bowel diseases. The abdominal tomographic images of these patients are a series of slices stacked along the z-axis, each slice with a resolution of 512×512 pixels and each pixel being 16 bits. One arterial and one venous CT image were taken from each patient in the experiment, for a total of 280 small bowel CT images.

[0068] 2. Dynamic random resonance enhanced small bowel CT images.

[0069] In 1981, Benzi discovered that under the influence of short-term climate change, the Earth's glacial changes are synchronized with weak periodic perturbations in solar radiation, a phenomenon he named stochastic resonance (SR). The theory of stochastic resonance demonstrates that, under certain conditions, noise can improve the output performance of nonlinear systems.

[0070] The classical theory of stochastic resonance can be described by the motion of overdamped particles between bistable potential traps:

[0071]

[0072] In the formula, t is time. ξ(t) is the rate of change of the ball's position, f(t) is the external periodic signal input to the system, ξ(t) is the noise in the image, and U(t) is the potential function, which is often represented by a bistable fourth-order potential well function in stochastic resonance theory, and its expression is shown in equation (2):

[0073]

[0074] This potential well has two locations located at A stable state, an unstable state at x=0, such as Figure 2As shown. Between stable and unstable states, there exists... The height of the barrier.

[0075] When no external periodic driving signal is applied to the system, the particles in the bistable system will oscillate slightly near the steady state, with the oscillation intensity proportional to the noise intensity. When a periodic signal is introduced into the system, the symmetrical bistable well begins to oscillate with the periodic signal; that is, under the excitation of the driving signal, the particles begin to oscillate violently in the non-uniform wells on both sides. However, because the amplitude of the driving signal is relatively small, the particle's potential cannot drive the particle through the potential barrier. By appropriately adding noise to the periodic signal, the particles can gain enough energy to pass through the potential barrier. In this way, the original chaotic noise, after being utilized by the bistable potential well, will output some statistical characteristics in the form of particle motion states. The bistable system completes the process of using noise energy to enhance a weak signal.

[0076] When formula (1) is used for image processing, it needs to be rewritten as the following iterative formula for Dynamic Stochastic Resonance (DSR):

[0077] x(n+1)=x(n)+Δt[(ax(n)-bx 3 (n))+Input] (3)

[0078] Where Δt represents the iteration step size, and Input = f(t) + ξ(t) represents the noise driving force. Input can be a noisy image, or the wavelet coefficients, discrete cosine coefficients, and Fourier coefficients of the noisy image during wavelet transform, discrete cosine transform, and Fourier transform. For small bowel CT image enhancement, Input can be the original CT image data.

[0079] The specific process of enhancing CT images using dynamic stochastic resonance is as follows:

[0080] S21, scan the small bowel CT images in the dataset line by line, and scan the images into a one-dimensional sequence x(n);

[0081] S22, set Δt and n to a fixed value, and set a = 0; take a step size of 0.001, let b traverse [0,20]. Each time b changes, substitute x(n) in S21 into formula (1) for iterative processing to obtain the enhanced x(n+1). Calculate and record the Pearson correlation coefficient between x(n) and x(n+1):

[0082]

[0083] Where r is the Pearson correlation coefficient. Let x(n) and x(n+1) represent the mean values ​​of x(n) and x(n+1), respectively.

[0084] S23, take a step size of 0.001, let a traverse [0,20]. Each time a changes, substitute x(n) in S21 into formula (1) for iterative processing to obtain the enhanced x(n+1). Calculate and record the Pearson correlation coefficient between x(n) and x(n+1).

[0085] S24. The x(n+1) corresponding to the highest Pearson correlation coefficient recorded in S22 and S23 is taken as the result of dynamic stochastic resonance enhancement. Because, when Δt and n are fixed values, the higher the Pearson correlation coefficient between x(n) and x(n+1), the better the enhancement effect of dynamic stochastic resonance. Therefore, a and b corresponding to the highest Pearson correlation coefficient can be regarded as the optimal parameters of the dynamic stochastic resonance system at this time, and the x(n+1) corresponding to this parameter is the optimal result of dynamic stochastic resonance enhancement.

[0086] S25: Reconstruct the enhanced one-dimensional x(n+1) into an enhanced two-dimensional small bowel CT image.

[0087] Figure 3 (a) and (b) are CT images of veins and arteries in the small intestine, respectively. (c) and (d) are images of (a) and (b) after dynamic random resonance processing, respectively, showing significant enhancement.

[0088] 3. Build the U-Net network model and train the network model.

[0089] U-shaped network structures, with their advantages of contextual information combination, fast training speed, and small data requirements, meet the technical requirements of medical image segmentation and have therefore been widely used in medical image analysis. However, due to the large amount of memory required during computation, a complete CT image needs to be segmented into a series of 3D blocks for input, which leads to the loss of some global information. Although significant research results have been achieved in the image segmentation of important organs such as the heart, kidneys, and lungs, progress in CT image recognition and segmentation of small intestinal lesions has been minimal due to the difficulty in distinguishing the characteristics of certain small intestinal lesions, such as inflammatory bowel disease and small intestinal non-lymphoma, and the relatively large range of images that need to be analyzed and processed. Therefore, this invention constructs a U-Net network that can directly input the entire small intestinal CT image.

[0090] like Figure 4As shown, it consists of a compressed channel on the left and an expanded channel on the right. The compressed channel follows a typical convolutional neural network structure, repeatedly employing two convolutional layers and one max-pooling layer, doubling the dimension of the feature map after each pooling operation. In the expanded channel, a deconvolution operation is performed first, halving the dimension of the feature map. Then, the feature maps cropped from the compressed channel are concatenated to form a new feature map twice the size. Two convolutional layers are then used for feature extraction, and this structure is repeated. In the final output layer, two convolutional layers map the 16-dimensional feature map into a 2-dimensional output map.

[0091] Figure 4 The end-to-end U-shaped segmentation network structure can be divided into two stages: upsampling and downsampling. Downsampling improves robustness to small perturbations in the input image (such as image translation and rotation), reduces the risk of overfitting, reduces computational cost, and increases the receptive field. The most significant effect of upsampling is decoding abstract features to the size of the original image, ultimately yielding the segmentation result. The encoder and decoder of the U-Net network are completely symmetrical, using the same number of convolutional layers. Therefore, the U-Net network structure can be interpreted as a shrinking path and an expanding path. The shrinking path is used to extract image features, capture context, and compress the image into a feature map composed of features. The expanding path is used for precise localization, decoding the extracted features into a segmentation prediction image of the same size as the original image.

[0092] (1) Embed attention mechanism and batch normalization module

[0093] To further improve the training efficiency of the network, this invention adds an attention module and a batch normalization module to the U-Net network.

[0094] The core objective of the attention mechanism is to enable the network to focus on key information relevant to the task, thereby improving training efficiency and recognition accuracy. In other words, the attention mechanism model can select a small amount of information that is more critical to the current task objective from a large amount of data, and then focus on this key data during network training, ignoring or weakening relatively unimportant information. The attention mechanism optimizes traditional search methods by adjusting the network's visual processing algorithm. In this process, the focus is reflected in the calculation of weight coefficients. The larger the weight, the more attention the corresponding data receives. In short, the weight represents the importance of the information, and the value is the corresponding data. This method not only reduces the amount of sample data to be processed but also increases the possibility of feature matching between samples. The attention module embedded in this invention is as follows... Figure 5 As shown.

[0095] In this invention, the specific structure of the attention module includes: a 1×1 convolutional layer, a random dropout layer, a spatial max pooling layer, an inter-class averaging layer, and a scaling layer. The 1×1 convolutional layer is used to compress the data, the random dropout layer is used to discard unimportant features, the spatial max pooling layer is used to restore the data to a three-dimensional structure, the inter-class averaging layer is used to find the relationships between features of different classes, and the scaling layer is used to adjust the input feature weights based on the inter-class relationships.

[0096] (2) Add batch standardization module

[0097] During neural network training, the distribution of input values ​​in each layer is influenced by the previous layer. Therefore, the initialization of hyperparameters and the setting of the learning rate determine the convergence time and training time of the network. Batch processing standardizes the inputs of each layer in the network so that the processed inputs exhibit a standard normal distribution with a variance of 1 and a mean of 0. This process can significantly shorten the training time. In other words, batch standardization allows the model to choose a higher learning rate, thereby accelerating the convergence speed of the network and achieving efficient information processing.

[0098] U-Net networks with embedded attention mechanisms and batch normalization modules, such as Figure 6 As shown.

[0099] Assume the input is x1, x2, ..., x m There are a total of m items, and the batch standardization process can be divided into the following three steps.

[0100] The first step is to calculate the mean μ and variance σ of each batch of input training samples. 2 :

[0101]

[0102] The second step is to normalize the training samples for each batch of input based on the mean and variance obtained in the previous step:

[0103]

[0104] Here, ε is a small positive number used to prevent the denominator from being zero.

[0105] The third step is to calculate the batch standardization results:

[0106] Here, α represents the scaling factor and β represents the translation factor. These two parameters can be obtained through network training.

[0107] During training, the relevant parameters in the U-Net network are set as follows:

[0108]

[0109]

[0110] A total of 140 arterial images and 140 venous images of the small intestine were obtained after dynamic random resonance enhancement. 80% of each image was used as training data.

[0111] Training process:

[0112] The CT images in the training set are traversed and normalized.

[0113] Configure U-Net model parameters, i.e., set the ModelCheckpoint function parameters, including specifying the save address of the network model during training, monitoring the loss parameters, setting the control panel output type to berbose=1 (i.e., the control panel outputs input log information with a progress bar), and setting save_best_only to True, which means that only the best-performing model on the training set is saved after each epoch of training.

[0114] Configure parameters for TensorFlow's visualization tool, TensorBoard, including specifying the file paths for TensorBoard logs and other files, and setting the frequency of calculating activation values ​​and model weight histograms for each layer in the model.

[0115] Training data is fed into the U-Net network in batches according to the batch size (batch_size=16) to begin training. The generated model is saved along with the visualization file according to the parameters set in steps b) and c). The network model training ends after 1000 epochs.

[0116] Launch the Tsensorboard application in your browser to view the changes in the number of training data parameters during model training.

[0117] Training program runtime environment:

[0118] GPU: NVIDIA GeForce GTX 1080 Ti (using CUDA 10.1 to accelerate network model training)

[0119] Operating system: Ubuntu 16.04.7 LTS

[0120] Programming language: Python 3.7

[0121] Development environment: PyCharm 2020, Anaconda

[0122] Deep learning framework: Keras 2.3.1

[0123] 4. Test the network model and complete its construction.

[0124] To better evaluate and analyze the effectiveness of these mechanisms in identifying and segmenting small intestinal lesions, it is necessary to define the evaluation metrics and methods in the experiment. There are four possible scenarios: true positive (TP), false positive (FP), false negative (FN), and true negative (TN). TP indicates that it is a small intestinal disease, and the model also predicts it as a true disease. FP means that the model judges it to be a true disease, but it is not. FN indicates that the model determines it to be a healthy state, but the opposite is true. TN means that it is not a disease, and the model correctly predicts it. This invention uses the Joint Intersection (IoU), a segmentation metric commonly used in medical images, to evaluate the segmentation and recognition performance.

[0125]

[0126] Where T represents the label segmentation of small bowel lesions, and P is the segmentation result obtained through the model, both ranging from [0,1].

[0127] The specific process for testing the network model is as follows:

[0128] S41, Traverse the CT images to be tested in the test set and normalize them;

[0129] S42, load the optimal model saved during training into the U-Net network model, and then use the U-Net network with the model parameters to predict the normalized test images in S41 one by one.

[0130] S43, Based on the segmentation results predicted by the network model, different colors are assigned to the lesion areas to distinguish different lesion conditions;

[0131] S44, Based on the segmentation prediction results of the network model and the lesion annotation images made by the doctor, calculate the IoU value of each image predicted by the network, specifically:

[0132] S441, iterate through each image with lesion annotations;

[0133] S442, Read the network prediction result image corresponding to the labeled image;

[0134] S443, calculate the range of intersection between the small bowel lesion labeled region T and the network model segmentation result region P, |T∩P|, where T represents the small bowel lesion labeled region and P is the segmentation result obtained by the model;

[0135] S444, calculate the range of the merged region |T∪P| between the small bowel lesion labeled region T and the network model segmentation result region P;

[0136] S445, thus calculating the IoU index value:

[0137] Figure 7 The images show the enhanced CT images and segmentation results of the small intestine of a patient with small bowel lymphoma. (a) Enhanced CT image of veins, (b) Enhanced CT image of arteries. (c) shows the small bowel lymphoma lesion area in (a), with IoU = 0.9923; (d) shows the small bowel lymphoma lesion area in (b), with IoU = 0.9612.

[0138] Figure 8 The image shows the enhanced CT image and segmentation results of the small intestine of a patient with inflammatory bowel disease. (a) is the enhanced CT image of the vein, and (b) is the enhanced CT image of the artery. (c) shows the inflammatory bowel disease lesion area in (a), with IoU = 0.9808; (d) shows the inflammatory bowel disease lesion area in (b), with IoU = 0.9692.

[0139] Figure 9 The images show the enhanced CT images and segmentation results of the small intestine in a patient with non-lymphoma of the small intestine. (a) Enhanced CT image of the vein, (b) Enhanced CT image of the artery. (c) shows the non-lymphoma lesion area in the small intestine of (a), with IoU = 0.9985; (d) shows the non-lymphoma lesion area in the small intestine of (b), with IoU = 0.9967.

[0140] The comparison shows that the segmentation and recognition effect has been greatly improved.

[0141] The experimental environment for this invention was built on Ubuntu 16.04. The graphics accelerator card was an Nvidia GTX 2080 Ti 11GB, the framework was Keras, and the Python version was 3.6. Other parameters are shown in Table 1. The training and testing times for U-Net directly were 123.5 minutes and 2.38 minutes, respectively. The training and testing times for U-Net with embedded attention mechanism and batch normalization module were 98.15 minutes and 1.75 minutes, respectively.

[0142] Experimental parameter settings

[0143] Item Parameters Learning rate 0.002 Optimization Adam Number of iterations 1000 epochs Loss function categorical_crossentrop

[0144] After the small bowel CT image recognition and segmentation network model is built, its application in a real-world scenario includes the following steps: First, acquire the original small bowel CT image; then, enhance the original small bowel CT image using dynamic random resonance enhancement; input the enhanced small bowel CT image into the recognition and segmentation network model built by the construction method described in any one of claims 1 to 6; finally, acquire the small bowel CT image after processing by the network model.

[0145] The final obtained small bowel CT images are used by doctors for diagnosis.

[0146] Example 2:

[0147] like Figure 10 As shown, this invention also provides a device for small bowel CT image recognition and segmentation. The device includes at least one processor and at least one memory, as well as a communication interface and an internal bus. The memory stores a program for a recognition and segmentation network model constructed using the method described in Embodiment 1. When the processor executes the program stored in the memory, it can realize the recognition and segmentation of small bowel CT images. The internal bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus in the accompanying drawings is not limited to only one bus or one type of bus. The memory may include high-speed RAM memory, and may also include non-volatile memory (NVM), such as at least one disk storage device, or it can be a USB flash drive, portable hard drive, read-only memory, disk, or optical disk, etc.

[0148] The device can be provided as a terminal, server, or other form of device.

[0149] Figure 10 This is a block diagram illustrating an illustrative device. The device may include one or more of the following components: a processing component, a memory, a power supply component, a multimedia component, an audio component, an input / output (I / O) interface, a sensor component, and a communication component. The processing component typically controls the overall operation of the electronic device, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. The processing component may include one or more processors to execute instructions to perform all or part of the steps of the methods described above. Furthermore, the processing component may include one or more modules to facilitate interaction between the processing component and other components. For example, the processing component may include a multimedia module to facilitate interaction between the multimedia component and the processing component.

[0150] Memory is configured to store various types of data to support the operation of electronic devices. Examples of this data include instructions for any application or method used to operate on an electronic device, contact data, phonebook data, messages, pictures, videos, etc. Memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0151] A power supply component provides power to various components of an electronic device. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device. A multimedia component includes a screen that provides an output interface between the electronic device and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component includes a front-facing camera and / or a rear-facing camera. When the electronic device is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0152] The audio component is configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the electronic device is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals may be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals. The I / O interface provides an interface between the processing component and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, volume buttons, a power button, and a lock button.

[0153] The sensor assembly includes one or more sensors for providing state assessments of various aspects of the electronic device. For example, the sensor assembly can detect the on / off state of the electronic device, the relative positioning of components such as the display and keypad of the electronic device, changes in the position of the electronic device or a component of the electronic device, the presence or absence of user contact with the electronic device, the orientation or acceleration / deceleration of the electronic device, and temperature changes of the electronic device. The sensor assembly may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0154] The communication component is configured to facilitate wired or wireless communication between electronic devices and other devices. The electronic device can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0155] In an exemplary embodiment, the electronic device may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0156] Example 3:

[0157] The present invention also provides a non-volatile computer-readable storage medium storing a computer program or instructions for a recognition and segmentation network model constructed by the construction method described in Embodiment 1. When the computer executes the program or instructions, the computer executes them to realize the recognition and segmentation of small bowel CT images.

[0158] Specifically, a system, apparatus, or device may be provided equipped with a readable storage medium on which software program code implementing the functions of any of the embodiments described above is stored, and the computer or processor of the system, apparatus, or device reads and executes the instructions stored in the readable storage medium. In this case, the program code read from the readable medium itself can implement the functions of any of the embodiments described above, therefore, the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0159] The aforementioned storage media can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disks or optical disks (such as CD-ROM, CD-R, CD-RW, DVD-20ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tape, etc. The storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0160] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0161] It should be understood that the storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, the processor and storage medium can exist as discrete components in a terminal or server.

[0162] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0163] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0164] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0165] While the specific embodiments of the present invention have been described above, they are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for constructing a small bowel CT image recognition and segmentation network model, characterized in that, Includes the following steps: Step 1: Obtain several CT images of the small intestine showing lesions and build an image dataset. Divide the dataset into a training set and a test set according to the proportions. Step 2: Enhance the image dataset from Step 1 using dynamic stochastic resonance enhancement; the iterative formula for dynamic stochastic resonance is: （1） in Indicates that the image has passed through The result after the next iteration of enhancement This represents the number of iterations for dynamic stochastic resonance. and These are two parameters in a bistable system. Indicates the step size of the iteration. This represents the noise driving force, i.e., the raw image data; The specific process is as follows: S21, Perform line-by-line scanning of the small bowel CT images in the dataset, scanning the images into a one-dimensional sequence. ; S22, Settings and Let it be a fixed value. ; Let the step size be 0.001, and let b traverse the range [0, 20]. Each time b changes, change the value in S21. Substituting into formula (1) and performing iterative processing, we obtain the enhanced... ,calculate and Record the Pearson correlation coefficient between them: in The Pearson correlation coefficient is used. , They represent , The mean; S23, take a step size of 0.001, let Traverse the [0,20] region. Each time it is changed, the value in S21 will be... Substituting into formula (1) and performing iterative processing, we obtain the enhanced... ,calculate and Record the Pearson correlation coefficient between them; S24 corresponds to the highest Pearson correlation coefficient recorded in S22 and S23. As a result of dynamic stochastic resonance enhancement; S25: The enhanced one-dimensional Restored to an enhanced 2D small bowel CT image; Step 3: Build the U-Net network model and train the network model using the enhanced training set obtained in Step 2; The U-Net network model consists of a compressed channel on the left and an expanded channel on the right. The compressed channel is a typical convolutional neural network structure, repeatedly using two convolutional layers and one max-pooling layer. After each pooling operation, the dimension of the feature map doubles. The expanded channel first performs a deconvolution operation to halve the dimension of the feature map, then concatenates the feature maps cropped from the compressed channel to form a new feature map twice the size. Two convolutional layers are then used for feature extraction, and this structure is repeated. In the final output layer, two convolutional layers map the 16-dimensional feature map into a 2-dimensional output map. The U-Net network model embeds an attention module at the bottom layer of the U-shaped network structure, before the first upper convolutional layer. The specific structure of the attention module includes: a 1×1 convolutional layer, a random dropout layer, a spatial max pooling layer, an inter-class averaging layer, and a scaling layer. The 1×1 convolutional layer is used to compress the data, the random dropout layer is used to discard unimportant features, the spatial max pooling layer is used to restore the data to a three-dimensional structure, the inter-class averaging layer is used to find the relationship between features of different classes, and the scaling layer is used to adjust the weights of the input features based on the inter-class relationship. Each convolutional layer in the U-Net network model incorporates a batch normalization module. The specific steps for batch normalization are as follows: S31, Calculate the mean and variance of the input data; S32, normalize the input data based on the mean and variance obtained in S31; S33, multiply the normalized data by a scaling factor and add a shift factor to obtain the batch standardization result of the data. The scaling factor and shift factor are obtained through network training. Step 4: Use the enhanced test set obtained in Step 2 to test the network model and complete the model building.

2. The method for constructing a small bowel CT image recognition and segmentation network model as described in claim 1, characterized in that, The specific process of testing the network model in step 4 is as follows: S41, Traverse the CT images to be tested in the test set and normalize them; S42, load the optimal model saved during training into the U-Net network model, and then use the U-Net network with the model parameters to predict the normalized test images in S41 one by one. S43, Based on the segmentation results predicted by the network model, different colors are assigned to the lesion areas to distinguish different lesion conditions; S44, Based on the segmentation prediction results of the network model and the lesion annotation images made by the doctor, calculate the IoU value of each image predicted by the network, specifically: S441, iterate through each image with lesion annotations; S442, Read the network prediction result image corresponding to the labeled image; S443, Calculate the labeled area of ​​small bowel lesions. Segmentation results region with network model Intersecting range ,in The marked area indicates small bowel lesions. It is the segmentation result obtained through the model; S444, Calculate the labeled area of ​​small bowel lesions. Segmentation results region with network model Scope of merger ; S445, thus calculating the IoU index value: .

3. A method for recognizing small bowel CT images, characterized in that, Includes the following steps: First, the original CT image of the small intestine is acquired; then, the original CT image of the small intestine is enhanced by dynamic random resonance enhancement; the enhanced CT image of the small intestine is input into the recognition and segmentation network model constructed by the construction method described in claim 1 or 2; finally, the CT image of the small intestine after processing by the network model is acquired.

4. A device for small bowel CT image recognition and segmentation, characterized in that: The device includes at least one processor and at least one memory, the processor and the memory being coupled together; the memory stores a program for a recognition and segmentation network model constructed by the construction method as described in claim 1 or 2; when the processor executes the program stored in the memory, it can realize the recognition and segmentation of small bowel CT images.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or instructions for a recognition and segmentation network model constructed by the construction method as described in claim 1 or 2, which, when executed by a processor, can realize the recognition and segmentation of small bowel CT images.

Images