Artificial intelligence-assisted ultrasound diagnosis of spleen trauma

Abstract

The present invention relates to a method for artificial intelligence-assisted ultrasonic diagnosis of spleen trauma, and belongs to the field of artificial intelligence. The present invention connects a medical imaging device to a portable device to obtain an ultrasonic image of the spleen of a patient with spleen trauma; performs sampling and frame extraction processing on the ultrasonic image video stream to generate a video image frame sequence; uses a spleen trauma segmentation model to predict the video image frame sequence to obtain a corresponding segmentation prediction result sequence; uses a spleen trauma grade classification model to predict to obtain a spleen trauma grade classification label corresponding to the ultrasound image; and outputs the corresponding visualized segmentation classification result. The present invention improves segmentation precision and accuracy, greatly reduces data set storage space, makes full use of the correlation between information between images, obtains sufficient context information, and improves the accuracy of trauma grade classification.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence, and in particular relates to a method for artificial intelligence-assisted ultrasonic diagnosis of spleen trauma. Background Art

[0002] In trauma treatment and rescue work, saving lives is the top priority, and timeliness, accuracy and professionalism are required. Transferring the injured within the "golden hour" and quickly conducting medical intervention are important guarantees for saving the injured. However, if such attacks occur in crowded places in large and medium-sized cities, traffic congestion is an unavoidable problem for emergency rescue. Once professional medical forces, vehicles and equipment cannot enter, the injured are "difficult to sort" and "slow to transfer", which will directly affect the timeliness of treatment; if it occurs in remote areas far away from urban areas, timely medical treatment becomes a problem; at the same time, the proficiency of medical personnel rushing to the scene in using ultrasound equipment may also affect the accuracy of diagnosis. Therefore, rapid, accurate and simple injury assessment of the injured is conducive to saving lives to the greatest extent.

[0003] Abdominal trauma is a common injury. Since some patients with closed abdominal injuries do not have obvious early symptoms, they are often accompanied by serious organ problems, which requires more timely and accurate diagnosis. The spleen is the most vulnerable organ due to its fragile texture and rich blood supply. Splenic trauma accounts for 13-25% of abdominal multiple injuries. If not diagnosed and treated in time, it may endanger the patient's life. In the past, surgical procedures such as diagnostic peritoneal lavage and laparotomy were usually used to diagnose abdominal injuries. However, this often causes unnecessary trauma to some hemodynamically stable patients and increases the risk of complications such as infection. At present, preliminary screening of imaging has become the main way to quickly assess the injury. Imaging examination methods mainly include CT examination and ultrasound examination. The advantage of CT examination is that it has a high compliance rate and is not affected by gas interference, patient breathing, etc. during the examination. However, due to its instrument conditions, CT cannot be moved at will, so it cannot be used to scan patients at the pre-hospital and emergency bedside. In addition, CT is radioactive, and children and pregnant women will increase the risk of exposure to radiation, and the examination is limited. The thickness of the CT scanning layer may also miss some tiny lesions. Therefore, CT scanning has limitations. Ultrasound can be used for rapid injury assessment in pre-hospital and emergency rooms because of its convenient operation, portability, and low difficulty. Ultrasound can determine whether there is organ damage by observing the continuity of the spleen capsule, the presence of subcapsular hematoma, and whether the echo of the parenchyma is uniform. At the same time, ultrasound can also infer whether there is organ damage through some indirect signs, such as retroperitoneal hematoma and peritoneal effusion. Color Doppler ultrasound can detect abnormal blood supply in the lesion. However, the diagnostic sensitivity of conventional ultrasound for solid organ injury is only about 41%. Contrast-enhanced ultrasound (CEUS) can detect solid organ damage and active bleeding from multiple abdominal injuries, improve the accuracy of ultrasound examination of spleen injury, provide a more reliable means of evaluating solid organ injury, and make up for the shortcomings of conventional ultrasound. Studies have shown that CEUS is as accurate as CT in the detection and staging of traumatic spleen injury. However, ultrasound contrast is an invasive examination, and the cost of contrast agents is expensive, so the examination has limitations.

[0004] As a strategic technology leading the future, artificial intelligence is increasingly being used in various fields of society. In 2017, my country issued documents such as the "Plan for the Development of a New Generation of Artificial Intelligence" and the "Three-Year Action Plan for Promoting the Development of a New Generation of Artificial Intelligence Industry (2018-2020)" to promote the research and development and industrialization of artificial intelligence technology. In 2016, the world's first artificial intelligence ultrasound product series was released, and artificial intelligence began to be applied to the diagnosis of thyroid nodules, which can improve the accuracy of thyroid nodule diagnosis. Deep learning, as one of the current popular analysis methods of machine learning, can represent and learn massive data by building a multi-layer artificial neural network. The rapid growth of graphics processing capabilities has made it possible to develop the most advanced algorithms, which have more stable and powerful image analysis capabilities. Therefore, it has been widely used in the diagnosis and identification of ultrasound images of various organs and has been proven to have a high accuracy rate. At present, there are few reports on artificial intelligence-assisted ultrasound diagnosis of spleen trauma. This study uses deep learning to establish an artificial intelligence spleen trauma segmentation and classification model, and realizes a lightweight spleen trauma ultrasound image segmentation and classification system that obtains segmentation and classification prediction results from ultrasound images.

[0005] Maorui He et al. proposed a technical solution for the classification and diagnosis algorithm of spleen injury ultrasound images based on a double-channel convolutional neural network in the article "Ultrasonic Image Diagnosis of Liver and Spleen InjuryBased on a Double-Channel Convolutional Neural Network". First, the anisotropic diffusion denoising model is used to implement data preprocessing of spleen ultrasound images to improve the image quality of ultrasound images. Secondly, the lesion location is detected to obtain external edge features, and the extracted rotation-invariant local binary pattern features of the image are used as the internal texture features of the image. Finally, the double-channel convolutional neural network uses the texture information and edge information of the damaged image as two input channels to classify and identify spleen injury ultrasound images.

[0006] Step 1: Image preprocessing: Due to the limitations of the imaging mechanism of medical ultrasound images, the spleen ultrasound image has serious noise interference, mainly manifested as additive thermal noise and multiplicative speckle noise. Therefore, it is necessary to denoise the spleen ultrasound image.

[0007] Step 2: Extract texture features: The shallow texture feature values ​​of the input image are obtained by calculating the histogram of the local binary pattern translation values; the deep features of the image can be obtained by inputting the spleen ultrasound image to be processed into GoogLeNet. The deep and shallow features are integrated, normalized and fused respectively to obtain the internal texture features.

[0008] Step 3: Extract edge information: Use the improved Canny edge detection method to detect horizontal, vertical, 45° and 135° contour edges and extract external edge features.

[0009] Step 4: Classify and identify spleen injury ultrasound images: In clinical diagnosis, the outer edge information and inner texture information of the spleen are often used as important bases for doctors' diagnosis. Therefore, this paper uses the outer edge features and inner texture features as two input channels of the convolutional neural network. Then, combined with the dynamic K-max pooling method, a spleen injury ultrasound image classification algorithm based on dual-channel CNN is proposed.

[0010] The network consists of two relatively independent network structures. Each network consists of an input layer, a convolution layer, a pooling layer, and a fully connected layer. In the input layer, the external edge features and the internal texture features are used as input information of the dual-channel convolutional neural network respectively. In the first layer of the convolution layer, the size of the convolution kernel is set according to the features, so the two networks usually use convolution checks of different sizes to extract the two input layers. After the pooling layer, the two channels are connected to a fully connected layer for full connection mapping, and then the two fully connected layers are combined through a fully connected layer, and then the extracted features are classified using the Softmax classifier.

[0011] There are several problems in the existing technical solutions that are not applicable to the scenarios required by the present invention;

[0012] 1. The output is the classification results of each level of trauma. If only a single image is analyzed, the image of the most seriously injured part may not be obtained, resulting in classification errors. At the same time, because multiple images are not combined for analysis, the correlation between image information is ignored, and sufficient context information is not obtained, resulting in insufficient feature extraction and inaccurate classification.

[0013] 2. In the scenario required by the present invention, the spleen organ and the location of the trauma need to be accurately located. The classification results of each level of trauma alone are not sufficient to provide sufficient anatomical structure and pathological information, and further screening and analysis require the doctor's energy.

[0014] 3. Using two feature extraction results as input information for the dual convolutional neural network takes a long time and has low performance:

[0015] In the existing technical solutions, the network complexity of the two feature extractions and the dual convolutional neural network is high, which reduces the computing performance. When processing ultrasound image video streams, the data volume is large, and the technical solution has a slow response speed and cannot make corresponding processing quickly. Summary of the invention

[0016] 1. Technical issues to be resolved

[0017] The technical problem to be solved by the present invention is how to provide an artificial intelligence-assisted ultrasonic diagnosis method for spleen trauma, so as to solve the problems that the existing technical solutions have high network complexity, reduced computing performance, large data volume, slow response speed and cannot make corresponding processing quickly when processing ultrasonic image video streams.

[0018] (II) Technical solution

[0019] In order to solve the above technical problems, the present invention proposes a method for artificial intelligence-assisted ultrasound diagnosis of spleen trauma, which comprises the following steps:

[0020] S1. Connect the medical imaging device to the portable device to obtain an ultrasound image of the spleen of the injured person;

[0021] S2, performing sampling and frame extraction processing on the ultrasound image video stream to generate a video image frame sequence;

[0022] S3, importing the video image frame sequence generated by each ultrasound image into the spleen trauma ultrasound image segmentation classification system, using the spleen trauma segmentation model to predict the video image frame sequence, and obtaining a corresponding segmentation prediction result sequence, wherein the segmentation prediction result sequence corresponding to each video image includes spleen contour and / or trauma contour position information;

[0023] S4, importing the segmentation prediction result sequence generated in step S3 into the spleen trauma ultrasound image segmentation classification system, using the spleen trauma grade classification model to perform prediction, and obtaining the spleen trauma grade classification label corresponding to the ultrasound image;

[0024] S5. The spleen trauma ultrasound image segmentation and classification system outputs the corresponding visualized segmentation and classification results.

[0025] Furthermore, before step S1, the method further includes: S0, training a spleen trauma segmentation model and a spleen trauma grade classification model in a spleen trauma ultrasound image segmentation classification system.

[0026] Furthermore, the step S0 specifically includes:

[0027] S0.1. The training ultrasound images include ultrasound videos of spleen trauma and normal spleen. Each training ultrasound image is processed by frame extraction to generate a set of training image frame sequences, corresponding to a set of manually annotated segmented image frame sequences and corresponding spleen trauma grade classification labels, which respectively constitute the original image data set, the segmentation label data set and the classification label data set; wherein the manually annotated segmented image frame sequence includes the spleen contour and / or trauma contour position information,

[0028] S0.2, training the spleen trauma segmentation model based on the original image dataset and segmentation label dataset obtained in step S0.1;

[0029] S0.3. Train the spleen trauma grade classification model based on the segmentation label dataset and classification label dataset obtained in step S0.1.

[0030] Furthermore, the spleen trauma segmentation model and spleen trauma grade classification model obtained by training in step S0 are respectively converted into model scripts through the TorchScript method, and the spleen trauma ultrasound image segmentation classification system loads the above scripts as prediction tools and outputs intuitive prediction results in a visual interface.

[0031] Furthermore, in step S2, OpenCV video processing technology is used to uniformly extract multiple frames of images of the corresponding video, and data preprocessing is performed to remove irrelevant information.

[0032] Furthermore, when extracting frames in step S2, a frame extraction scheme based on time interval, frame interval or inter-frame differential strength is adopted.

[0033] Furthermore, in step S3, the spleen trauma segmentation model adopts a residual network structure combined with a U-Net architecture, and the specific network structure is designed as follows:

[0034] S3.1, the input video image frame sequence is passed through a three-dimensional convolutional layer through an image scaling method to make its size and dimension the same as the feature map size and dimension of the corresponding decoding stage;

[0035] S3.2, sequentially passes through the three-dimensional convolution layer, BatchNorm normalization layer, ReLU activation function and maximum pooling layer, and then performs the supervision and loss calculation of the encoding stage (encoder). The encoding has four stages, including 3, 4, 6, and 3 encoding network structures respectively. The input of each stage is the down-sampling of the output result of the previous stage, and the output result of each stage is saved at the same time;

[0036] S3.3. Perform supervision and loss calculation in the decoding stage (decoder), and perform four-stage decoding. The first stage uses the output of the fourth encoding stage as input, and the second, third, and fourth stage decoding use the upsampling of the output result of the previous decoding stage and the output of the third, second, and first encoding stages as input respectively. Each decoding stage includes a convolutional layer, a BatchNorm normalization layer, and a ReLU activation function. After the fourth stage outputs the feature maps of each channel, the final result is convolved to obtain a sequence of segmentation prediction results.

[0037] Furthermore, the encoding network structure is a three-dimensional ResNet residual network structure, a two-dimensional combined three-dimensional ResNet residual network structure, a three-dimensional CNN network structure or an LSTM network structure.

[0038] Furthermore, in step S4, the network structure of the spleen trauma grade classification model includes: a first three-dimensional ResNet residual network structure, a second three-dimensional ResNet residual network structure and a Softmax classifier, the segmentation prediction result sequence is input into the first three-dimensional ResNet residual network structure, the downsampling encoding of the first three-dimensional ResNet residual network structure is input into the second three-dimensional ResNet residual network structure, and the downsampling encoding of the second three-dimensional ResNet residual network structure is input into the Softmax classifier. The extracted features are classified by the Softmax classifier to obtain the spleen trauma grade classification label corresponding to the input segmentation prediction result sequence.

[0039] Furthermore, the ResNet residual network structure includes one or more residual modules, and a short-circuit connection is added between the input and output of the residual module, that is, the output of the residual module is the result of the bitwise addition of the learning feature F(x) after the weight matrix and the input x.

[0040] (III) Beneficial effects

[0041] The present invention proposes a method for artificial intelligence-assisted ultrasound diagnosis of spleen trauma. The present invention trains model parameters through spleen trauma or normal spleen ultrasound video, has the ability to analyze ultrasound video streams, analyzes the video stream, and generates original image sequences that help to complete the features of a single image, so that the model fully understands the contextual information and improves the segmentation accuracy. At the same time, frame extraction processing can maximize the use of information in the video, greatly reducing the data set storage space. On this basis, it can also support the predictive segmentation of a single spleen trauma image.

[0042] The ultrasonic diagnosis of spleen trauma segmentation model in the present invention adopts a residual network structure combined with a U-Net architecture, which simultaneously obtains the advantages of the two network structures and improves the accuracy of lesion segmentation.

[0043] In the present invention, step S4 combines the segmentation results of multiple spleen trauma ultrasound images to perform comprehensive judgment and classification based on the ultrasound diagnosis spleen trauma grade classification model, making full use of the correlation between image information, obtaining sufficient context information, and improving the accuracy of trauma grade classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of the method of the present invention;

[0045] Figure 2 This is the data preprocessing flow chart;

[0046] Figure 3 This is the network structure diagram of the ultrasound diagnosis of spleen trauma segmentation model;

[0047] Figure 4A network structure diagram of spleen injury grade classification in a certain embodiment;

[0048] Figure 5 This is a schematic diagram of the spleen trauma ultrasound image segmentation and classification system;

[0049] Figure 6 is a schematic diagram of the residual structure;

[0050] Figure 7 Schematic diagram of U-Net network structure. DETAILED DESCRIPTION

[0051] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0052] The purpose of the present invention is to provide a deep learning-based spleen ultrasound image trauma diagnosis method, which analyzes the image video generated during the ultrasound diagnosis process, identifies the spleen anatomical structure contour and spleen trauma location that appear in the image sequence generated by the segmented video, and completes the classification of spleen trauma grade based on this, thereby realizing artificial intelligence ultrasound spleen trauma diagnosis. The present invention also provides a lightweight spleen trauma ultrasound image segmentation and classification system, which can be directly run on a portable device, is suitable for pre-hospital emergency scene environment, and outputs spleen segmentation visualization results and trauma classification results at the same time.

[0053] The spleen ultrasound imaging trauma diagnosis method based on deep learning of the present invention comprises the following steps:

[0054] S1. Connect the medical imaging device to the portable device to obtain ultrasound images of the spleen of patients with spleen trauma;

[0055] S2, performing sampling and frame extraction processing on the ultrasound image video stream to generate a video image frame sequence;

[0056] S3, importing the video image frame sequence generated by each ultrasound image into the spleen trauma ultrasound image segmentation classification system, using the spleen trauma segmentation model to predict the video image frame sequence, and obtaining a corresponding segmentation prediction result sequence, wherein the segmentation prediction result sequence corresponding to each video image includes spleen contour and / or trauma contour position information;

[0057] S4, importing the segmentation prediction result sequence generated in step S3 into the spleen trauma ultrasound image segmentation classification system, using the spleen trauma grade classification model to perform prediction, and obtaining the spleen trauma grade classification label corresponding to the ultrasound image;

[0058] S5. The spleen trauma ultrasound image segmentation and classification system outputs the corresponding visualized segmentation and classification results.

[0059] Embodiment 1:

[0060] like Figure 1 As shown, the method for artificial intelligence-assisted ultrasound diagnosis of spleen trauma of the present invention comprises the following steps:

[0061] S1. Connect the medical imaging device to the portable device to obtain an ultrasound image of the spleen of the injured person.

[0062] S2, extracting image frames from the video according to the time interval or frame interval, sampling and frame extraction processing is performed on the ultrasound image video stream, and optionally using video processing technology such as OpenCV to uniformly extract multiple frames of images of the corresponding video, and performing data preprocessing at the same time, removing irrelevant information, and generating a video image frame sequence, such as Figure 2 shown.

[0063] S3. Import each ultrasound image into the spleen trauma ultrasound image segmentation and classification system according to the video image frame sequence generated in step S2, use the spleen trauma segmentation model to predict the video image frame sequence, and obtain the corresponding segmentation prediction result sequence. The segmentation prediction result sequence corresponding to each video image includes position information such as spleen contour and / or trauma contour.

[0064] The ultrasound diagnosis of spleen trauma segmentation model adopts a residual network structure combined with a U-Net architecture. The specific network structure design is as follows:

[0065] S3.1. The input video image frame sequence is enlarged or reduced by an image scaling method, preferably a bilinear interpolation method, and then passed through a three-dimensional convolutional layer to make its size and dimension the same as the size and dimension of the feature map of the corresponding decoding stage.

[0066] S3.2, passes through the three-dimensional convolution layer, BatchNorm normalization layer, ReLU activation function and maximum pooling layer in turn, and then performs supervision and loss calculation in the encoding stage (encoder). There are four stages in total for encoding, including 3, 4, 6, and 3 three-dimensional ResNet residual network structures respectively. Each ResNet network uses residual learning to solve the degradation problem. The input of each stage is the down-sampling of the output result of the previous stage, and the output result of each stage is saved at the same time.

[0067] S3.3, perform supervision and loss calculation in the decoding stage (decoder), and perform four stages of decoding, where the first stage uses the output of the fourth encoding stage as input, and the second, third, and fourth stages of decoding use the upsampling of the output result of the previous decoding stage, as well as the output of the third, second, and first encoding stages as input. Each decoding stage includes a convolution layer, a BatchNorm normalization layer, and a ReLU activation function. After the fourth stage outputs the feature map of each channel, the final result is convolved to obtain a sequence of segmentation prediction results. Figure 3 shown.

[0068] S4. Import the segmentation prediction result sequence generated in step S3 into the spleen trauma ultrasound image segmentation classification system, use the spleen trauma grade classification model to perform prediction, and obtain the spleen trauma grade classification label corresponding to the ultrasound image.

[0069] The network structure of the spleen trauma grade classification model includes: a first three-dimensional ResNet residual network structure, a second three-dimensional ResNet residual network structure and a Softmax classifier. The segmentation prediction result sequence is input into the first three-dimensional ResNet residual network structure, the down-sampling encoding of the first three-dimensional ResNet residual network structure is input into the second three-dimensional ResNet residual network structure, and the down-sampling encoding of the second three-dimensional ResNet residual network structure is input into the Softmax classifier. The extracted features are classified by the Softmax classifier to obtain the spleen trauma grade classification label corresponding to the input segmentation prediction result sequence. Figure 4 shown.

[0070] S5. The spleen trauma ultrasound image segmentation and classification system outputs the corresponding visualized segmentation and classification results.

[0071] Furthermore, before step S1, the method further comprises:

[0072] S0. Training the spleen trauma segmentation model and spleen trauma grade classification model in the spleen trauma ultrasound image segmentation classification system, specifically including:

[0073] S0.1. The training ultrasound images include ultrasound videos of spleen trauma and normal spleen. Each training ultrasound image is processed by frame extraction as in step S2 to generate a set of training image frame sequences, corresponding to a set of manually annotated segmented image frame sequences and corresponding spleen trauma grade classification labels, which respectively constitute an original image data set, a segmentation label data set, and a classification label data set; wherein the manually annotated segmented image frame sequence includes position information such as spleen contour and / or trauma contour,

[0074] S0.2, training the spleen trauma segmentation model based on the original image dataset and segmentation label dataset obtained in step S0.1;

[0075] S0.3. Train the spleen trauma grade classification model based on the segmentation label dataset and classification label dataset obtained in step S0.1.

[0076] Furthermore, the spleen trauma segmentation model and spleen trauma grade classification model obtained through training mentioned in step S0 are respectively converted into model scripts through the TorchScript method, converting the model from a pure Python program into a TorchScript program that can run independently of Python and can be loaded into a process without Python dependency.

[0077] The spleen trauma ultrasound image segmentation and classification system loads the above script as a prediction tool and outputs intuitive prediction results in a visual interface. The system can run in a portable device environment without Python dependency, achieving lightweight and easy portability. Figure 5 shown.

[0078] Furthermore, in step S2, the adjustment of the frame extraction scheme

[0079] Adjustments to the frame extraction scheme based on time interval or frame interval, such as the frame extraction scheme based on the differential intensity between frames, can be used as an alternative solution to complete the frame extraction process to form an image sequence in the artificial intelligence-assisted ultrasound diagnosis of spleen trauma system.

[0080] Furthermore, in step S3, the spleen injury segmentation model structure is adjusted

[0081] Adjusting the network structure of the ultrasound diagnosis of spleen trauma segmentation model, for example, adjusting the three-dimensional ResNet network in the network to a two-dimensional combined with three-dimensional form, or using other three-dimensional CNN network structures or LSTM network structures, can serve as an alternative to the segmentation prediction process in the spleen trauma ultrasound image segmentation classification system.

[0082] Furthermore, in step S4, the adjustment of the spleen injury grade classification model structure

[0083] Adjusting the network structure of the ultrasound diagnosis of spleen trauma grade classification model, such as other CNN multi-classification network structures, can be used as an alternative to the classification prediction process in the spleen trauma ultrasound image segmentation classification system.

[0084] Furthermore, the residual network (Residual Network, referred to as ResNet) includes one or more residual modules, such as Figure 6 As shown in the figure, by adding a short-circuit connection between the input and output of the residual module, that is, the output of the residual module is the result of the positional addition of the learning feature F(x) after the weight matrix and the input x, the gradient vanishing problem that may be caused by the deep convolutional network is avoided. The cross-layer data channel uses residual learning to solve the degradation problem, so that an effective deep neural network can be trained.

[0085] Furthermore, if Figure 7As shown in the figure, U-net uses a classic encoder-decoder structure. The network consists of two parts: the encoding stage uses a contracting path to gradually reduce the spatial dimension of the feature map to obtain high-level context information, and the decoding stage uses a symmetrical expanding path corresponding to upsampling to restore the resolution. In addition, the skip-layer connection that passes the encoder output to the corresponding level decoder is used to reduce the spatial information loss caused by the downsampling process. The feature map is spliced ​​in the channel dimension so that the up-sampled restored feature map contains more low-level semantic information, which helps to restore the details of the lesion and the spatial dimension for precise positioning. Because its shape is similar to a "U" shape, it is called U-Net.

[0086] The ultrasound diagnosis of spleen trauma segmentation model in the present invention uses ResNet as the BackBone of U-Net, introduces a residual network structure into the original U-Net, and directly connects the encoder and decoder to improve the accuracy, which reduces the processing time to a certain extent. In this way, the information lost in different layers of the encoding part is retained, and at the same time, no additional parameters and operations are added when relearning the lost information, which greatly improves the accuracy of spleen trauma segmentation as a whole.

[0087] The present invention trains model parameters through ultrasound videos of spleen trauma or normal spleen, has the ability to analyze ultrasound video streams, analyzes video streams, and generates original image sequences that help to complete the features of a single image, so that the model fully understands the contextual information and improves segmentation accuracy. At the same time, frame extraction processing can maximize the use of information in the video, greatly reducing the storage space of the data set. On this basis, it can also support the predictive segmentation of a single spleen trauma image.

[0088] The ultrasonic diagnosis of spleen trauma segmentation model in the present invention adopts a residual network structure combined with a U-Net architecture, which simultaneously obtains the advantages of the two network structures and improves the accuracy of lesion segmentation.

[0089] In the present invention, step S4 combines the segmentation results of multiple spleen trauma ultrasound images to perform comprehensive judgment and classification based on the ultrasound diagnosis spleen trauma grade classification model, making full use of the correlation between image information, obtaining sufficient context information, and improving the accuracy of trauma grade classification.

[0090] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for artificial intelligence-assisted ultrasound diagnosis of spleen trauma, characterized in that: The method comprises the following steps: S1. Connect the medical imaging device to the portable device to obtain an ultrasound image of the spleen of the injured person; S2, performing sampling and frame extraction processing on the ultrasound image video stream to generate a video image frame sequence; S3, importing the video image frame sequence generated by each ultrasound image into the spleen trauma ultrasound image segmentation and classification system, using the spleen trauma segmentation model to predict the video image frame sequence, and obtaining a corresponding segmentation prediction result sequence, wherein the segmentation prediction result sequence includes spleen contour and / or trauma contour position information; S4, importing the segmentation prediction result sequence generated in step S3 into the spleen trauma ultrasound image segmentation classification system, using the spleen trauma grade classification model to perform prediction, and obtaining the spleen trauma grade classification label corresponding to the ultrasound image; S5. The spleen trauma ultrasound image segmentation and classification system outputs the corresponding visualized segmentation and classification results; in, Before step S1, the step further includes: S0, training a spleen trauma segmentation model and a spleen trauma grade classification model in a spleen trauma ultrasound image segmentation classification system; The step S0 specifically includes: S0.

1. The training ultrasound images include ultrasound images of spleen trauma and normal spleen. Each training ultrasound image is processed by frame extraction to generate a set of training image frame sequences, corresponding to a set of manually annotated segmented image frame sequences and corresponding spleen trauma grade classification labels, which respectively constitute the original image data set, the segmentation label data set and the classification label data set; wherein the manually annotated segmented image frame sequence includes the spleen contour and / or trauma contour position information, S0.2, training the spleen trauma segmentation model based on the original image dataset and segmentation label dataset obtained in step S0.1; S0.3, training a spleen injury grade classification model based on the segmentation label dataset and classification label dataset obtained in step S0.1; The spleen trauma segmentation model and spleen trauma grade classification model obtained by training in step S0 are respectively converted into model scripts by the TorchScript method, and the spleen trauma ultrasound image segmentation classification system loads the above scripts as prediction tools, and outputs intuitive prediction results in a visual interface; In step S3, the spleen trauma segmentation model adopts a residual network structure combined with a U-Net architecture, and the specific network structure is designed as follows: S3.1, the input image frame sequence is passed through a three-dimensional convolutional layer through an image scaling method to make its size and dimension the same as the feature map size and dimension of the corresponding decoding stage; S3.2, pass through the three-dimensional convolution layer, BatchNorm normalization layer, ReLU activation function and maximum pooling layer in sequence, and then perform the supervision and loss calculation of the encoding stage (encoder). The encoding has four stages, including 3, 4, 6, and 3 three-dimensional ResNet residual network structures respectively. The input of each stage is the down-sampling of the output result of the previous stage, and the output result of each stage is saved at the same time; S3.3, perform supervision and loss calculation of the decoding stage (decoder), and perform four stages of decoding, wherein the first stage uses the output of the fourth encoding stage as input, and the second, third, and fourth stages of decoding use the upsampling of the output result of the previous decoding stage, and the output of the third, second, and first encoding stages as input, respectively. Each decoding stage includes a convolutional layer, a BatchNorm normalization layer, and a ReLU activation function. After the fourth stage outputs the feature maps of each channel, the final result is convolved to obtain a sequence of segmentation prediction results; In the step S4, the network structure of the spleen injury grade classification model includes: a first three-dimensional ResNet residual network structure, a second three-dimensional ResNet residual network structure and a Softmax classifier, the segmentation prediction result sequence is input into the first three-dimensional ResNet residual network structure, the down-sampling code of the first three-dimensional ResNet residual network structure is input into the second three-dimensional ResNet residual network structure, and the down-sampling code of the second three-dimensional ResNet residual network structure is input into the Softmax classifier, and the extracted features are classified by the Softmax classifier to obtain the spleen injury grade classification label corresponding to the input segmentation prediction result sequence; The ResNet residual network structure includes one or more residual modules, and a short-circuit connection is added between the input and output of the residual module, that is, the output of the residual module is the result of the bitwise addition of the learning feature F(x) after the weight matrix and the input x.

2. The method for artificial intelligence-assisted ultrasound diagnosis of spleen trauma according to claim 1, characterized in that: In step S2, OpenCV video processing technology is used to uniformly extract multiple frames of images of the corresponding video, and data preprocessing is performed to remove irrelevant information.

3. The method for artificial intelligence-assisted ultrasound diagnosis of spleen trauma according to claim 1, characterized in that: When extracting frames in step S2, a frame extraction scheme based on time interval, frame interval or inter-frame differential intensity is adopted.

Images