Method and system for delineating organs at risk based on deep learning

Abstract

The present application provides a method and system for outlining organs at risk based on deep learning, wherein the outlining method comprises the following steps: step S10, based on a computer vision model, detecting a loaded medical image, and determining that the medical image contains organs at risk pre-stored in a labeling database, wherein the labeling database contains a plurality of different organs at risk; step S20, calling a pre-built organ outlining model, and automatically outlining the contours of the organs at risk in the medical image. The technical solution of the present application can achieve high-precision automatic outlining of organs at risk, and can effectively assist doctors in outlining the contours of organs at risk, thereby greatly improving the doctor's outlining efficiency and accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of medical image technology, and specifically relates to a method and system for delineating organs at risk based on deep learning. Background Art

[0002] Malignant tumors are one of the main causes of death in the world. Radiotherapy, abbreviated as radiotherapy, is one of the three important means of tumor treatment.

[0003] The ideal state of radiotherapy is to irradiate only the tumor area without irradiating the normal tissues around the tumor. This is difficult to achieve in reality. We can only irradiate the normal tissues near the tumor-at-risk organs as little as possible under the premise that the irradiation range of the tumor-at-risk organs is sufficient and the irradiation dose of these surrounding organs is ensured to be within the safe value range. Therefore, before implementing radiotherapy, it is necessary to carefully outline the contours of the tumor-at-risk organs and their adjacent normal organs, and the irradiated normal organs around the irradiated organs are the organs at risk (OARs).

[0004] Before radiotherapy, patients generally undergo a variety of imaging examinations, such as CT, MRI, PET-CT, etc. These imaging data often have dozens or even hundreds of fault planes. Doctors need to accurately outline the tumor organs at risk and the contours of organs at risk on each plane of these imaging data. At present, the delineation of OARs is mainly done manually by radiotherapy doctors, and this manual delineation takes half an hour to several hours. The accuracy of manual delineation is highly dependent on the clinical experience of radiologists. The process is cumbersome and inefficient, resulting in a limited number of patients treated, low delineation accuracy, and difficulty in achieving consistency in the delineation level, ability and standards of different doctors. Therefore, medical workers are eager to have a high-precision automatic segmentation technology for organs at risk to free them from this cumbersome, repetitive and monotonous workflow, saving time for receiving and treating more patients.

[0005] Based on the urgent needs of radiation oncologists, the automatic segmentation and delineation technology of organs at risk has always been the research direction of many scholars. At present, the technology of medical image segmentation can be roughly divided into three categories: traditional image processing methods, traditional machine learning methods, and deep learning methods. Traditional image processing methods mostly use the "template matching" method, which has high requirements for data sets. This method needs to save templates with huge data in the database. It is a low-adaptive solution with poor delineation accuracy. Doctors still need to spend a lot of work to modify and improve the delineation results. Traditional machine learning is a method based on statistical features and function fitting. When segmenting medical images, inaccurate classification will cause serious false positive phenomena in individual image instances. Artificial intelligence deep learning is a general term for a class of pattern analysis methods. Due to its rapid development in recent years, deep learning technology has been increasingly used in various fields. The convolutional neural network (CNN) model is one of the most important methods in deep learning technology. CNN is composed of multiple layers of neurons, which gives it a strong feature learning ability. Its basic principle is to use neural networks to automatically learn image features that are useful for segmentation, and then transform the feature representation of the sample in the original space into another new feature space through layer-by-layer feature transformation. After continuous feature extraction and transformation, the segmentation task is performed. Compared with traditional machine learning methods, deep learning models have more parameters and stronger expression capabilities. The existing CNN-based fully automatic segmentation method has greatly improved the outline accuracy, but it still cannot achieve the high enough accuracy required by medical clinical practice, and doctors still need to manually and further repair segmentation errors.

[0006] Therefore, it is necessary to provide an improved technical solution to address the above-mentioned deficiencies in the prior art. Summary of the invention

[0007] The object of the present invention is to provide a method and system for delineating organs at risk based on deep learning, so as to achieve high-precision automatic delineation of organs at risk.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A method for delineating organs at risk based on deep learning, the method comprising the following steps:

[0010] Step S10: Based on the computer vision model, the loaded medical image is detected to determine whether the medical image contains an organ at risk pre-stored in a labeling database, wherein the labeling database contains a plurality of different organs at risk;

[0011] Step S20: calling a pre-built organ delineation model to automatically delineate the contours of the organs at risk in the medical image.

[0012] As an optional embodiment of the deep learning-based organ delineation method described above, step S10 specifically comprises: based on a pre-trained Vector Boosting model, detecting the loaded medical image to determine whether the medical image contains the organ at risk pre-stored in the annotation database.

[0013] The deep learning-based organ delineation method as described above is an optional embodiment. In step S10, feature extraction is performed on the medical image based on the modified center transform and the local gradient pattern, respectively, and a Vector Boosting model is obtained based on the Vector Boosting algorithm according to the extracted MCT features and LGP features of the medical image.

[0014] The deep learning-based organ delineation method as described above is an optional embodiment. In step S10, feature extraction is performed on the medical image based on the modified center transform and the local gradient pattern, respectively, and a Vector Boosting model is obtained based on the Vector Boosting algorithm according to the extracted MCT features and LGP features of the medical image.

[0015] As an optional embodiment, the deep learning-based organ delineation method described above is based on modifying the center transformation according to the formula:

[0016]

[0017] Extracting features from the medical image to obtain MCT features of the medical image;

[0018] Wherein, N'(x)=N'(x)∪x is a plurality of pixels adjacent to pixel x in the medical image, I(x) is the gray value of pixel x, is the average gray value of all neighboring pixels of pixel x, The definition is equivalent to C;

[0019] Based on the local gradient mode, according to the formula:

[0020]

[0021] Extracting features of the medical image to obtain LGP features of the medical image;

[0022] Among them, (x c ,y c ) is the pixel center point, p refers to the current pixel center point (x c ,y c ) The number of neighboring pixels around, r indicates how to sample the surrounding pixels, the center point i c With adjacent point in The gray value difference between n =|i n -i c |, the average grayscale difference is

[0023] As an optional embodiment of the deep learning-based organ delineation method described above, step S10 is replaced by: detecting the loaded medical image based on a nonlinear support vector machine model of an RBF kernel function to determine whether the medical image contains an organ at risk pre-stored in the annotation database;

[0024] The nonlinear support vector machine model of the RBF kernel function is:

[0025]

[0026] 0≤α i ≤D

[0027]

[0028] Among them, α i is the hidden variable for the i-th medical image, y i is the category label of the image, with a value of ±1, corresponding to positive samples and negative samples respectively, n is the number of training samples, x i is the MCT feature and LGP feature of the i-th medical image, feature, k(x i ,x j ) is used to calculate the similarity between the i-th medical image and the j-th medical image, and D is the penalty parameter.

[0029] As an optional embodiment of the deep learning-based organ delineation method as described above, in step S20, based on the residual neural network and the fully connected convolutional neural network, the organ delineation model is constructed according to the contour data of the organ at risk stored in the annotation database.

[0030] As an optional embodiment of the deep learning-based organ delineation method as described above, in step S20, the constructed organ delineation model is trained according to a cross entropy loss function and a pre-constructed identifiable loss function;

[0031] The cross entropy loss function is:

[0032]

[0033] Where M is the number of organs at risk in the medical image, m is the number of organs at risk in the medical image, and y o,m represents the pixel of the organ at risk m in the medical image, p o，mrepresents the probability that the pixel o predicted by the organ delineation model belongs to the endangered organ m;

[0034] The identifiable loss function includes: variance loss function L var And maximize the marginal loss function L dist , the variance loss function is:

[0035]

[0036] Among them, δ v represents the predefined mean, N c represents the total number of pixels in the organ at risk m, N m represents the total pixel value of all pixels corresponding to the organ at risk m;

[0037] The maximized edge loss function is:

[0038]

[0039] Where M is the number of organs at risk in the medical image, x i is a vector representing each pixel, μ m is the average vector of all pixels of all training samples of the organ at risk m in the medical image, m A is the pixel index A,m in the organ at risk m B is the pixel index B in the organ at risk m, δ d is the difference between the pixel corresponding to the predefined pixel index A and the pixel corresponding to the pixel index B. is the vector value of the pixel corresponding to the pixel index A of the current training sample of the organ at risk m in the medical image, It is the vector value of the pixel corresponding to the pixel index B of the current training sample of the endangered organ m in the medical image.

[0040] As an optional embodiment of the deep learning-based organ delineation method as described above, in step S20, the constructed organ delineation model is trained according to the preset weights of the cross entropy loss function and the pre-constructed identifiable loss function.

[0041] As a preferred embodiment, the preset weight of the cross entropy loss function is 0.8, and the preset weight of the identifiable loss function is 0.2.

[0042] As an optional embodiment of the deep learning-based organ delineation method as described above, in step S20, a pre-built organ delineation model is called to detect the contours of the organs at risk in the medical image, and different contour detection results of the organs at risk are displayed in different colors.

[0043] The present application also proposes a deep learning-based risk organ delineation system, the delineation system comprising:

[0044] A detection unit, the detection unit being configured to detect the loaded medical image based on a computer vision model, and determine that the medical image contains an organ at risk pre-stored in a labeling database, wherein the labeling database contains a plurality of different organs at risk;

[0045] A delineation unit is configured to call a pre-built organ delineation model to automatically delineate the contour of the organ at risk in the medical image.

[0046] Compared with the closest prior art, the technical solution of the embodiment of the present application has the following beneficial effects:

[0047] This application detects the loaded medical images based on the computer vision model. When it is determined that the medical image contains the endangered organs pre-stored in the annotation database, the pre-built organ delineation model is called to automatically outline the contours of the endangered organs in the medical image. Here, the outlines of the endangered organs of the human body are outlined in combination with computer vision technology and the latest artificial intelligence deep learning technology. High-precision automatic delineation of endangered organs can be achieved, and it can effectively assist doctors in outlining the contours of endangered organs, thereby greatly improving the doctor's delineation efficiency and accuracy. In addition, the present application has the advantages of more accurate algorithms, stronger robustness, more extreme situations that can be handled, and effective elimination of noise caused by various external imaging interferences. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. Among them:

[0049] Figure 1 A flowchart of an organ-at-risk delineation method based on deep learning according to some embodiments of the present application;

[0050] Figure 2 A logical block diagram of an organ-at-risk delineation method based on deep learning according to some embodiments of the present application;

[0051] Figure 3 An LGP feature extraction diagram provided according to some embodiments of the present application;

[0052] Figure 4 An MCT feature extraction diagram provided according to some embodiments of the present application;

[0053] Figure 5A schematic diagram of a cascade classifier according to some embodiments of the present application;

[0054] Figure 6 A diagram of a Vector Boost training process provided according to some embodiments of the present application;

[0055] Figure 7 A schematic diagram of a convolutional neural network architecture provided according to some embodiments of the present application;

[0056] Figure 8 A coordinate diagram of a RELU function provided according to some embodiments of the present application;

[0057] Fig. 9 A schematic diagram of a delineation result provided according to some embodiments of the present application;

[0058] Fig.10 A schematic diagram of an organ-at-risk delineation system based on deep learning according to some embodiments of the present application. DETAILED DESCRIPTION

[0059] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments. Each example is provided by way of explanation of the present application and does not limit the present application. In fact, it will be clear to those skilled in the art that modifications and variations may be made in the present application without departing from the scope or spirit of the present application. For example, a feature shown or described as a part of an embodiment may be used in another embodiment to produce yet another embodiment. Therefore, it is desired that the present application includes such modifications and variations within the scope of the appended claims and their equivalents.

[0060] This application detects the loaded medical images based on the computer vision model. When it is determined that the medical images contain the endangered organs pre-stored in the annotation database, the pre-built organ delineation model is called to automatically delineate the contours of the endangered organs in the medical images. Here, the contours of the endangered organs of the human body are delineated by combining computer vision technology with the latest artificial intelligence deep learning technology, which can achieve high-precision automatic delineation of endangered organs and can effectively assist doctors in delineating the contours of endangered organs, thereby greatly improving the doctor's delineation efficiency and accuracy.

[0061] Figure 1 A flowchart of an organ-at-risk delineation method based on deep learning according to some embodiments of the present application; Figure 2 A logical block diagram of an organ-at-risk delineation method based on deep learning according to some embodiments of the present application.

[0062] like Figure 1 As shown, the method for delineating organs at risk in radiotherapy based on deep learning in this application includes the following steps:

[0063] Step S10: Based on the computer vision model, the loaded medical image is detected to determine whether the medical image contains an organ at risk pre-stored in a labeling database, wherein the labeling database contains a plurality of different organs at risk;

[0064] Step S20: calling a pre-built organ delineation model to automatically delineate the contours of the organs at risk in the medical image.

[0065] The medical images of the present application include CT images, MRI images, PET-CT images and other images. The images loaded during the specific operation may be one of the above medical images. Preferably, the medical images are CT images. When detecting the loaded medical images, the entire medical image is compared with the image data containing the organs at risk pre-stored in the annotation database. If it is detected that the medical image contains the pre-stored organs at risk, the pre-built organ delineation model is called to automatically delineate the contours of the organs at risk in the medical image. It should be noted that the annotation database includes multiple different organs at risk, and the medical image may contain multiple different organs at risk. When comparing, it is determined in turn that the medical image contains multiple different organs at risk pre-stored. The annotation database of the present application includes the following organs: body surface, trachea, lungs, esophagus, aorta, spinal cord, etc. As the annotation database increases, more organs can be automatically delineated.

[0066] The hardware platform on which the organ delineation model algorithm pre-built in this application relies is Nvidia's GPU.

[0067] In addition, when detecting the loaded medical image, if it is not detected that the medical image contains the endangered organ pre-stored in the annotation database, the subsequent processing is abandoned and the doctor is prompted that the medical image does not contain the endangered organ until the doctor loads the medical image containing the endangered organ.

[0068] like Figure 2 As shown, in the specific embodiment of the present application, the operation steps of the above-mentioned delineation method are specifically as follows:

[0069] (1) Obtain a CT image scanned by a CT scanning machine.

[0070] (2) Calling the medical image detection module to detect whether the medical image currently acquired by the hardware is marked with the endangered organs pre-stored in the database. If not, the doctor is prompted that the input image is incorrect until the software can detect the medical image containing the endangered organs of the human body.

[0071] (3) If it is detected that the medical image contains an organ at risk that is pre-stored in the annotation database, the software interface displays a prompt to start the detection algorithm.

[0072] (4) Read the information of the medical image and call the pre-built organ delineation model to automatically delineate the contours of the organs at risk in the medical image.

[0073] (5) If you receive a message from the user to close the software, close the software, release the memory, and exit.

[0074] It should be noted that, during the specific operation, medical images are continuously loaded, and medical images containing organs at risk pre-stored in the annotation database are continuously read, and at the same time, the contours of the organs at risk in the medical images are automatically outlined using a pre-built organ outline model.

[0075] From the above operating steps, it can be seen that this application mainly includes two functional modules: one is the corresponding computer vision module, which is used to analyze whether the image contains medical images of endangered organs pre-stored in the annotation database. The second is the endangered organ delineation algorithm, which is used to outline the contours of endangered organs to assist doctors in making decisions, such as whether a patient has lung cancer. Specifically, the computer vision algorithm is used to analyze the obtained CT images and outline different endangered organs. The precise organ segmentation information of this application can assist doctors in efficiently completing the delineation of endangered organs and the calculation of cumulative irradiation doses.

[0076] In a specific embodiment of the present application, step S10 specifically includes: based on a pre-trained vector boosting algorithm (VectorBoosting) model, the loaded medical image is detected to determine whether the medical image contains the endangered organs pre-stored in the annotation database.

[0077] In this embodiment, the Vector Boosting model is used as the classifier framework to train the entire algorithm, which specifically includes the following stages:

[0078] (1) Feature extraction stage: Modified Census Transform (MCT) and Local Gradient Patterns (LGP) are used to extract the features of all medical images and non-medical images (including infrared database and general database). The specific MCT and LGP feature extraction methods are shown in the attached Figure 3 and 4 As shown in the figure, this figure shows how to sample the surrounding pixels when the current central pixel is given.

[0079] (2) Positive and negative sample construction stage: A database of loaded CT images is collected and manually classified into images that contain contour data of endangered organs pre-stored in the annotation database (i.e., medical images) and images that do not contain contour data of endangered organs pre-stored in the annotation database (i.e., non-medical images). Medical images are regarded as positive samples, and non-medical images are regarded as negative samples.

[0080] (3) Original training stage: The Vector Boosting algorithm is used on this database to construct a cascade classifier. The features used by the classifier are a combination of MCT features and LGP features, that is, the MCT feature vector and the LGP feature vector are linked together in a certain dimension. For example, if the MCT feature vector is a and the LGP feature vector is b, then the linked vector of the two is [a, b]. The principle of the cascade classifier is shown in Figure 5 shown.

[0081] (4) Detection phase: Using the Vector Boosting model learned in the training phase, the organ regions are detected on all loaded medical images using a vector tree model structure.

[0082] It should be noted that in the feature extraction stage or the positive and negative sample construction stage, the image is scaled to 360×240 pixels.

[0083] In the specific embodiment of the present application, based on the modified center transformation, according to the formula:

[0084]

[0085] Extract features of medical images to obtain MCT features of medical images;

[0086] Wherein, N'(x)=N'(x)∪x is a plurality of pixels adjacent to pixel x in the medical image, I(x) is the gray value of pixel x, is the average gray value of all neighboring pixels of pixel x, The definition is equivalent to C;

[0087] In the specific embodiment of the present application, based on the local gradient mode, according to the formula:

[0088]

[0089] Extract features of medical images to obtain LGP features of medical images;

[0090] Among them, (x c ,y c ) is the pixel center point, p refers to the current pixel center point (x c ,y c) The number of neighboring pixels around, r indicates how to sample the surrounding pixels, the center point i c With adjacent point i n The gray value difference between n =|i n -i c |, the average grayscale difference is

[0091] Different from other algorithms, the MCT features and LGP features in this application complement each other. Combining them together can improve the stability of the entire algorithm. Moreover, under the same error detection rate, the correct detection rate will be greatly improved, and the entire feature extraction calculation time will still be low.

[0092] In the positive and negative sample construction stage, the present application constructs more non-chest organ images for many non-chest organ images, such as brain CT, abdominal CT, limb CT, MRI images, X-ray images, B-ultrasound images, etc., so that the ratio of positive and negative samples in the entire image sample is unbalanced. By accepting a large number of non-chest CT as negative samples, the false alarm rate of the entire model for human body endangered organ detection can be reduced.

[0093] In the original training stage, this application directly uses Vector Boosting to train the model. Each time, a part of the feature values ​​are selected from the high-dimensional LGP and MCT feature values, and a certain weight is given to each weak classifier. Combined with the results of the current classifier, the weights of each image are re-assigned. The wrongly classified ones are given a larger weight, and the correctly classified ones are given a smaller weight. The specific training process is shown in Figure 6 .from Figure 6 As can be seen from the figure, the specific training process is as follows: first input the feature value of the training data, build a weak classifier under the current sample weight, give each weak classifier a certain weight, update the strong classifier, and combine the results of the current classifier to adjust the sample weight to re-weight each image, give larger weights to incorrectly classified images, and smaller weights to correctly classified images; finally, output the final strong classifier.

[0094] The formula for selecting a weak classifier is:

[0095]

[0096] Where exp is an exponential function, m represents the number of positive and negative training samples, and f(x i ) is a weak classifier, v i is the current classification tag, The weight of sample i at iteration t.

[0097] In the testing phase, the final output strong classifier is Since the present application adopts the VectorBoosting algorithm, it is relatively robust and can process human organ images of different forms, making the range of human organ detection relatively large. Furthermore, unlike the general Haar feature extraction method, since the present application uses a combination of the modified center change feature extraction algorithm (MCT) and the local gradient pattern (LGP) feature extraction, the algorithm is very insensitive to changes in the image illumination, and the detection effect is better.

[0098] In a specific embodiment of the present application, after obtaining the model parameters of the entire algorithm, given a new medical image, the MCT features and local gradient pattern LGP of the medical image are first extracted, the features of the entire medical image are calculated, and then the Vector Boosting model is used to evaluate whether the medical image is a CT image containing endangered organs pre-stored in the annotation database.

[0099] In some other optional embodiments of the present application, step S10 is replaced by: detecting the loaded medical image based on a nonlinear support vector machine model (Support Vector Machines, SVM) of an RBF kernel function to determine whether the medical image contains contour data of the organ at risk pre-stored in the annotation database;

[0100] The nonlinear support vector machine model of the RBF kernel function is:

[0101]

[0102] 0≤α i ≤D

[0103]

[0104] Among them, α i is the hidden variable for the i-th medical image, y i is the category label of the image, with a value of ±1, corresponding to positive samples and negative samples respectively, n is the number of training samples, x i are the MCT features and LGP features of the i-th medical image, k(x i ,x j ) is used to calculate the similarity between the ith medical image and the jth medical image. D is a penalty parameter used to penalize incorrectly classified training samples. It is an artificially set parameter whose value is a real number greater than 0. During the training process, if the result classified by the current classifier is inconsistent with the actual annotation result, a larger penalty parameter is given to the current classification result to correct the classifier error in the next training iteration.

[0105] It should be noted that the RBF kernel function is the radial basis function (RBF), which is a scalar function that is radially symmetric.

[0106] In a specific embodiment of the present application, in step S20, based on the residual neural network and the fully connected convolutional neural network, the organ delineation model is constructed according to the contour data of the organ at risk stored in the annotation database. In this embodiment, step S20 specifically includes the following stages:

[0107] (1) Training sample construction stage: Through the annotation software, doctors outline the contours of different human organs, and then the annotation software stores the annotation results in the form of masks. Specifically, the areas where different organs are located are filled with 1 to N annotations, and 0 represents the non-human organ parts in the image.

[0108] (2) Training phase: Deep learning is used to construct an OAR delineation classifier. The human organ segmentation network used is a wider and deeper residual neural network and a fully connected convolutional neural network. The residual neural network is used to extract features from the input image and transform the two-dimensional input image into a high-dimensional vector. The fully connected convolutional neural network is used to transform the high-dimensional vector into a pixel-level delineation result.

[0109] This application uses a variant of a residual neural network, that is, a wider and deeper residual neural network, as the backbone network for radiotherapy endangered organ delineation, which is used to extract features from input image images and convert two-dimensional input images into high-dimensional vectors. On this basis, a fully connected convolutional neural network is added to convert high-dimensional vectors into pixel-level delineation results. Specifically, a fully connected convolutional neural network is a reverse operation of a convolutional neural network. It converts any input feature vector into a two-dimensional space probability map by upsampling the input with a 1×1 kernel. The value of each probability map is the classification of the corresponding pixel. In order to convert the probability corresponding to each pixel in the two-dimensional space probability map into the label of the actual annotated image, the linear difference method is used to scale the two-dimensional space probability map to the actual image size.

[0110] In the specific embodiment of the present application, the parameters of the wider and deeper residual neural network used in the training stage are as follows: there are a total of M layers of neural networks, the convolution kernel size of each layer is k×k, and each layer has a total of C channels. This parameter is manually adjusted based on experimental results. M is usually dozens, k is generally 3, and C is 3 or 16. Compared with the original residual neural network, the vector dimension of each layer of this network model is larger, and each layer of the neural network is wider, that is, k can be set larger, C can also be set larger, and the overall model has deeper layers, so the overall model has more parameters and is more suitable for feature extraction.

[0111] The human organ segmentation network used in the training phase of this application is a wider and deeper residual neural network and a fully convolutional neural network. Figure 7 .from Figure 7 As can be seen, the first layer input is the feature value extracted from the image, the second layer is the convolution calculation layer, the third layer is the hidden variable layer, and the fourth layer is the full-volume neural network layer and the classification layer. The filter size of the convolution calculation layer is 10×10 pixels, and the interval of each filter is 2 pixel values. The hidden variable layer and the classification layer are all connected, and the activation function is the RELU function. The specific calculation formula is:

[0112] f(x)=max(0,x).

[0113] The shape of the RELU function is shown in Figure 8 As shown in the figure, it can be seen that the RELU function is a linear function. Compared with the general Sigmoid or Tanh activation function, the RELU function does not have the problem of derivative disappearance or derivative explosion during training, which makes the entire training process more stable. In addition, the calculation of the RELU function is simpler and does not require floating-point operations, which will greatly reduce the processing time on the embedded platform.

[0114] It can be understood that the deep learning algorithm for the delineation of organs at risk in the present application, after determining that the loaded CT image is a CT image of organs at risk pre-stored in the annotation database, normalizes the image into an image of 360×240 pixels through linear interpolation and affine transformation, and then inputs the entire image into a wider and deeper residual neural network and a fully connected convolutional neural network for automatic delineation of organs at risk. Moreover, the final output result of the delineation of organs at risk is to classify different pixels into different organs at risk of the human body. The delineation algorithm for organs at risk of the present application can delineate 7 types of organs, including organs on the body surface, lungs, etc. Since the delineation method for organs at risk of the present application is universal, as long as there is more high-quality annotation data for other organs, the delineation model can also support the automatic delineation of more organs as needed.

[0115] In actual operation, since the organ contours drawn by the deep learning algorithm will have certain noise, this application uses the corrosion algorithm to further post-process the segmentation results to ensure that the obtained organ contours are smoother.

[0116] In the specific embodiment of the present application, considering that the background of the CT image is relatively simple compared to other images and there is less overlap between different organs, in the training phase of step S20, the constructed organ delineation model is trained according to the cross entropy loss function and the pre-constructed identifiable loss function;

[0117] The cross entropy loss function is:

[0118]

[0119] Where M is the number of organs at risk in the medical image, m is the number of organs at risk in the medical image, and y o,m represents the pixel of the organ at risk m in the medical image, p o，m represents the probability that the pixel o predicted by the organ delineation model belongs to the endangered organ m;

[0120] Identifiable loss functions include: Variance loss function L var And maximize the marginal loss function L dist , the variance loss function is:

[0121]

[0122] Among them, δ v represents the predefined mean, N c represents the total number of pixels in the organ at risk m, N m represents the total pixel value of all pixels corresponding to the organ at risk m;

[0123] The maximum marginal loss function is:

[0124]

[0125] Where M is the number of organs at risk in the medical image, x i is a vector representing each pixel, μ m is the average vector of all pixels of all training samples of the organ at risk m in the medical image, m A is the pixel index A,m in the organ at risk m B is the pixel index B in the organ at risk m, δ d is the difference between the pixel corresponding to the predefined pixel index A and the pixel corresponding to the pixel index B. is the vector value of the pixel corresponding to the pixel index A of the current training sample of the organ at risk m in the medical image, It is the vector value of the pixel corresponding to the pixel index B of the current training sample of the endangered organ m in the medical image.

[0126] On top of the cross loss function, the variance loss function L var And maximize the marginal loss function L dist These two functions will make the probability distributions of the same organ closer and the probability distributions of different organs farther apart.

[0127] When actually training the network, the present application trains the constructed organ delineation model according to the preset weights of the cross entropy loss function and the pre-constructed identifiable loss function. Compared with using one loss function, the combination of two loss functions is used here to measure different aspects of the model, thereby making the model effect better. In a preferred embodiment, the weight of the cross entropy loss function is 0.8, and the weight of the identifiable loss function is 0.2.

[0128] Since most computer vision algorithms require training, different training data may affect the final results. In order to achieve better results, the training data details used by the organ segmentation training module are as follows: training images of approximately 50 patients, each patient containing 200 labeled images.

[0129] After the training is completed, all modules can only keep the test program and the trained model. In addition, fixed-point implementation is used to avoid floating-point operations, which greatly speeds up the operation of the entire system.

[0130] (3) Detection phase: The deep learning-based OAR delineation model learned during the training phase is used to detect medical images on all loaded medical images.

[0131] In a specific embodiment of the present application, in the detection stage of step S20, a pre-constructed organ delineation model is called to detect the contours of the organs at risk in the medical image, and different contour detection results of the organs at risk are displayed in different colors.

[0132] like Fig. 9 As shown, Fig. 9 The first row is the result of the outlining method of this application, and the second row is the result of the outlining by the doctor. It can be seen from the figure that the picture drawn by the outlining method of this application is closer to the picture drawn by the doctor and has certain practicality.

[0133] On Nvidia's K40GPU hardware, the deep learning algorithm of this application can process at least 2 frames of images per second. In summary, this application uses a combination of computer vision technology and the latest artificial intelligence deep learning technology to outline the contours of the human body's endangered organs, which can achieve efficient and accurate delineation of endangered organs, and can effectively assist doctors in outlining the contours of endangered organs, thereby greatly improving the doctor's delineation efficiency and accuracy. Overall, this application has the advantages of more accurate algorithms, stronger robustness, more extreme situations that can be handled, and the ability to effectively eliminate noise caused by various external imaging interferences. Fig.10 A schematic diagram of an organ-at-risk delineation system based on deep learning according to some embodiments of the present application is provided. Fig.10As shown, the deep learning-based organ-at-risk delineation system includes a detection unit 101 and a delineation unit 102. The detection unit 101 is configured to detect the loaded medical image based on a computer vision model to determine whether the medical image contains an organ-at-risk pre-stored in a labeling database, wherein the labeling database contains a plurality of different organs-at-risk; the delineation unit 102 is configured to call a pre-built organ delineation model to automatically delineate the contour of the organ-at-risk in the medical image.

[0134] In some optional embodiments, the detection unit 101 is further configured to detect the loaded medical image based on a pre-trained VectorBoosting model to determine whether the medical image contains an organ at risk pre-stored in the annotation database.

[0135] In some optional embodiments, the detection unit 101 is further configured to perform feature extraction on the medical image based on the modified center transform and the local gradient pattern, respectively, and obtain a Vector Boosting model based on the Vector Boosting algorithm according to the extracted MCT features and LGP features of the medical image.

[0136] In some optional embodiments, the detection unit 101 is further configured to modify the center transformation according to the formula:

[0137]

[0138] Extracting features from the medical image to obtain MCT features of the medical image;

[0139] Where N'(x) = N' (x)∪x are multiple pixels adjacent to pixel x in the medical image, and I(x) is

[0140] The gray value of pixel x, is the average gray value of all neighboring pixels of pixel x, The definition is equivalent to C;

[0141] Based on the local gradient mode, according to the formula:

[0142]

[0143] Extracting features of the medical image to obtain LGP features of the medical image;

[0144] Among them, (x c ,y c ) is the pixel center point, center point i c With adjacent point i n The gray value difference betweenn =|i n -i c |, the average grayscale difference is

[0145] In some optional embodiments, the detection unit 101 is further configured to detect the loaded medical image based on a nonlinear support vector machine model of an RBF kernel function, and determine whether the medical image contains an organ at risk pre-stored in the annotation database;

[0146] The nonlinear support vector machine model of the RBF kernel function is:

[0147]

[0148] 0≤α i ≤D

[0149]

[0150] Among them, α i is the hidden variable for the i-th medical image, y i is the category label of the image, with a value of ±1, corresponding to positive samples and negative samples respectively, and n is the number of training samples x i are the MCT features and LGP features of the i-th medical image, k(x i ,x j ) is the similarity between the ith medical image and the jth medical image, and D is a penalty parameter used to penalize misclassified training samples. It is an artificially set parameter, and its value is a real number greater than 0.

[0151] In some optional embodiments, the delineation unit 102 is further configured to construct the organ delineation model based on the residual neural network and the fully connected convolutional neural network according to the contour data of the organ at risk stored in the annotation database.

[0152] In some optional embodiments, the delineation unit 102 is further configured to train the constructed organ delineation model according to a cross entropy loss function and a pre-constructed discriminable loss function;

[0153] The cross entropy loss function is:

[0154]

[0155] Where M is the number of organs at risk in the medical image, m is the number of organs at risk in the medical image, and y o,m represents the pixel of the organ at risk m in the medical image, p o，m represents the probability that the pixel o predicted by the organ delineation model belongs to the endangered organ m;

[0156] The identifiable loss function includes: variance loss function L var And maximize the marginal loss function L dist , the variance loss function is:

[0157]

[0158] Among them, δ v represents the predefined mean, N c represents the total number of pixels in the organ at risk m, N m represents the total pixel value of all pixels corresponding to the organ at risk m;

[0159] The maximized edge loss function is:

[0160]

[0161] Where M is the number of organs at risk in the medical image, x i is a vector representing each pixel, μ m is the average vector of all pixels of all training samples of the organ at risk m in the medical image, m A is the pixel index A,m in the organ at risk m B is the pixel index B in the organ at risk m, δ d is the difference between the pixel corresponding to the predefined pixel index A and the pixel corresponding to the pixel index B. is the vector value of the pixel corresponding to the pixel index A of the current training sample of the organ at risk m in the medical image, is the vector value of the pixel corresponding to the pixel index B of the current training sample of the organ at risk m in the medical image. In some optional embodiments, the delineation unit 102 is further configured to train the constructed organ delineation model according to the preset weights of the cross entropy loss function and the pre-constructed identifiable loss function; preferably, the preset weight of the cross entropy loss function is 0.8, and the preset weight of the identifiable loss function is 0.2.

[0162] In some optional embodiments, the delineation unit 102 is further configured to call a pre-built organ delineation model to detect the contours of the organs at risk in the medical image, and display different contour detection results of the organs at risk in different colors.

[0163] The deep learning-based organ-at-risk delineation system provided in the embodiments of the present application can implement the processes and steps of any of the above-mentioned deep learning-based organ-at-risk delineation method embodiments and achieve the same technical effects, which will not be described one by one here.

[0164] The above description is only a preferred embodiment of the present application and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for delineating organs at risk based on deep learning, characterized in that: The delineation method comprises the following steps: Step S10: Based on the computer vision model, the loaded medical image is detected to determine whether the medical image contains an organ at risk pre-stored in a labeling database, wherein the labeling database contains a plurality of different organs at risk; Step S20, calling a pre-built organ delineation model to automatically delineate the contour of the organ at risk in the medical image; Among them, in step S20, based on the residual neural network and the fully connected convolutional neural network, the organ delineation model is constructed according to the contour data of the endangered organ stored in the annotation database, the residual neural network is used to extract features from the medical image and convert the medical image into a high-dimensional vector, and the fully connected convolutional neural network is used to convert the high-dimensional vector into a pixel-level delineation result.

2. The deep learning-based organ delineation method according to claim 1, characterized in that: Step S10 specifically includes: based on the pre-trained Vector Boosting model, detecting the loaded medical image to determine whether the medical image contains the organs at risk pre-stored in the annotation database.

3. The deep learning-based organ delineation method according to claim 2, characterized in that: In step S10, feature extraction is performed on the medical image based on the modified center transform and the local gradient pattern, and a Vector Boosting model is obtained based on the Vector Boosting algorithm according to the extracted MCT features and LGP features of the medical image.

4. The deep learning-based organ delineation method according to claim 3, characterized in that: Based on the modified center transformation, according to the formula: Extracting features from the medical image to obtain MCT features of the medical image; Wherein, N'(x)=N'(x)∪x is a plurality of pixels adjacent to pixel x in the medical image, I(x) is the gray value of pixel x, is the average gray value of all neighboring pixels of pixel x, The definition is equivalent to C; Based on the local gradient mode, according to the formula: Extracting features of the medical image to obtain LGP features of the medical image; Among them, (x c ,y c ) is the pixel center point, p refers to the current pixel center point (x c ,y c ) The number of neighboring pixels around, r indicates how to sample the surrounding pixels, the center point i c With adjacent point i n The gray value difference between them is g n =|i n -i c |, the average grayscale difference is 5. The deep learning-based organ delineation method according to claim 1, characterized in that: Step S10 is replaced by: detecting the loaded medical image based on a nonlinear support vector machine model of an RBF kernel function to determine whether the medical image contains an organ at risk pre-stored in the annotation database; The nonlinear support vector machine model of the RBF kernel function is: 0≤α i ≤D Among them, α i is the hidden variable for the i-th medical image, y i is the category label of the image, with a value of ±1, corresponding to positive samples and negative samples respectively, n is the number of training samples, x i is the MCT feature and LGP feature of the i-th medical image, feature, k(x i ,x j ) is used to calculate the similarity between the i-th medical image and the j-th medical image, and D is the penalty parameter.

6. The deep learning-based organ delineation method according to claim 1, characterized in that: In step S20, the constructed organ delineation model is trained according to the cross entropy loss function and the pre-constructed discriminable loss function; The cross entropy loss function is: Where M is the number of organs at risk in the medical image, m is the number of organs at risk in the medical image, and y o,m represents the pixel of the organ at risk m in the medical image, p o，m represents the probability that the pixel o predicted by the organ delineation model belongs to the endangered organ m; The identifiable loss function includes: variance loss function L var And maximize the marginal loss function L dist , the variance loss function is: Among them, δ v represents the predefined mean, N c represents the total number of pixels in the organ at risk m, N m represents the total pixel value of all pixels corresponding to the organ at risk m; The maximized edge loss function is: Where M is the number of organs at risk in the medical image, x i is a vector representing each pixel, μ m is the average vector of all pixels of all training samples of the organ at risk m in the medical image, m A is the pixel index A,m in the organ at risk m B is the pixel index B in the organ at risk m, δ d is the difference between the pixel corresponding to the predefined pixel index A and the pixel corresponding to the pixel index B, μ mA is the vector value of the pixel corresponding to the pixel index A of the current training sample of the endangered organ m in the medical image, μ mB It is the vector value of the pixel corresponding to the pixel index B of the current training sample of the endangered organ m in the medical image.

7. The deep learning-based organ delineation method according to claim 6, characterized in that: In step S20, the constructed organ delineation model is trained according to the cross entropy loss function and the preset weights of the pre-constructed identifiable loss function; The preset weight of the cross entropy loss function is 0.8, and the preset weight of the identifiable loss function is 0.

2.

8. The deep learning-based organ delineation method according to claim 1, characterized in that: In step S20, a pre-constructed organ delineation model is called to detect the contours of the organs at risk in the medical image, and different contour detection results of the organs at risk are displayed in different colors.

9. A deep learning-based organ delineation system, characterized in that: The delineation system comprises: A detection unit, the detection unit being configured to detect the loaded medical image based on a computer vision model, and determine that the medical image contains an organ at risk pre-stored in a labeling database, wherein the labeling database contains a plurality of different organs at risk; a delineation unit, the delineation unit being configured to call a pre-built organ delineation model to automatically delineate the contour of the organ at risk in the medical image; Among them, the delineation unit is specifically configured to be based on a residual neural network and a fully connected convolutional neural network, and to construct the organ delineation model according to the contour data of the endangered organ stored in the annotation database. The residual neural network is used to extract features from the medical image and convert the medical image into a high-dimensional vector, and the fully connected convolutional neural network is used to convert the high-dimensional vector into a pixel-level delineation result.

Images