A multi-model combined CT image processing method and system

Abstract

The present invention provides a multi-model combined CT image processing method and system, which relates to the technical field of medical image processing. The method includes: inputting a lesion CT image into a lesion detection model to obtain the region where the lesion is located in the lesion CT image, and obtaining a lesion image block according to the region where the lesion is located; inputting the lesion image block into the encoding module of the trained cancer recognition model to obtain a feature map of the lesion image block; inputting the feature map into the cancer probability recognition module to obtain a probability feature vector; inputting the feature map into the disease progression stage recognition module to obtain a disease progression stage feature vector; inputting the feature map, the probability feature vector and the disease progression stage feature vector into the decoding module to obtain a cancer recognition result; and obtaining a lesion analysis report according to the cancer recognition results of each lesion image block. According to the present invention, the probability of cancer occurrence in the lesion and the development stage of the lesion can be determined, and the performance of the cancer recognition model and the judgment accuracy of the lesion can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and particularly relates to a multi-model combined CT image processing method and system. Background Art

[0002] In the related art, CN119313677A discloses a medical injury CT image processing method and system, which relates to the field of medical image processing. The method includes: performing data preprocessing on a CT image; determining the noise type of the noise in the CT image; determining the noise intensity of the noise in the CT image; determining a training data set according to the noise type and the noise intensity; training a noise processing model according to the training data set to obtain a trained noise processing model; processing the CT image according to the trained noise processing model to obtain a denoised CT image; determining the injury area at the injury position in the denoised CT image, and determining the second CT value of the pixels at the injury position; determining an injury severity coefficient according to the injury area and the second CT value; and generating an injury report according to the injury severity coefficient. According to this solution, the accuracy and effectiveness of the detailed analysis of the injury position in the CT image can be improved.

[0003] CN119205714A discloses a chest CT image processing method and system, which relates to the technical field of medical image processing. The main solution is as follows: obtaining image data of two groups of different energy X-rays and attenuation measurement values of each pixel point by using dual-energy CT scanning, respectively calculating the respective concentration values of soft tissues and blood vessels and generating a concentration map; generating an initial image based on the soft tissue and blood vessel concentration maps and calculating pixel values, obtaining target image pixel values through contrast stretching and segmenting to obtain a binary image, distinguishing the blood vessel area and the soft tissue area in the binary image by identifying and analyzing the connected regions in the binary image; extracting the center line of the blood vessel area and calculating the local diameter, and respectively comparing with a blood vessel dilation threshold and a blood vessel constriction threshold to determine abnormal blood vessel dilation or constriction; extracting features of the soft tissue area to form a first feature vector and comparing with a preset second feature vector to determine abnormality; automatically annotating or highlighting abnormal situations to improve the diagnostic accuracy and efficiency.

[0004] CN119151967B discloses a medical image analysis method and system based on plain CT data, which relates to the technical field of medical image analysis. The medical image analysis method based on plain CT data obtains the plain chest CT data of a patient, and the plain chest CT data includes chest cross-sectional image data at a plurality of scanning positions; a three-dimensional model of the patient's chest is established based on the plain chest CT data of the patient; the three-dimensional model of the patient's chest is segmented to obtain a three-dimensional model of the patient's lungs. In the present invention, a three-dimensional model of a healthy lung is automatically reconstructed by using a deep learning model and compared with the actual lung model of the patient, so as to provide a more accurate disease diagnosis. This automated comparison reduces human error and improves the analysis efficiency, enabling doctors to quickly and accurately identify the lesion area and disease degree, and thus formulate a more effective treatment plan.

[0005] Therefore, although the related technology can automatically process CT images through computer image processing methods, it is unable to identify the development stage of lesions in CT images, making it difficult to obtain accurate detection results and subsequent treatment plans.

[0006] The information disclosed in the background art part of this application is only intended to deepen the understanding of the general background art of this application, and should not be regarded as an admission or any form of implication that this information constitutes the prior art known to those skilled in the art. Summary of the Invention

[0007] The present invention provides a multi-model combined CT image processing method and system, which can solve the technical problem that the related technology is unable to identify the development stage of lesions in CT images.

[0008] According to the first aspect of the present invention, there is provided a multi-model combined CT image processing method, including:

[0009] Input a lesion CT image into a lesion detection model to obtain the region where the lesion is located in the lesion CT image, and obtain a lesion image block according to the region where the lesion is located;

[0010] Input the lesion image block into the encoding module of the trained cancer recognition model to obtain a feature map of the lesion image block;

[0011] Input the feature map into a cancer probability recognition module to obtain a probability feature vector of the lesion in the lesion image block having canceration;

[0012] Input the feature map into a disease development stage recognition module to obtain a disease development stage feature vector of the lesion in the lesion image block;

[0013] Input the feature map, the probability feature vector, and the disease progression stage feature vector into the decoding module of the cancer transformation recognition module to obtain the cancer transformation recognition result of the lesion in the lesion image patch;

[0014] Obtain the lesion analysis report of the lesion CT image according to the cancer transformation recognition results of the lesions in each lesion image patch.

[0015] According to a second aspect of the present invention, there is provided a multi-model combined CT image processing system, including:

[0016] A detection module, configured to input a lesion CT image into a lesion detection model to obtain the region where the lesion is located in the lesion CT image, and obtain a lesion image patch according to the region where the lesion is located;

[0017] An input module, configured to input the lesion image patch into the encoding module of the trained cancer transformation recognition model to obtain the feature map of the lesion image patch;

[0018] A probability module, configured to input the feature map into a cancer transformation probability recognition module to obtain a probability feature vector of the occurrence of cancer transformation of the lesion in the lesion image patch;

[0019] A stage module, configured to input the feature map into a disease progression stage recognition module to obtain a disease progression stage feature vector of the lesion in the lesion image patch;

[0020] A result module, configured to input the feature map, the probability feature vector, and the disease progression stage feature vector into the decoding module of the cancer transformation recognition module to obtain the cancer transformation recognition result of the lesion in the lesion image patch;

[0021] A report module, configured to obtain the lesion analysis report of the lesion CT image according to the cancer transformation recognition results of the lesions in each lesion image patch.

[0022] By adopting the above technical solutions, the present invention can achieve the following technical effects:

[0023] According to the present invention, the probability feature vector of canceration occurrence for each lesion can be determined by the canceration probability recognition module of the canceration recognition model to describe the probability of canceration occurrence of the lesion, and the disease development stage feature vector of each lesion can be determined by the disease development stage recognition module to describe the development stage of the lesion. Furthermore, based on the probability of canceration occurrence of the lesion and the development stage of the lesion, the accuracy of the canceration recognition result can be improved, so that the canceration recognition result can more accurately describe the risk of canceration, improve the performance of the canceration recognition model and the judgment accuracy of the lesion. When determining the image authenticity loss function, the image authenticity loss function can be determined by means of adversarial training, so that while improving the verisimilitude of the image generated by the image generation model, the discrimination ability of the discrimination model can be improved, thereby being able to balance the performance of the two models, improve the verisimilitude of the first generated lesion CT image or the second generated lesion CT image, and improve the accuracy of subsequent comparison. When determining the disease feature loss function, the canceration recognition model can be separately trained using the sample lesion CT images of cancer patients and non-cancer patients, and during the training, the image generation model can be used to predict the generated lesion CT image of the next disease development stage based on the sample lesion CT images of any disease development stage. Thus, the accuracy of the encoding module, the canceration probability recognition module, and the disease development stage recognition module can be determined by using the error between the generated lesion CT image and the real lesion CT image. Also, when the number of samples is small, the generated images can be used to increase the number of training times and the training intensity of the encoding module, the canceration probability recognition module, and the disease development stage recognition module, improve the training effect, and during the training process, a higher weight can be assigned to the disease development stage with critical condition to specifically improve the accuracy of the canceration recognition model. When determining the first canceration risk loss function, the error between the first canceration recognition result and the canceration risk annotation information can be solved, and the error factors of the first sample probability feature vector and the first sample disease development stage feature vector can be removed to determine the actual error between the canceration risk of the lesion in the first sample feature map and the annotation information. Also, during the training process, the errors of the first sample probability feature vector, the first sample disease development stage feature vector, and the first canceration recognition result can be reduced to improve the overall accuracy of the canceration recognition model. When determining the second canceration risk loss function, the error between the second canceration recognition result and the canceration risk annotation information can be solved, and the error factors of the second sample probability feature vector and the second sample disease development stage feature vector can be removed to determine the actual error between the canceration risk of the lesion in the second sample feature map and the annotation information. Also, during the training process, the errors of the second sample probability feature vector, the second sample disease development stage feature vector, and the second canceration recognition result can be reduced to improve the overall accuracy of the canceration recognition model.

[0024] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the present invention. Other features and aspects of the present invention will become clearer according to the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other embodiments can also be obtained based on these drawings.

[0026] Figure 1 Exemplarily shown is a schematic flowchart of a multi-model combined CT image processing method according to an embodiment of the present invention;

[0027] Figure 2 Exemplarily shown is a block diagram of a multi-model combined CT image processing system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, rather than all, embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0029] The following will detail the technical solutions of the present invention with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0030] Figure 1 Exemplarily shown is a schematic flowchart of a multi-model combined CT image processing method according to an embodiment of the present invention, and the method includes:

[0031] Step S101: Input the lesion CT image into the lesion detection model to obtain the area where the lesion is located in the lesion CT image, and obtain the lesion image block according to the area where the lesion is located;

[0032] Step S102: Input the lesion image block into the encoding module of the trained cancer recognition model to obtain the feature map of the lesion image block;

[0033] Step S103: Input the feature map into the cancer probability recognition module to obtain the probability feature vector of the occurrence of cancer in the lesion in the lesion image block;

[0034] Step S104: Input the feature map into the disease progression stage recognition module to obtain the disease progression stage feature vector of the lesion in the lesion image patch.

[0035] Step S105: Input the feature map, the probability feature vector, and the disease progression stage feature vector into the decoding module of the canceration recognition module to obtain the canceration recognition result of the lesion in the lesion image patch.

[0036] Step S106: Obtain the lesion analysis report of the lesion CT image according to the canceration recognition results of the lesions in each lesion image patch.

[0037] According to the multi-model combined CT image processing method of the embodiment of the present invention, the probability feature vector of canceration of each lesion can be determined by the canceration probability recognition module of the canceration recognition model to describe the probability of canceration of the lesion, and the disease progression stage feature vector of each lesion can be determined by the disease progression stage recognition module to describe the development stage of the lesion. Furthermore, based on the probability of canceration of the lesion and the development stage of the lesion, the accuracy of the canceration recognition result can be improved, so that the canceration recognition result can more accurately describe the risk of canceration, and the performance of the canceration recognition model and the judgment accuracy of the lesion can be improved.

[0038] According to an embodiment of the present invention, in step S101, the lesion detection model is a deep learning neural network model. For example, a convolutional neural network model can be used to detect the region where the lesion is located in the CT image. For example, the lesion can be framed by a rectangular box, and the region where the rectangular box is located is the region where the lesion is located. Further, the rectangular box can be captured to obtain the lesion image patch.

[0039] According to an embodiment of the present invention, in step S102, the canceration recognition model is a deep learning neural network model. The deep learning neural network model can include an encoding module, a canceration probability recognition module, a disease progression stage recognition module, and a decoding module. Each module can include multiple layers, such as convolutional layers, activation layers, pooling layers, etc., and each module has a specific function. The encoding module can be used to encode the lesion image patch to obtain multiple feature maps. The resolution of each feature map can be lower than that of the lesion image patch. Each feature map can be regarded as an image obtained by observing the lesion image patch from a specific perspective. Therefore, multiple feature maps from multiple perspectives, that is, feature maps of multiple feature channels, can be obtained.

[0040] According to an embodiment of the present invention, in step S103, the cancer probability recognition module can process multiple feature maps to obtain a probability feature vector of the occurrence of cancer in the lesion. For example, this vector is composed of two components. The first component is the probability of the occurrence of cancer in the lesion, and the second component is the probability of non-occurrence of cancer in the lesion. The sum of the values of the two components is 1.

[0041] According to an embodiment of the present invention, in step S104, the disease development stage recognition module can be used to recognize the development stage of the lesion. The development stage feature vector can be composed of multiple components, and the number of components is the same as the number of development stages of the lesion. For example, the stages of cancer include the precancerous lesion stage, the atypical growth stage, the cancer stage, etc. Then the development stage feature vector can include three components, and each component is the probability that the development stage of the lesion is in one of the above three stages respectively. The sum of the values of the three components is 1. If a certain lesion is not a cancerous lesion, its recovery stage can also include three stages, such as the early recovery stage, the mid-term recovery stage, and the full recovery stage, etc. The development stage feature vector can also include three components. In order to distinguish from cancerous lesions, each component is the opposite of the probability that the development stage of the lesion is in one of the above three stages respectively, and the sum of the values of the three components is -1.

[0042] According to an embodiment of the present invention, in step S105, the decoding module of the cancer recognition module can process the feature map, the probability feature vector, and the disease development stage feature vector to obtain a cancer recognition result. The cancer recognition result can be information used to describe the cancer risk. For example, the numerical value of the cancer risk coefficient score. The higher this value, the higher the risk of cancer. For example, in the advanced stage of cancer with a large risk of metastasis, this value is relatively high, such as close to 1. In the early stage of cancer with a small risk of metastasis, this value is relatively low, such as close to 0.5. If the lesion is not a tumor, this value is even lower, such as close to 0.

[0043] According to an embodiment of the present invention, in step S106, the cancer recognition results of the lesions in each lesion image block can be summarized to obtain a lesion analysis report of the lesion CT image. For example, text can be generated to describe the risk of each lesion, and focus on describing the lesions with a higher cancer risk to prompt the patient and the doctor to observe and treat, which is beneficial to formulating an accurate treatment plan subsequently.

[0044] According to an embodiment of the present invention, before performing the above processing using the cancer recognition model, the cancer recognition model can be trained. The training steps of the cancer recognition model include: obtaining first sample lesion CT images of multiple disease development stages of cancer patients, and second sample lesion CT images of multiple disease development stages of non-cancer patients; inputting the first sample lesion CT images of multiple disease development stages into the encoding module of the cancer recognition model to obtain first sample feature maps; inputting the first sample feature maps into the cancer probability recognition module to obtain first sample probability feature vectors; inputting the first sample feature maps into the disease development stage recognition module to obtain first sample disease development stage feature vectors; inputting the second sample lesion CT images of multiple disease development stages into the encoding module of the cancer recognition model to obtain second sample feature maps; inputting the second sample feature maps into the cancer probability recognition module to obtain second sample probability feature vectors; inputting the second sample feature maps into the disease development stage recognition module to obtain second sample disease development stage feature vectors; obtaining a disease development similarity loss function according to the first sample probability feature vectors, the first sample disease development stage feature vectors, the second sample probability feature vectors, the second sample disease development stage feature vectors, the image generation model and the discriminant model; inputting the first sample probability feature vectors, the first sample disease development stage feature vectors and the first sample feature maps into the decoding module of the cancer recognition module to obtain a first cancer recognition result; inputting the second sample probability feature vectors, the second sample disease development stage feature vectors and the second sample feature maps into the decoding module of the cancer recognition module to obtain a second cancer recognition result; obtaining a cancer recognition loss function according to the first sample probability feature vectors, the first sample disease development stage feature vectors, the first cancer recognition result, the second sample probability feature vectors, the second sample disease development stage feature vectors and the second cancer recognition result; obtaining a loss function of the cancer recognition model according to the disease development similarity loss function and the cancer recognition loss function; training the cancer recognition model according to the loss function of the cancer recognition model to obtain a trained cancer recognition model.

[0045] According to an embodiment of the present invention, during training, the first sample lesion CT images of multiple cancer patients can be combined into a data set, and the second sample lesion CT images of multiple non-cancer patients can be combined into another data set. During each training, the first sample lesion CT images of multiple disease development stages of a cancer patient and the second sample lesion CT images of multiple disease development stages of a non-cancer patient can be selected to train the cancer recognition model.

[0046] According to an embodiment of the present invention, the first sample feature map, the first sample probability feature vector, and the first sample disease progression stage feature vector of the first sample lesion CT image can be obtained through the encoding module, the cancer probability recognition module, and the disease progression stage recognition module of the cancer recognition model respectively. Similarly, the second sample feature map, the second sample disease progression stage feature vector, and the second sample probability feature vector of the second sample lesion CT image can also be obtained.

[0047] According to an embodiment of the present invention, during training, other neural network models can be used to assist in training, thereby enhancing the training intensity, and when the samples are few, the training volume can be increased. According to the first sample probability feature vector, the first sample disease progression stage feature vector, the second sample probability feature vector, the second sample disease progression stage feature vector, the image generation model, and the discriminant model, a disease progression similarity loss function is obtained, including: inputting the first sample lesion CT image of the i-th disease progression stage, its corresponding first sample probability feature vector, and the first sample disease progression stage feature vector into the image generation model to obtain the first generated lesion CT image of the (i + 1)-th disease progression stage, where i is a positive integer; inputting the second sample lesion CT image of the j-th disease progression stage, its corresponding second sample probability feature vector, and the second sample disease progression stage feature vector into the image generation model to obtain the second generated lesion CT image of the (j + 1)-th disease progression stage, where j is a positive integer; obtaining an image authenticity loss function according to the first generated lesion CT image of the (i + 1)-th disease progression stage, the first sample lesion CT image of the (i + 1)-th disease progression stage, the second generated lesion CT image of the (j + 1)-th disease progression stage, the second sample lesion CT image of the (j + 1)-th disease progression stage, and the discriminant model; obtaining the first generated probability feature vector and the first generated disease progression stage feature vector of the first generated lesion CT image of the (i + 1)-th disease progression stage, and obtaining the second generated probability feature vector and the second generated disease progression stage feature vector of the second generated lesion CT image of the (j + 1)-th disease progression stage; obtaining a disease feature loss function according to the first sample probability feature vector, the first sample disease progression stage feature vector, the first generated probability feature vector, the first generated disease progression stage feature vector, the second sample probability feature vector, the second sample disease progression stage feature vector, the second generated probability feature vector, and the second generated disease progression stage feature vector; obtaining the disease progression similarity loss function according to the image authenticity loss function and the disease feature loss function.

[0048] According to an embodiment of the present invention, based on the first sample lesion CT image of a certain disease progression stage of a cancer patient, the first sample probability feature vector, and the first sample disease progression stage feature vector, an image generation model can be used to predict the CT image of the next disease progression stage. Then, based on the comparison between the predicted CT image of the next disease progression stage and the first sample lesion CT image of the actually captured next disease progression stage, the accuracy of the image generation model, as well as the accuracy of the first sample probability feature vector and the first sample disease progression stage feature vector, can be determined. Similarly, based on the second sample lesion CT image of a certain disease progression stage of a non-cancer patient, the second sample probability feature vector, and the second sample disease progression stage feature vector, an image generation model can be used to predict the CT image of the next disease progression stage. Then, based on the comparison between the predicted CT image of the next disease progression stage and the second sample lesion CT image of the actually captured next disease progression stage, the accuracy of the image generation model, as well as the accuracy of the second sample probability feature vector and the second sample disease progression stage feature vector, can be determined.

[0049] According to an embodiment of the present invention, the authenticity of the images generated by the image generation model can be trained first, that is, the generated images are made similar to the actually captured images in terms of image effects, so as to make the comparison results more accurate in the subsequent comparison process. The discriminant model can be used to assist in the training. Based on the first generated lesion CT image of the (i + 1)-th disease progression stage, the first sample lesion CT image of the (i + 1)-th disease progression stage, the second generated lesion CT image of the (j + 1)-th disease progression stage, the second sample lesion CT image of the (j + 1)-th disease progression stage, and the discriminant model, an image authenticity loss function is obtained, including: inputting the first generated lesion CT image, the first sample lesion CT image, the second generated lesion CT image, or the second sample lesion CT image into the discriminant model to obtain the authenticity discriminant probability distribution; obtaining the image authenticity loss function according to formula (1) ,

[0050] (1)

[0051] wherein, is the authenticity discriminant probability distribution obtained when the image input into the discriminant model is the first sample lesion CT image or the second sample lesion CT image, is the authenticity discriminant probability distribution obtained when the image input into the discriminant model is the first generated lesion CT image or the second generated lesion CT image, represents adjusting the parameters of the image generation model in the direction of minimizing , represents adjusting the parameters of the discriminant model in the direction of maximizing .

[0052] According to an embodiment of the present invention, theoretically, the probability that the discrimination model determines the first sample lesion CT image or the second sample lesion CT image as a real image is 100%, and the probability that the discrimination model determines the first generated lesion CT image or the second generated lesion CT image as a real image is 0. However, as the training process progresses, the authenticity of the generated model becomes higher and higher, resulting in an increase in the probability that the discrimination model determines the first generated lesion CT image or the second generated lesion CT image as a real image. Moreover, as the discrimination model also improves its discrimination ability during training, it can again lead to a decrease in the probability that the discrimination model determines the first generated lesion CT image or the second generated lesion CT image as a real image. Eventually, the performance of the discrimination model and the image generation model can reach a balance. Thus, even when the discrimination accuracy of the discrimination model is very high, it is still difficult to determine whether the first generated lesion CT image or the second generated lesion CT image generated by the image generation model is a real image, that is, the authenticity of the image generated by the image generation model is relatively high.

[0053] According to an embodiment of the present invention, in accordance with the above training method for balancing the performance of the image generation model and the discrimination model, the discrimination model can be adjusted in the direction of maximizing the image authenticity loss function represented by formula (1), and the image generation model can be adjusted in the direction of minimizing the image authenticity loss function. In formula (1), if the input image is the first sample lesion CT image or the second sample lesion CT image, then during the training of the discrimination model, make maximize, that is, be as close to 1 as possible, so as to be able to make maximize, that is, maximize the image authenticity loss function, thereby improving the accuracy of the discrimination model when judging real images. If the input image is the first generated lesion CT image or the second generated lesion CT image, then during the training of the discrimination model, make minimize, that is, be as close to 0 as possible, so as to be able to make maximize, thereby improving the accuracy of the discrimination model when judging generated images.

[0054] According to an embodiment of the present invention, on the other hand, in formula (1), if the input image is the first generated lesion CT image or the second generated lesion CT image, then during the training of the image generation model, make maximize, that is, be as close to 1 as possible, so as to be able to make minimize, increasing the probability that the first generated lesion CT image or the second generated lesion CT image generated by the image generation model is recognized as a real image by the discrimination model, thereby improving the authenticity of the image generated by the image generation model and enhancing the performance of the image generation model.

[0055] In this way, through adversarial training, the loss function for image authenticity can be determined, thereby improving the authenticity of the images generated by the image generation model while enhancing the discrimination ability of the discrimination model, thus balancing the performance of the two models, improving the authenticity of the first generated lesion CT image or the second generated lesion CT image, and improving the accuracy of subsequent comparison.

[0056] According to an embodiment of the present invention, the first generated probability feature vector and the first generated disease progression stage feature vector of the first generated lesion CT image in the (i + 1)-th disease progression stage can be obtained, and the second generated probability feature vector and the second generated disease progression stage feature vector of the second generated lesion CT image in the (j + 1)-th disease progression stage can be obtained. For example, the above-mentioned first generated probability feature vector, the first generated disease progression stage feature vector, the second generated probability feature vector, and the second generated disease progression stage feature vector can also be obtained through the encoding module, the cancer probability recognition module, and the disease progression stage recognition module, which will not be elaborated here.

[0057] According to an embodiment of the present invention, based on the first sample probability feature vector, the first sample disease progression stage feature vector, the first generated probability feature vector, the first generated disease progression stage feature vector, the second sample probability feature vector, the second sample disease progression stage feature vector, the second generated probability feature vector, and the second generated disease progression stage feature vector, the accuracy of the encoding module, the cancer probability recognition module, and the disease progression stage recognition module can be determined, and then the encoding module, the cancer probability recognition module, and the disease progression stage recognition module can be trained.

[0058] According to an embodiment of the present invention, based on the first sample probability feature vector, the first sample disease progression stage feature vector, the first generated probability feature vector, the first generated disease progression stage feature vector, the second sample probability feature vector, the second sample disease progression stage feature vector, the second generated probability feature vector, and the second generated disease progression stage feature vector, a disease feature loss function is obtained, including: obtaining the disease feature loss function according to formula (2) ,

[0059] (2)

[0060] wherein, is the first sample probability feature vector corresponding to the first sample lesion CT image in the i-th disease progression stage, is the labeled cancer probability feature vector of cancer patients, and n is the number of disease progression stages of cancer patients, is the first sample disease progression stage feature vector corresponding to the first sample lesion CT image in the i-th disease progression stage, is the first disease progression stage annotation feature vector corresponding to the first sample lesion CT image of the i-th disease progression stage, is the first generation probability feature vector of the first generated lesion CT image of the (i + 1)-th disease progression stage, is the first generated disease progression stage feature vector of the first generated lesion CT image of the (i + 1)-th disease progression stage, is the first disease progression stage annotation feature vector corresponding to the first sample lesion CT image of the (i + 1)-th disease progression stage, is the second sample probability feature vector corresponding to the second sample lesion CT image of the j-th disease progression stage, is the annotation canceration probability feature vector of non-cancer patients, and m is the number of disease progression stages of non-cancer patients, is the second sample disease progression stage feature vector corresponding to the second sample lesion CT image of the j-th disease progression stage, is the second disease progression stage annotation feature vector corresponding to the second sample lesion CT image of the j-th disease progression stage, is the second generation probability feature vector of the second generated lesion CT image of the (j + 1)-th disease progression stage, is the second sample disease progression stage feature vector of the second generated lesion CT image of the (j + 1)-th disease progression stage, is the second disease progression stage annotation feature vector corresponding to the second sample lesion CT image of the (j + 1)-th disease progression stage, , , , , , , and are preset weights. i, j, n, and m are all positive integers, and i ≤ n, j ≤ m.

[0061] According to an embodiment of the present invention, in formula (2), is the error between the first sample probability feature vector corresponding to the first sample lesion CT image of the i-th disease progression stage and the annotation canceration probability feature vector (for example, ). During the training process, this error can be reduced to improve the accuracy of the encoding module and the canceration probability recognition module. And, can be used as the weight of this error. Since this patient is a cancer patient, the larger the value of the disease development stage, the closer the disease development is to canceration, that is, the more critical the condition. Therefore, a higher weight can be assigned to increase the weight of the error of the first sample probability feature vector with a larger disease development stage value during training, and specifically improve the accuracy of the encoding module and the canceration probability recognition module when dealing with the situation of a larger disease development stage value. The errors of the first sample probability feature vectors of each disease development stage can be weighted and summed to obtain , which is taken as one item of the disease feature loss function.

[0062] According to an embodiment of the present invention, in formula (2), is the error between the first sample disease development stage feature vector corresponding to the first sample lesion CT image of the i-th disease development stage and the first disease development stage annotation feature vector corresponding to the first sample lesion CT image of the i-th disease development stage (that is, the vector used to describe its disease development stage obtained based on the annotation information of the first sample lesion CT image of the i-th disease development stage), The meaning of is as described above and will not be elaborated here. By weighting and summing the errors of the first sample disease development stage feature vectors corresponding to the first sample lesion CT images of each disease development stage, we obtain , which is taken as one item of the disease feature loss function. During the training process, this item is minimized to improve the accuracy of the encoding module and the disease development stage recognition module.

[0063] According to an embodiment of the present invention, in formula (2), is the error between the first generated probability feature vector of the first generated lesion CT image of the (i + 1)-th disease development stage and the annotated canceration probability feature vector, The meaning of is as described above and will not be elaborated here. By weighting and summing the errors of the first generated probability feature vectors of the first generated lesion CT images of each disease development stage, we obtain , which is taken as one item of the disease feature loss function. During the training process, this item is minimized to improve the accuracy of the encoding module, the canceration probability recognition module, and the image generation model, and in the case of a small number of samples, the images generated by the image generation model can be used to increase the number of training times and the training intensity of the encoding module and the canceration probability recognition module, thereby improving the training effect.

[0064] According to an embodiment of the present invention, in formula (2), is the error between the first generated disease progression stage feature vector of the first generated lesion CT image of the (i + 1)-th disease progression stage and the first disease progression stage annotation feature vector corresponding to the first sample lesion CT image of the (i + 1)-th disease progression stage. has the meaning as described above and will not be elaborated here. By weighted summing the errors of the first generated disease progression stage feature vectors of the first generated lesion CT images of each disease progression stage, we get , which is taken as one item of the disease feature loss function. During the training process, this item is minimized to improve the accuracy of the encoding module, the disease progression stage recognition module, and the image generation model. Moreover, in the case of a small number of samples, the images generated by the image generation model can be used to increase the number of training times and the training intensity of the encoding module and the disease progression stage recognition module, thereby improving the training effect.

[0065] According to an embodiment of the present invention, in formula (2), is the error between the second sample probability feature vector corresponding to the second sample lesion CT image of the j-th disease progression stage and the annotated cancer probability feature vector of a non-cancer patient (for example, ). During the training process, this error can be minimized to improve the accuracy of the encoding module and the cancer probability recognition module. And can be used as the weight of this error. Since the patient is a non-cancer patient, the larger the value of the disease progression stage, the closer the disease development is to recovery, that is, the lower the critical degree of the disease. Therefore, a lower weight can be assigned. Conversely, the smaller the value of the disease progression stage, the higher the weight can be assigned. Thus, the weight of the error of the second sample probability feature vector with a smaller disease progression stage value during training is increased, and the accuracy of the encoding module and the cancer probability recognition module in dealing with the situation of a larger disease progression stage value is improved specifically. The errors of the second sample probability feature vectors of each disease progression stage can be weighted and summed to obtain , which is taken as one item of the disease feature loss function.

[0066] According to an embodiment of the present invention, in formula (2), is the error between the second sample disease progression stage feature vector corresponding to the second sample lesion CT image of the j-th disease progression stage and the second disease progression stage annotation feature vector corresponding to the second sample lesion CT image of the j-th disease progression stage (that is, the vector used to describe its disease progression stage obtained based on the annotation information of the second sample lesion CT image of the j-th disease progression stage). has the meaning as described above and will not be elaborated here. By weighted summing the errors of the second sample disease progression stage feature vectors corresponding to the second sample lesion CT images of each disease progression stage, we get As one of the disease feature loss functions, during the training process, this item is minimized to improve the accuracy of the encoding module and the disease development stage recognition module.

[0067] According to an embodiment of the present invention, in formula (2), is the error between the second generated probability feature vector of the second generated lesion CT image at the (j + 1)-th disease development stage and the labeled cancer probability feature vector of non-cancer patients. The meaning of is as described above and will not be elaborated here. By weighted summation of the errors of the second generated probability feature vectors of the second generated lesion CT images at each disease development stage, we obtain , as one of the disease feature loss functions. During the training process, this item is minimized to improve the accuracy of the encoding module, the cancer probability recognition module, and the image generation model. Also, in the case of a small number of samples, the images generated by the image generation model can be used to increase the number of training times and the training intensity of the encoding module and the cancer probability recognition module, thereby improving the training effect.

[0068] According to an embodiment of the present invention, in formula (2), is the error between the second sample disease development stage feature vector of the second generated lesion CT image at the (j + 1)-th disease development stage and the second disease development stage annotation feature vector corresponding to the second sample lesion CT image at the (j + 1)-th disease development stage. The meaning of is as described above and will not be elaborated here. By weighted summation of the errors of the second sample disease development stage feature vectors of the second generated lesion CT images at each disease development stage, we obtain , as one of the disease feature loss functions. During the training process, this item is minimized to improve the accuracy of the encoding module, the disease development stage recognition module, and the image generation model. Also, in the case of a small number of samples, the images generated by the image generation model can be used to increase the number of training times and the training intensity of the encoding module and the disease development stage recognition module, thereby improving the training effect.

[0069] According to an embodiment of the present invention, by weighted summation of the above items, the disease feature loss function can be obtained. During the training process, the parameters of the cancer recognition model and the parameters of the image generation model can be adjusted by the gradient descent method, thereby improving the accuracy of the encoding module, the cancer probability recognition module, the disease development stage recognition module, and the image generation model.

[0070] In this way, the cancer recognition model can be separately trained using the sample lesion CT images of cancer patients and non-cancer patients. During the training, the image generation model can predict the generated lesion CT image of the next disease development stage based on the sample lesion CT image at any disease development stage. Thus, the accuracy of the encoding module, the cancer probability recognition module, and the disease development stage recognition module can be determined using the error between the generated lesion CT image and the real lesion CT image. Also, when the number of samples is small, the generated images can be used to increase the number of training times and the training intensity of the encoding module, the cancer probability recognition module, and the disease development stage recognition module, improving the training effect. Additionally, during the training process, a higher weight can be assigned to the disease development stage with a critical condition to specifically improve the accuracy of the cancer recognition model.

[0071] According to an embodiment of the present invention, after obtaining the image authenticity loss function and the disease feature loss function, the two can be weighted and summed to obtain the disease development similarity loss function.

[0072] According to an embodiment of the present invention, the decoding module can process the first sample probability feature vector, the first sample disease development stage feature vector, and the first sample feature map to obtain the first cancer recognition result. Similarly, the decoding module can process the second sample probability feature vector, the second sample disease development stage feature vector, and the second sample feature map to obtain the second cancer recognition result. Furthermore, the cancer recognition loss function can be obtained based on the error between the first cancer recognition result and the second cancer recognition result.

[0073] According to an embodiment of the present invention, obtaining the cancer recognition loss function based on the first sample probability feature vector, the first sample disease development stage feature vector, the first cancer recognition result, the second sample probability feature vector, the second sample disease development stage feature vector, and the second cancer recognition result includes: obtaining the first cancer risk loss function based on the first sample probability feature vector, the first sample disease development stage feature vector, and the first cancer recognition result; obtaining the second cancer risk loss function based on the second sample probability feature vector, the second sample disease development stage feature vector, and the second cancer recognition result; and obtaining the cancer recognition loss function based on the first cancer risk loss function and the second cancer risk loss function.

[0074] According to an embodiment of the present invention, obtaining the first cancer risk loss function based on the first sample probability feature vector, the first sample disease development stage feature vector, and the first cancer recognition result includes: obtaining the first cancer risk loss function according to formula (3) ,

[0075] (3)

[0076] Among them, is the first cancer recognition result of the first sample lesion CT image in the i-th disease development stage, is the cancer risk annotation information of the first sample lesion CT image in the i-th disease development stage, is the first sample probability feature vector corresponding to the first sample lesion CT image in the i-th disease development stage, is the transposed vector of is the annotated cancer probability feature vector of cancer patients, is the first sample disease development stage feature vector corresponding to the first sample lesion CT image in the i-th disease development stage, is the transposed vector of is the first disease development stage annotation feature vector corresponding to the first sample lesion CT image in the i-th disease development stage. n is the number of disease development stages of cancer patients, and both i and n are positive integers, and i ≤ n.

[0077] According to an embodiment of the present invention, in formula (3), is the error between the first cancer recognition result of the first sample lesion CT image in the i-th disease development stage and the cancer risk annotation information (for example, the risk coefficient score of the manually annotated cancer). During the training process, the reduction of this error can reduce the first cancer risk loss function, thereby improving the accuracy of the decoding module, the cancer probability recognition module, the disease development stage recognition module, and the encoding module. Further, in the process of decoding to obtain the first cancer recognition result, the first sample probability feature vector and the first sample disease development stage feature vector are referred to, and there are also errors in these two vectors. Therefore, to determine the cancer risk in the first sample feature map, the error factors of the above two vectors can be excluded, that is, the cosine similarity between the two and the corresponding annotated feature vectors is used as the denominator, so as to remove the error factors of the two vectors and determine the actual error between the cancer risk of the lesion in the first sample feature map and the annotation. And using the two cosine similarities as the denominator can also increase the value of the cosine similarity during the training process, reduce the errors of the first sample probability feature vector and the first sample disease development stage feature vector, and improve the overall accuracy of the cancer recognition model. The meaning of is as described above and will not be elaborated here. By performing weighted summation on

[0078] In this way, the error between the first canceration recognition result and the canceration risk annotation information can be solved, and the error factors of the first sample probability feature vector and the first sample disease development stage feature vector can be removed, so as to determine the actual error between the canceration risk of the lesion in the first sample feature map and the annotation information. Moreover, during the training process, the errors of the first sample probability feature vector, the first sample disease development stage feature vector, and the first canceration recognition result can be reduced, improving the overall accuracy of the canceration recognition model.

[0079] According to an embodiment of the present invention, a second canceration risk loss function is obtained based on the second sample probability feature vector, the second sample disease development stage feature vector, and the second canceration recognition result, including: obtaining the second canceration risk loss function according to formula (4) ,

[0080] (4)

[0081] wherein, is the second canceration recognition result of the second sample lesion CT image at the j-th disease development stage, is the canceration risk annotation information of the second sample lesion CT image at the j-th disease development stage, is the second sample probability feature vector corresponding to the second sample lesion CT image at the j-th disease development stage, is the annotated canceration probability feature vector of non-cancer patients, m is the number of disease development stages of non-cancer patients, is the second sample disease development stage feature vector corresponding to the second sample lesion CT image at the j-th disease development stage, is the second disease development stage annotation feature vector corresponding to the second sample lesion CT image at the j-th disease development stage, is the transposed vector of is the transposed vector of, j and m are both positive integers, and j ≤ m.

[0082] According to an embodiment of the present invention, The error between the second canceration recognition result of the second sample lesion CT image at the j-th disease development stage and the canceration risk annotation information (e.g., the risk coefficient score of canceration manually annotated) of the second sample lesion CT image at the j-th disease development stage. During the training process, the reduction of this error can reduce the second canceration risk loss function, thereby improving the accuracy of the decoding module, the canceration probability recognition module, the disease development stage recognition module, and the encoding module. Further, during the process of decoding to obtain the second canceration recognition result, the second sample probability feature vector and the second sample disease development stage feature vector are referred to, and there are also errors in these two vectors. Therefore, to determine the canceration risk in the second sample feature map, the error factors of the above two vectors can be excluded, that is, the cosine similarity between the two and the corresponding annotated feature vector is used as the denominator, so as to remove the error factors of the two vectors and determine the actual error between the canceration risk of the lesion in the second sample feature map and the annotation. Moreover, using the two cosine similarities as the denominator can also increase the value of the cosine similarity during the training process, reduce the errors of the second sample probability feature vector and the second sample disease development stage feature vector, and improve the overall accuracy of the canceration recognition model. The meaning of which is as described above and will not be elaborated here. By performing weighted summation, the second canceration risk loss function can be obtained.

[0083] In this way, the error between the second canceration recognition result and the canceration risk annotation information can be solved, the error factors of the second sample probability feature vector and the second sample disease development stage feature vector can be removed, the actual error between the canceration risk of the lesion in the second sample feature map and the annotation information can be determined, the errors of the second sample probability feature vector and the second sample disease development stage feature vector, as well as the error of the second canceration recognition result, can be reduced during the training process, and the overall accuracy of the canceration recognition model can be improved.

[0084] According to an embodiment of the present invention, the first canceration risk loss function and the second canceration risk loss function can be weighted and summed to obtain the canceration recognition loss function. Further, the disease development similarity loss function and the canceration recognition loss function can be weighted and summed to obtain the loss function of the canceration recognition model, and then the parameters of the canceration recognition model can be adjusted by the gradient descent method to make the loss function of the canceration recognition model shrink, so as to train the canceration recognition model. After multiple trainings, the trained canceration recognition model can be obtained.

[0085] The multi-model combined CT image processing method according to an embodiment of the present invention can determine the probability feature vector of canceration for each lesion through the canceration probability recognition module of the canceration recognition model to describe the probability of canceration of the lesion, and determine the disease development stage feature vector of each lesion through the disease development stage recognition module to describe the development stage of the lesion. Furthermore, based on the probability of canceration of the lesion and the development stage of the lesion, the accuracy of the canceration recognition result can be improved, so that the canceration recognition result can more accurately describe the risk of canceration, improve the performance of the canceration recognition model and the judgment accuracy of the lesion. When determining the image authenticity loss function, the image authenticity loss function can be determined through adversarial training, so as to improve the vividness of the image generated by the image generation model and the discrimination ability of the discrimination model at the same time, so as to balance the performance of the two models, improve the vividness of the first generated lesion CT image or the second generated lesion CT image, and improve the accuracy of subsequent comparison. When determining the disease feature loss function, the canceration recognition model can be trained separately using the sample lesion CT images of cancer patients and non-cancer patients, and during the training, the image generation model can be used to predict the generated lesion CT image of the next disease development stage based on the sample lesion CT images of any disease development stage. Therefore, the error between the generated lesion CT image and the real lesion CT image can be used to determine the accuracy of the encoding module, the canceration probability recognition module and the disease development stage recognition module. When the number of samples is small, the generated images can also be used to increase the number of training times and the training intensity of the encoding module, the canceration probability recognition module and the disease development stage recognition module, improve the training effect, and during the training process, a higher weight can be assigned to the disease development stage with critical condition to improve the accuracy of the canceration recognition model specifically. When determining the first canceration risk loss function, the error between the first canceration recognition result and the canceration risk annotation information can be solved, and the error factors of the first sample probability feature vector and the first sample disease development stage feature vector can be removed to determine the actual error between the canceration risk of the lesion in the first sample feature map and the annotation information. During the training process, the error of the first sample probability feature vector and the first sample disease development stage feature vector, as well as the error of the first canceration recognition result, can also be reduced to improve the overall accuracy of the canceration recognition model. When determining the second canceration risk loss function, the error between the second canceration recognition result and the canceration risk annotation information can be solved, and the error factors of the second sample probability feature vector and the second sample disease development stage feature vector can be removed to determine the actual error between the canceration risk of the lesion in the second sample feature map and the annotation information. During the training process, the error of the second sample probability feature vector and the second sample disease development stage feature vector, as well as the error of the second canceration recognition result, can also be reduced to improve the overall accuracy of the canceration recognition model.

[0086] Figure 2The block diagram of a multi-model combined CT image processing system according to an embodiment of the present invention is exemplarily shown. The system includes:

[0087] A detection module, configured to input a lesion CT image into a lesion detection model, obtain the region where the lesion is located in the lesion CT image, and obtain a lesion image block according to the region where the lesion is located;

[0088] An input module, configured to input the lesion image block into the encoding module of the trained cancer recognition model to obtain a feature map of the lesion image block;

[0089] A probability module, configured to input the feature map into a cancer probability recognition module to obtain a probability feature vector of the occurrence of cancer in the lesion in the lesion image block;

[0090] A stage module, configured to input the feature map into a disease progression stage recognition module to obtain a disease progression stage feature vector of the lesion in the lesion image block;

[0091] A result module, configured to input the feature map, the probability feature vector, and the disease progression stage feature vector into the decoding module of the cancer recognition model to obtain a cancer recognition result of the lesion in the lesion image block;

[0092] A report module, configured to obtain a lesion analysis report of the lesion CT image according to the cancer recognition results of the lesions in each lesion image block.

[0093] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The object of the present invention has been fully and effectively achieved. The function and structural principle of the present invention have been shown and described in the embodiments. Without departing from the principle, the embodiments of the present invention can have any deformation or modification.

[0094] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-model combined CT image processing method, characterized in that Including: Input the CT image of the lesion into the lesion detection model to obtain the area where the lesion is located in the CT image of the lesion, and obtain the lesion image block according to the area where the lesion is located; Input the lesion image block into the encoding module of the trained canceration recognition model to obtain the feature map of the lesion image block; Input the feature map into the canceration probability recognition module to obtain the probability feature vector of the occurrence of canceration of the lesion in the lesion image block; Input the feature map into the disease progression stage recognition module to obtain the disease progression stage feature vector of the lesion in the lesion image block; Input the feature map, the probability feature vector, and the disease progression stage feature vector into the decoding module of the canceration recognition module to obtain the canceration recognition result of the lesion in the lesion image block; Obtain the lesion analysis report of the CT image of the lesion according to the canceration recognition results of the lesions in each lesion image block: The training steps of the canceration recognition model include: Obtain the first sample CT images of the lesions at multiple disease progression stages of cancer patients, and the second sample CT images of the lesions at multiple disease progression stages of non-cancer patients; Input the first sample CT images of the lesions at multiple disease progression stages into the encoding module of the canceration recognition model to obtain the first sample feature map; Input the first sample feature map into the canceration probability recognition module to obtain the first sample probability feature vector; Input the first sample feature map into the disease progression stage recognition module to obtain the first sample disease progression stage feature vector; Input the second sample CT images of the lesions at multiple disease progression stages into the encoding module of the canceration recognition model to obtain the second sample feature map; Input the second sample feature map into the canceration probability recognition module to obtain the second sample probability feature vector; Input the second sample feature map into the disease progression stage recognition module to obtain the second sample disease progression stage feature vector; Obtain the disease progression similarity loss function according to the first sample probability feature vector, the first sample disease progression stage feature vector, the second sample probability feature vector, the second sample disease progression stage feature vector, the image generation model, and the discriminant model; Input the first sample probability feature vector, the first sample disease progression stage feature vector, and the first sample feature map into the decoding module of the canceration recognition module to obtain the first canceration recognition result; Input the second sample probability feature vector, the second sample disease progression stage feature vector, and the second sample feature map into the decoding module of the canceration recognition module to obtain the second canceration recognition result; Obtain the canceration recognition loss function according to the first sample probability feature vector, the first sample disease progression stage feature vector, the first canceration recognition result, the second sample probability feature vector, the second sample disease progression stage feature vector, and the second canceration recognition result; Obtain the loss function of the canceration recognition model according to the disease progression similarity loss function and the canceration recognition loss function; Train the canceration recognition model according to the loss function of the canceration recognition model to obtain the trained canceration recognition model: Based on the first sample probability feature vector, the first sample disease development stage feature vector, the second sample probability feature vector, the second sample disease development stage feature vector, the image generation model, and the discriminant model, obtain a disease development similarity loss function, including: Input the first sample lesion CT image of the i-th disease development stage, its corresponding first sample probability feature vector, and the first sample disease development stage feature vector into the image generation model to obtain the first generated lesion CT image of the (i + 1)-th disease development stage, where i is a positive integer; Input the second sample lesion CT image of the j-th disease development stage, its corresponding second sample probability feature vector, and the second sample disease development stage feature vector into the image generation model to obtain the second generated lesion CT image of the (j + 1)-th disease development stage, where j is a positive integer; Based on the first generated lesion CT image of the (i + 1)-th disease development stage, the first sample lesion CT image of the (i + 1)-th disease development stage, the second generated lesion CT image of the (j + 1)-th disease development stage, the second sample lesion CT image of the (j + 1)-th disease development stage, and the discriminant model, obtain an image authenticity loss function; Obtain the first generated probability feature vector and the first generated disease development stage feature vector of the first generated lesion CT image of the (i + 1)-th disease development stage, and obtain the second generated probability feature vector and the second generated disease development stage feature vector of the second generated lesion CT image of the (j + 1)-th disease development stage; Based on the first sample probability feature vector, the first sample disease development stage feature vector, the first generated probability feature vector, the first generated disease development stage feature vector, the second sample probability feature vector, the second sample disease development stage feature vector, the second generated probability feature vector, and the second generated disease development stage feature vector, obtain a disease feature loss function; Based on the image authenticity loss function and the disease feature loss function, obtain the disease development similarity loss function.

2. The multi-model combined CT image processing method according to claim 1, wherein, Based on the first generated lesion CT image of the (i + 1)-th disease development stage, the first sample lesion CT image of the (i + 1)-th disease development stage, the second generated lesion CT image of the (j + 1)-th disease development stage, the second sample lesion CT image of the (j + 1)-th disease development stage, and the discriminant model, obtain an image authenticity loss function, including: Input the first generated lesion CT image, the first sample lesion CT image, the second generated lesion CT image, or the second sample lesion CT image into the discriminant model to obtain an authenticity discriminant probability distribution; According to the formula ； Obtain the image authenticity loss function , where is the authenticity discrimination probability distribution obtained when the image input to the discrimination model is the first sample lesion CT image or the second sample lesion CT image, is the authenticity discrimination probability distribution obtained when the image input to the discrimination model is the first generated lesion CT image or the second generated lesion CT image, represents adjusting the parameters of the image generation model in the direction that minimizes , represents adjusting the parameters of the discrimination model in the direction that maximizes .

3. The multi-model combined CT image processing method according to claim 1, characterized in that Based on the first sample probability feature vector, the first sample disease development stage feature vector, the first generated probability feature vector, the first generated disease development stage feature vector, the second sample probability feature vector, the second sample disease development stage feature vector, the second generated probability feature vector, and the second generated disease development stage feature vector, obtain a disease feature loss function, including: According to the formula ； Obtain the disease condition feature loss function , where is the first sample probability feature vector corresponding to the first sample lesion CT image of the i-th disease development stage, is the labeled cancer probability feature vector of cancer patients, n is the number of disease development stages of cancer patients, is the first sample disease development stage feature vector corresponding to the first sample lesion CT image of the i-th disease development stage, is the first disease development stage annotation feature vector corresponding to the first sample lesion CT image of the i-th disease development stage, is the first generated probability feature vector of the first generated lesion CT image of the (i + 1)-th disease development stage, is the first generated disease development stage feature vector of the first generated lesion CT image of the (i + 1)-th disease development stage, is the first disease development stage annotation feature vector corresponding to the first sample lesion CT image of the (i + 1)-th disease development stage, is the second sample probability feature vector corresponding to the second sample lesion CT image of the j-th disease development stage, is the labeled cancer probability feature vector of non-cancer patients, m is the number of disease development stages of non-cancer patients, is the second sample disease development stage feature vector corresponding to the second sample lesion CT image of the j-th disease development stage, is the second disease development stage annotation feature vector corresponding to the second sample lesion CT image of the j-th disease development stage, is the second generated probability feature vector of the second generated lesion CT image of the (j + 1)-th disease development stage, is the second sample disease development stage feature vector of the second generated lesion CT image of the (j + 1)-th disease development stage, is the second disease development stage annotation feature vector corresponding to the second sample lesion CT image of the (j + 1)-th disease development stage, , , , , , , and are preset weights where i, j, n, and m are all positive integers, and i ≤ n, j ≤ m.

4. The multi-model combined CT image processing method according to claim 1, wherein Obtain a canceration recognition loss function according to the first sample probability feature vector, the first sample disease progression stage feature vector, the first canceration recognition result, the second sample probability feature vector, the second sample disease progression stage feature vector, and the second canceration recognition result, including: Obtain a first canceration risk loss function according to the first sample probability feature vector, the first sample disease progression stage feature vector, and the first canceration recognition result; Obtain a second canceration risk loss function according to the second sample probability feature vector, the second sample disease progression stage feature vector, and the second canceration recognition result; Obtain a canceration recognition loss function according to the first canceration risk loss function and the second canceration risk loss function.

5. The multi-model combined CT image processing method according to claim 4, wherein Obtain a first canceration risk loss function according to the first sample probability feature vector, the first sample disease progression stage feature vector, and the first canceration recognition result, including: According to the formula ； Obtain the first carcinogenesis risk loss function , where is the first carcinogenesis recognition result of the first sample lesion CT image in the i-th disease development stage, is the carcinogenesis risk annotation information of the first sample lesion CT image in the i-th disease development stage, is the first sample probability feature vector corresponding to the first sample lesion CT image in the i-th disease development stage, is 's transposed vector, is the annotated carcinogenesis probability feature vector of cancer patients, is the first sample disease development stage feature vector corresponding to the first sample lesion CT image in the i-th disease development stage, is 's transposed vector, is the first disease development stage annotation feature vector corresponding to the first sample lesion CT image in the i-th disease development stage, n is the number of disease development stages of cancer patients, both i and n are positive integers, and i ≤ n.

6. The multi-model combined CT image processing method according to claim 4, wherein Obtain a second canceration risk loss function according to the second sample probability feature vector, the second sample disease progression stage feature vector, and the second canceration recognition result, including: According to the formula ； Obtain the second carcinogenesis risk loss function , where is the second carcinogenesis recognition result of the second sample lesion CT image at the j-th disease development stage, is the carcinogenesis risk annotation information of the second sample lesion CT image at the j-th disease development stage, is the second sample probability feature vector corresponding to the second sample lesion CT image at the j-th disease development stage, is the annotated carcinogenesis probability feature vector of non-cancer patients, and m is the number of disease development stages of non-cancer patients, is the second sample disease development stage feature vector corresponding to the second sample lesion CT image at the j-th disease development stage, is the second disease development stage annotation feature vector corresponding to the second sample lesion CT image at the j-th disease development stage, is 's transposed vector, is 's transposed vector. Both j and m are positive integers, and j ≤ m.

7. A multi-model combined CT image processing system for performing the method according to any one of claims 1-6, characterized in that, Including: A detection module, configured to input a lesion CT image into a lesion detection model, obtain the region where the lesion is located in the lesion CT image, and obtain a lesion image block according to the region where the lesion is located; An input module, configured to input the lesion image block into an encoding module of the trained canceration recognition model to obtain a feature map of the lesion image block; A probability module, configured to input the feature map into a canceration probability recognition module to obtain a probability feature vector of the occurrence of canceration of the lesion in the lesion image block; A stage module, configured to input the feature map into a disease progression stage recognition module to obtain a disease progression stage feature vector of the lesion in the lesion image block; A result module, configured to input the feature map, the probability feature vector, and the disease progression stage feature vector into a decoding module of the canceration recognition module to obtain a canceration recognition result of the lesion in the lesion image block; A report module, configured to obtain a lesion analysis report of the lesion CT image according to the canceration recognition results of the lesions in each lesion image block.

Images