Tumor prognosis prediction model training and using method, device, equipment and medium

Abstract

The invention relates to a tumor prognosis prediction model training method, a tumor prognosis prediction model using method, a tumor prognosis prediction model training device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a first sample image and a second sample image of different sample users for a target organ, and a tumor prognosis tag corresponding to the second sample image; pre-training a diffusion model according to the first sample image so as to enable the diffusion model to learn the feature extraction capability of the target organ; the diffusion model comprises a coding network for feature extraction; generating a tumor prognosis prediction model comprising a prognosis classification network and a pre-trained coding network; and according to the second sample image and the corresponding tumor prognosis label, performing fine adjustment on the tumor prognosis prediction model to obtain a trained tumor prognosis prediction model. According to the invention, the feature extraction capability of the model can be improved, and the prediction performance of the tumor prognosis prediction model is further improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for training a tumor prognosis prediction model, a method for using the model, an apparatus, a computer device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the development of artificial intelligence technology, technologies that utilize models to perform related tasks have emerged, especially in the medical field, where tumor prognosis prediction models are used to predict tumor prognosis. The key to the predictive performance of tumor prognosis prediction models lies in their training.

[0003] In traditional techniques, the feature extraction capabilities of the trained tumor prognosis prediction models are limited, resulting in poor predictive performance. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, method of use, device, computer equipment, computer-readable storage medium, and computer program product for training a tumor prognosis prediction model that can improve the feature extraction capability of the model, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a method for training a tumor prognosis prediction model, including:

[0006] Obtain first and second sample images of the target organ from different sample users, as well as the tumor prognosis label corresponding to the second sample image;

[0007] The diffusion model is pre-trained based on the first sample image to enable the diffusion model to learn the feature extraction capability of the target organ; the diffusion model includes an encoding network for feature extraction.

[0008] Generate a tumor prognostic prediction model that includes a prognostic classification network and a pre-trained coding network;

[0009] Based on the second sample image and the corresponding tumor prognosis label, the tumor prognosis prediction model is fine-tuned to obtain a trained tumor prognosis prediction model.

[0010] In one embodiment, the pre-training of the diffusion model based on the first sample image includes: during the forward propagation of the diffusion model, for each time step, adding noise to the original sample image corresponding to the time step to obtain a mixed sample image corresponding to the time step, and using the mixed sample image corresponding to the time step as the original sample image corresponding to the next time step; the original sample image corresponding to the first time step is the first sample image; during the backward propagation of the diffusion model, for each time step, denoising the mixed sample image corresponding to the time step, and pre-training the diffusion model based on the denoising result and the original sample image corresponding to the time step.

[0011] In one embodiment, the diffusion model further includes a decoding network for image reconstruction; the step of denoising the mixed sample image corresponding to each time step and pre-training the diffusion model based on the denoising result and the original sample image corresponding to the time step includes: for each time step, extracting features from the mixed sample image corresponding to the time step based on the encoding network to obtain the features to be reconstructed corresponding to the time step; reconstructing the image based on the features to be reconstructed corresponding to the time step based on the decoding network to obtain the reconstructed image corresponding to the time step; the reconstructed image corresponding to the time step is the denoising result corresponding to the time step; and pre-training the diffusion model based on the reconstructed image and the original sample image corresponding to the time step.

[0012] In one embodiment, the step of reconstructing the image of the feature to be reconstructed corresponding to the time step based on the decoding network to obtain the reconstructed image of the time step includes: performing concatenated encoding of multi-scale features on the mixed sample image corresponding to the time step based on the encoding network to obtain the feature to be reconstructed corresponding to the time step; wherein, in at least one feature encoding process, global attention enhancement is performed on the encoding process of the previous feature encoding result to obtain the current feature encoding result.

[0013] In one embodiment, the step of reconstructing the image of the feature to be reconstructed corresponding to the time step based on the decoding network to obtain the reconstructed image of the time step includes: performing cascaded decoding of multi-scale features of the feature to be reconstructed corresponding to the time step based on the decoding network to obtain the reconstructed image of the time step; wherein, in at least one feature decoding process, global attention enhancement is performed on the decoding process of the previous feature decoding result to obtain the current feature decoding result.

[0014] Secondly, embodiments of this application provide a method for using a tumor prognosis prediction model, including:

[0015] Acquire test images of the target organ from the target user;

[0016] The image to be tested is input into a trained tumor prognosis prediction model to obtain the tumor prognosis prediction result for the target user; wherein the trained tumor prognosis prediction model is determined based on any tumor prognosis prediction model training method provided in the first aspect embodiment.

[0017] Thirdly, embodiments of this application provide a tumor prognosis prediction model training device, including:

[0018] The first acquisition module is used to acquire first sample images and second sample images of target organs from different sample users, as well as tumor prognosis labels corresponding to the second sample images.

[0019] A pre-training module is used to pre-train the diffusion model based on the first sample image, so that the diffusion model learns the feature extraction capability of the target organ; the diffusion model includes an encoding network for feature extraction;

[0020] The model generation module is used to generate tumor prognostic prediction models that include a prognostic classification network and a pre-trained coding network.

[0021] The model fine-tuning module is used to fine-tune the tumor prognosis prediction model based on the second sample image and the corresponding tumor prognosis label to obtain a trained tumor prognosis prediction model.

[0022] Fourthly, embodiments of this application provide an apparatus for using a tumor prognosis prediction model, comprising:

[0023] The second acquisition module is used to acquire the test image of the target organ from the target user;

[0024] The model prediction module is used to input the image to be tested into a trained tumor prognosis prediction model to obtain the tumor prognosis prediction result of the target user; wherein, the trained tumor prognosis prediction model is determined based on any tumor prognosis prediction model training device provided in the third aspect embodiment.

[0025] Fifthly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0026] Obtain first and second sample images of the target organ from different sample users, as well as the tumor prognosis label corresponding to the second sample image;

[0027] The diffusion model is pre-trained based on the first sample image to enable the diffusion model to learn the feature extraction capability of the target organ; the diffusion model includes an encoding network for feature extraction.

[0028] Generate a tumor prognostic prediction model that includes a prognostic classification network and a pre-trained coding network;

[0029] Based on the second sample image and the corresponding tumor prognosis label, the tumor prognosis prediction model is fine-tuned to obtain a trained tumor prognosis prediction model.

[0030] or,

[0031] Acquire test images of the target organ from the target user;

[0032] The image to be tested is input into a trained tumor prognosis prediction model to obtain the tumor prognosis prediction result for the target user; wherein the trained tumor prognosis prediction model is determined based on any tumor prognosis prediction model training method provided in the first aspect embodiment.

[0033] Sixthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0034] Obtain first and second sample images of the target organ from different sample users, as well as the tumor prognosis label corresponding to the second sample image;

[0035] The diffusion model is pre-trained based on the first sample image to enable the diffusion model to learn the feature extraction capability of the target organ; the diffusion model includes an encoding network for feature extraction.

[0036] Generate a tumor prognostic prediction model that includes a prognostic classification network and a pre-trained coding network;

[0037] Based on the second sample image and the corresponding tumor prognosis label, the tumor prognosis prediction model is fine-tuned to obtain a trained tumor prognosis prediction model.

[0038] or,

[0039] Acquire test images of the target organ from the target user;

[0040] The image to be tested is input into a trained tumor prognosis prediction model to obtain the tumor prognosis prediction result for the target user; wherein the trained tumor prognosis prediction model is determined based on any tumor prognosis prediction model training method provided in the first aspect embodiment.

[0041] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0042] Obtain first and second sample images of the target organ from different sample users, as well as the tumor prognosis label corresponding to the second sample image;

[0043] The diffusion model is pre-trained based on the first sample image to enable the diffusion model to learn the feature extraction capability of the target organ; the diffusion model includes an encoding network for feature extraction.

[0044] Generate a tumor prognostic prediction model that includes a prognostic classification network and a pre-trained coding network;

[0045] Based on the second sample image and the corresponding tumor prognosis label, the tumor prognosis prediction model is fine-tuned to obtain a trained tumor prognosis prediction model.

[0046] or,

[0047] Acquire test images of the target organ from the target user;

[0048] The image to be tested is input into a trained tumor prognosis prediction model to obtain the tumor prognosis prediction result for the target user; wherein the trained tumor prognosis prediction model is determined based on any tumor prognosis prediction model training method provided in the first aspect embodiment.

[0049] The aforementioned method, application method, apparatus, computer equipment, computer-readable storage medium, and computer program product for training the tumor prognosis prediction model first pre-trains the diffusion model based on first sample images. Since the first sample images required for pre-training do not require corresponding label data, the difficulty of obtaining a large number of first sample images is reduced. Pre-training the diffusion model based on unlabeled first sample images allows the encoding network in the diffusion model to gradually learn prior knowledge of image detail information, enabling it to extract more image details and thus improving the feature extraction capability of the encoding network. Further, a tumor prognosis prediction model including the pre-trained encoding network and prognosis classification network is generated. Then, a second sample image including tumor prognosis labels is used to fine-tune the tumor prognosis prediction model, thereby transferring the encoding network with strong feature extraction capabilities to the tumor prognosis scenario, enabling the trained tumor prognosis prediction model to gradually possess tumor prognosis prediction functionality. Since the fine-tuning stage typically requires only a small number of samples, the accuracy of prediction results in the tumor prognosis prediction scenario can be improved with minimal label collection costs, significantly enhancing the predictive performance of the tumor prognosis prediction model. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating a tumor prognosis prediction model training method in one embodiment;

[0052] Figure 2 This is a schematic diagram of the process for determining a sample in one embodiment;

[0053] Figure 3 This is a flowchart illustrating step S120 in one embodiment;

[0054] Figure 4 This is a flowchart illustrating step S122 in one embodiment;

[0055] Figure 5 This is a schematic diagram of the pre-training process of a diffusion model in one embodiment;

[0056] Figure 6 Here is a block diagram of the UNet structure in one embodiment;

[0057] Figure 7 This is a structural block diagram of a tumor prognosis prediction model in one embodiment;

[0058] Figure 8 This is a flowchart illustrating the method of using a tumor prognosis prediction model in one embodiment;

[0059] Figure 9 This is a structural block diagram of a tumor prognosis prediction model training device in one embodiment;

[0060] Figure 10 A structural block diagram of a device used in a tumor prognosis prediction model in one embodiment;

[0061] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0063] The technical solutions provided in this application are applicable to the training and use of tumor prognosis prediction models. For ease of understanding, the training process of the tumor prognosis prediction model will first be described in detail.

[0064] In one exemplary embodiment, such as Figure 1 As shown, a method for training a tumor prognosis prediction model is provided. This embodiment illustrates the method using a terminal as an example. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0065] S110, acquire first and second sample images of the target organ from different sample users, as well as the tumor prognosis label corresponding to the second sample image.

[0066] In this context, a sample user can be understood as a user who provides sample images for model training. There must be at least one sample user, but to ensure sample diversity and further improve the accuracy and generality of the subsequently trained model, there are usually multiple sample users.

[0067] In this application embodiment, the tumor category corresponding to each first sample image may include at least one type to reduce the difficulty of obtaining the first sample image. This application does not limit the number or specific categories of tumor categories.

[0068] In this embodiment, the target organ can be the lung, liver, stomach, or other organs, as long as the target organ corresponding to the first sample image and the second sample image is the same.

[0069] The first sample image and the second sample image can be at least one of the sample images presented in different formats, such as CT (Computed Tomography) images, MRI (Magnetic Resonance Imaging) images, and ultrasound images of the sample user.

[0070] In this embodiment, the first sample image is used during pre-training, and there is no need to annotate the tumor prognosis label for each first sample image. The second sample image is used during fine-tuning, which requires fine-tuning based on the tumor prognosis label. Since the number of samples required in the fine-tuning stage is relatively small, the acquisition of sample images is relatively easy. It is worth noting that the tumor prognosis label can be automatically determined using at least one method in traditional techniques, or through expert manual annotation, etc., and this application does not impose any limitations on this.

[0071] Tumor prognostic labels refer to categorical data that reflects the prognostic effects of treatment for users with tumors. For example, the effectiveness of neoadjuvant therapy can be reflected by two indicators: MPR (major pathological response) and PCR (pathological complete response). Therefore, tumor prognostic labels can be categorical labels obtained by whether a user achieves MPR or PCR after receiving neoadjuvant therapy.

[0072] For example, the tumor could be non-small cell lung cancer, the target organ could be the lung, the first sample image and the second sample image could be lung CT scans, and the tumor prognostic label could be classification data showing the prognostic effect of a user with non-small cell lung cancer after receiving neoadjuvant therapy. Accordingly, the tumor prognostic prediction model obtained by the training method of this application embodiment is a prognostic prediction model for non-small cell lung cancer, which can be used in scenarios where the effect classification of a user with non-small cell lung cancer after receiving neoadjuvant therapy is predicted.

[0073] In real-world scenarios, to provide stronger support for subsequent model training, more than 400,000 first sample images can be collected in advance. Based on these sample images, a large-scale sample image set can be formed, enabling the model to learn rich prior knowledge related to tumors and build a strong knowledge reserve foundation. This reduces the problem that the feature extraction capability of the pre-trained diffusion model is limited due to the small number of sample images, thus failing to fully exploit medical image information.

[0074] For example, to improve the diversity of the first sample images, medical image images of the same organ from different regions can also be obtained as the first sample images. See, for example, [link to relevant documentation]. Figure 2 We collected 485,885 lung CT scans from multiple medical centers, covering diverse cases from multiple centers to ensure the richness and representativeness of the data. Based on this, we used image quality control methods and natural language processing technology to deeply process the data. Through image quality control, low-quality and interfering sample images were removed, ultimately retaining 105,184 high-quality data points, providing excellent data support for the pre-training of the diffusion model. This large-scale initial sample image set provided rich learning material for the model pre-training, allowing the pre-trained diffusion model to fully utilize the prior knowledge from the large-scale data. This enabled the diffusion model to learn more comprehensive and stable lung-related prior knowledge, significantly improving the generalization ability of the diffusion model.

[0075] S120, the diffusion model is pre-trained based on the first sample image so that the diffusion model learns the ability to extract features from the target organ; the diffusion model includes an encoding network for feature extraction.

[0076] The diffusion model includes an encoding network, which encodes image features to extract key features from the image. Therefore, during the pre-training of the diffusion model using the first sample image, the parameters of the encoding network are continuously optimized, thereby gradually improving the feature extraction capability of the encoding network.

[0077] Diffusion models, or diffusion models, are a type of machine learning model. Their working mechanism consists of two phases: The first phase corresponds to the forward process, also known as the forward diffusion process. This involves multiple cascading time steps. For the first time step, noise is added to the original image, resulting in a noisy image. For each subsequent time step, further noise is added to the noisy image output from the previous time step. The second phase corresponds to the reverse process, also known as the reverse diffusion process. This phase also involves multiple time steps corresponding to the forward process. In the reverse process, noise is denoised using the mixed sample images from the corresponding time steps in the forward process, attempting to reconstruct the image before noise addition in the forward process.

[0078] Understandably, the encoding network needs to be trained once for each time step in the reverse process. Since a first sample image needs to go through multiple time steps, multiple training operations can be performed on a single first sample image. This allows the trained model to gradually adapt to the extraction of detailed features under different levels of noise, thereby improving the feature extraction capability of the encoding network and enhancing the ability to mine more image details.

[0079] It is worth noting that, during the forward pass, the noise categories added at different time steps can be the same or at least partially different, and this application does not impose any limitations on this. To improve model training efficiency, the noise categories added at different time steps are typically the same. Furthermore, the intensity of the noise added at different time steps can be the same or at least partially different.

[0080] S130 generates a tumor prognostic prediction model that includes a prognostic classification network and a pre-trained coding network.

[0081] Specifically, the pre-trained encoding network is concatenated with the prognostic classification network to obtain an initial tumor prognostic prediction model.

[0082] The role of the encoding network is to extract features from the image and obtain image features; the role of the prognostic classification network is to map the image features output by the encoding network to the predicted prognostic classification result space, that is, to convert the image features output by the encoding network into the predicted prognostic classification result.

[0083] S140, Based on the second sample image and the corresponding tumor prognosis label, the tumor prognosis prediction model is fine-tuned to obtain a trained tumor prognosis prediction model.

[0084] Specifically, the second sample image can be input into the tumor prognosis prediction model to obtain the prognosis prediction classification result. The prognosis prediction classification result and the tumor prognosis label corresponding to the second sample image are compared to obtain the difference data between the two. Then, the parameters of the tumor prognosis prediction model are fine-tuned based on the difference data, so that the model gradually has the ability to predict tumor prognosis, and finally a trained tumor prognosis prediction model is obtained.

[0085] In the process of fine-tuning the parameters of the tumor prognosis prediction model, one parameter can be adjusted at a time, or multiple parameters can be adjusted simultaneously.

[0086] In S140, this fine-tuning method can be used to optimize the tumor prognosis prediction model for the specific prognosis prediction task of the target tumor. While ensuring the generalization ability of the tumor prognosis prediction model, it improves the performance of the tumor prognosis prediction model in the task of predicting the prognosis of tumors.

[0087] As can be seen, since the first sample images required in the pre-training process do not require corresponding label data, the difficulty of obtaining a large number of first sample images is reduced. Pre-training the diffusion model based on unlabeled first sample images allows the encoding network in the diffusion model to gradually learn prior knowledge of image detail information, enabling it to extract more image details and thus improving the feature extraction capability of the encoding network. Furthermore, a tumor prognosis prediction model including the pre-trained encoding network and prognostic classification network is generated. Then, a second sample image including tumor prognosis labels is used to fine-tune the tumor prognosis prediction model, thereby transferring the encoding network with strong feature extraction capabilities to the tumor prognosis scenario, allowing the trained tumor prognosis prediction model to gradually possess tumor prognosis prediction functionality. Since the fine-tuning stage typically requires only a small number of samples, a small label collection cost can improve the accuracy of prediction results in the tumor prognosis prediction scenario, significantly improving the predictive performance of the tumor prognosis prediction model.

[0088] Based on the above technical solution, this application also provides an optional embodiment in which the pre-training process corresponding to S120 is refined.

[0089] See Figure 3 The pre-training steps for the diffusion model include:

[0090] S310, in the forward process of the diffusion model, for each time step, noise is added to the original sample image corresponding to the time step to obtain the mixed sample image corresponding to the time step, and the mixed sample image corresponding to the time step is used as the original sample image corresponding to the next time step; the original sample image corresponding to the first time step is the first sample image.

[0091] The number of time steps in the forward process can be defined as T, where T is an integer greater than 1. The specific value of T can be set as needed or determined through repeated experiments, and this application does not impose any restrictions on it. For example, T can be 1500.

[0092] Specifically, the original sample image corresponding to the first time step of the forward process is the first sample image. Noise is added to the first sample image to obtain the mixed sample image corresponding to the first time step. This mixed sample image is then used as the original sample image for the second time step, and noise is added to it to obtain the mixed sample image for the second time step. This process continues until the mixed sample image for the Tth time step is obtained. Therefore, after T time steps, T mixed sample images with added noise can be obtained.

[0093] The type of noise added can be Gaussian noise, or other types of noise, such as salt and pepper noise, shot noise, etc.

[0094] S320, during the reverse process of the diffusion model, for each time step, the mixed sample image corresponding to the time step is denoised, and the diffusion model is pre-trained based on the denoising result and the original sample image corresponding to the time step.

[0095] Understandably, the forward process requires T time steps, and the backward process also requires T reverse time steps. The T time steps of the forward process and the T time steps of the backward process are in one-to-one correspondence: the last time step of the forward process corresponds to the first time step of the backward process, the second-to-last time step of the forward process corresponds to the second time step of the backward process, and so on, with the first time step of the forward process corresponding to the last time step of the backward process. To avoid confusion, this application uses the order of the time steps in the forward process to number each time step.

[0096] For a given time step, the mixed sample image in the forward process will be used as the image to be denoised in the backward process, and denoising will be performed to obtain the denoised result. The original sample image in the forward process will be used as the label for pre-training the diffusion model in the backward process. By analyzing the difference between the denoised result and the label at the corresponding time step, the network parameters of the diffusion model will be adjusted so that the diffusion model can gradually acquire the ability to extract detailed features of the target organ under different noise environments.

[0097] For example, in the i-th time step of the reverse process, the mixed sample image corresponding to the corresponding time step in the forward process is denoised to obtain the denoised result corresponding to the i-th time step of the reverse process. Based on this denoised result and the original sample image corresponding to the corresponding time step in the forward process, the diffusion model is pre-trained to complete one training cycle. The value of i ranges from [1, T]. Since there are T time steps, performing one reverse process is equivalent to performing T model training cycles. Therefore, compared with the pre-training process of other models, the pre-training process of the diffusion model in this embodiment can significantly improve the training effect of the diffusion model, that is, improve the feature extraction capability of the encoding network in the diffusion model.

[0098] The pre-training of the diffusion model is achieved through the above steps S310~S320, which is the above implementation method of forward noise addition and backward restoration. This method can be applied to pre-training scenarios of diffusion models with different structures.

[0099] It is worth noting that the above process only explains the function of the diffusion model and should not be interpreted as a specific limitation on the network structure of the diffusion model.

[0100] Based on the technical solutions of the above embodiments, furthermore, since the diffusion model includes other network structures besides the encoding network, and different other network structures have different functions, the specific process of pre-training the diffusion model based on the denoising result and the original sample image corresponding to the time step in S320 may differ for different other network structures. In an optional embodiment, the diffusion model may further include a decoding network for image reconstruction. It can be seen that the encoding network extracts image features from the image, and the decoding network decodes the image features output by the encoding network, thereby reconstructing the image based on the image features to obtain the reconstructed image. That is to say, the encoding network and the decoding network are symmetrical, and the structure formed by the two can be called a UNet structure.

[0101] Based on the diffusion model, which further includes a decoding network for image reconstruction, see [link to relevant documentation]. Figure 4 Step S320 may specifically include:

[0102] S410, for each time step, feature extraction is performed on the mixed sample image corresponding to the time step based on the coding network to obtain the features to be reconstructed corresponding to the time step.

[0103] For example, for the i-th time step of the reverse process, the feature to be reconstructed refers to the image feature corresponding to the i-th time step of the reverse process obtained by using the coding network to extract features from the mixed sample image corresponding to the i-th time step of the reverse process. Because this image feature is used for image reconstruction, it is called the feature to be reconstructed.

[0104] Furthermore, in S410, to extract richer image features, a stepwise extraction method using features at multiple scales can be employed. Therefore, in an optional embodiment, the step in S410 of reconstructing the image based on the decoding network for the features to be reconstructed at the time step, to obtain the reconstructed image at the time step, may specifically include the following steps:

[0105] Based on the encoding network, the mixed sample image corresponding to the time step is subjected to concatenated encoding of multi-scale features to obtain the features to be reconstructed corresponding to the time step; wherein, in at least one feature encoding process, global attention enhancement is performed on the encoding process of the previous feature encoding result to obtain the current feature encoding result.

[0106] In other words, at each time step in the reverse process, the encoding network uses cascaded encoding of multi-scale features to ensure that the extracted features to be reconstructed include features of multiple scales, resulting in rich features and reducing the possibility of missing important image information. Moreover, in one or more feature encoding processes, global attention enhancement is applied to the encoding process of the previous feature encoding result, i.e., a global attention mechanism is introduced, which better captures global features and achieves the fusion of local and global features.

[0107] S420, Based on the decoding network, image reconstruction is performed on the features to be reconstructed corresponding to the time step to obtain the reconstructed image corresponding to the time step.

[0108] For example, for the i-th time step of the reverse process, image reconstruction is performed based on the image features corresponding to the i-th time step of the reverse process. The resulting image is called the reconstructed image corresponding to the i-th time step of the reverse process. The reconstructed image corresponding to the i-th time step of the reverse process can be understood as the denoising result corresponding to the i-th time step of the reverse process.

[0109] Feature extraction is equivalent to extracting key detail features related to the target organ and filtering out background features and noise unrelated to the target organ; image reconstruction is equivalent to reconstructing the image based on key detail features. Since key detail features can include edge features and texture details, image reconstruction based on these features ensures that the reconstructed image no longer contains noise information added during the forward pass, nor does it contain other background noise unrelated to the target organ, thus achieving the purpose of denoising.

[0110] Furthermore, in S420, to better restore the image, i.e., to achieve better image reconstruction results, a stepwise decoding method using multiple scale features can be adopted. Therefore, in an optional embodiment, the step in S420 of reconstructing the image based on the decoding network for the features to be reconstructed at the time step to obtain the reconstructed image at the time step may specifically include the following steps:

[0111] Based on the decoding network, the features to be reconstructed at the time step are cascaded and decoded using multi-scale features to obtain the reconstructed image at the time step; wherein, in at least one feature decoding process, global attention enhancement is performed on the decoding process of the previous feature decoding result to obtain the current feature decoding result.

[0112] In other words, at each time step in the reverse process, the decoding network reconstructs the image through cascaded decoding of multi-scale features, i.e., through a step-by-step restoration. Each restoration step uses different feature scales to minimize the omission of key information in the reconstructed image and ensure high-quality reconstruction. Furthermore, during one or more feature decoding processes, global attention enhancement is applied to the decoding process of the previous feature decoding result—that is, a global attention mechanism is introduced—which better captures global features and achieves the fusion of local and global features.

[0113] S430, the diffusion model is pre-trained based on the reconstructed image corresponding to the time step and the original sample image corresponding to the time step.

[0114] For example, the original sample image corresponding to the time step can be used as a label to determine the difference between the reconstructed image corresponding to the time step and the original sample image corresponding to the time step. Then, the parameters of the diffusion model can be adjusted according to the difference to continuously reduce the difference and thereby continuously improve the accuracy of feature extraction by the coding network in the diffusion model.

[0115] For example, by using an encoding network to extract features from the mixed sample image corresponding to the i-th time step of the reverse process, the features to be reconstructed at the i-th time step of the reverse process can be obtained. Next, a decoding network is used to decode the features to be reconstructed at the i-th time step of the reverse process, thereby achieving image reconstruction and obtaining the reconstructed image corresponding to the i-th time step of the reverse process. This reconstructed image is the denoising result at the i-th time step of the reverse process. Furthermore, the diffusion model is pre-trained using the reconstructed image and the original sample image at the i-th time step of the reverse process.

[0116] Specifically, for the i-th time step of the reverse process, the mixed sample image corresponding to the i-th time step of the reverse process is the mixed sample image corresponding to the corresponding time step in the forward process, and the original sample image corresponding to the i-th time step of the reverse process is the original sample image corresponding to the corresponding time step in the forward process.

[0117] As can be seen, each time step in the denoising process during the reverse process requires two steps: the first step is to encode the features to be reconstructed using an encoding network; the second step is to reconstruct the image using the features to be reconstructed using a decoding network. Based on feature extraction and feature reconstruction, pre-training of the diffusion model can be achieved without the need for additional manual labeling of the first sample image, thereby improving the efficiency of model training and reducing the workload of manual annotation.

[0118] See Figure 5 This diagram illustrates the forward and backward processes of the diffusion model in this embodiment. In the first time step, image p1 is used as the original sample image for the first time step of the forward process. Noise is added to image p1 to obtain the mixed sample image p2 for the first time step. Image p2 is then used as the original sample image for the second time step, and noise is added to image p2 to obtain the mixed sample image for the second time step. This process continues until the (T-1)th time step, where noise is added to obtain the mixed sample image p2. T At time step T, noise is added to obtain the mixed sample image p. T+1 After T time steps, the noise addition is complete. Then, a reverse process is performed, where at each time step, denoising is done through an encoding network and a decoding network. For example, in the first time step of the reverse process, the mixed sample image p is... T+1 The denoising result is obtained after passing through the encoding and decoding networks, with image p T Assign labels to determine the denoising results and image p. T The differences are used to adjust the parameters of the encoding and decoding networks, thereby achieving model pre-training. Figure 5In the middle, X0~X T This refers to a network structure responsible for inputting and outputting images, and adjusting the parameters of the encoding and decoding networks based on differences during the forward and backward processes.

[0119] As can be seen, during training, the diffusion model models the distribution of training data through a forward process of gradually introducing noise and a reverse process of gradually denoising. In application, it can generate high-quality data through gradual denoising. The forward process in diffusion model training can be summarized as gradually adding Gaussian noise or other noise to the sample images to generate a latent random variable.

[0120] The following example, using lung CT scans as sample images, illustrates the pre-training of the diffusion model:

[0121] First, the diffusion model gradually converts the image into pure Gaussian noise using a predefined noise scheduling scheme. This process can be represented as:

[0122]

[0123] in, This is a CT image of the lungs. Indicates a normal distribution. This represents the noise control term at time step t. It is the identity matrix. It is a mean Right now variance is The normal distribution is given by t, which represents the time step number and has a range of values ​​[1, T].

[0124] Then, in the reverse process, the diffusion model learns how to denoise, thus gradually restoring noisy samples to clear images. The goal of this process is to parameterize the model. Learning an approximate inverse process This means gradually recovering the original data from the noise. Since the inverse distribution of the inverse process also conforms to a Gaussian distribution, for sufficiently small... The original problem can be further transformed into parameterizing the mean and variance of a Gaussian distribution. Therefore, the reverse process can also use a Gaussian distribution to model the denoising step.

[0125]

[0126] By applying the inverse formula to all time steps , can be Model the distribution of the original data:

[0127]

[0128] By adjusting the time steps, the model can learn to predict the Gaussian distribution parameters at each time step, thereby gradually recovering the distribution of the training data (i.e., the original sample images at the corresponding time step) from the noisy data (i.e., the mixed sample images).

[0129] Throughout the training phase, the diffusion model uses variational inference to optimize the model and minimize reconstruction error. By modeling the noise addition during the entire forward process and the denoising process during the reverse process, the diffusion model can effectively learn the data distribution. When the time step T is sufficiently large, typically set to 1000, it can effectively learn the data distribution. It approximates an isotropic Gaussian distribution; therefore, in the generation phase, traditional diffusion models deviate from the standard normal distribution. Sampling is performed and a reverse denoising process is applied step by step to generate a new image.

[0130] To implement the above encoding process, the encoding network can adopt various structural forms. Here, an optional structure of the encoding network is provided: the encoding network includes at least two downsampling modules connected in sequence, at least one of the at least two downsampling modules includes the global attention unit, wherein the global attention unit is used to implement the above-mentioned global attention mechanism, and the at least two downsampling modules are used to implement the cascaded encoding of multi-scale features.

[0131] To implement the above decoding process of the decoding network, the decoding network can be implemented using various structures. Here, one optional structure of the decoding network is provided. The decoding network includes at least two upsampling modules connected in sequence, and at least one of the at least two upsampling modules includes the global attention unit. The global attention unit is used to implement the above-mentioned global attention mechanism, and the at least two upsampling modules are used to implement cascaded decoding of multi-scale features.

[0132] Furthermore, since the encoding and decoding processes are two opposite processes, the encoding process extracts features from the mixed sample images, while the decoding process reconstructs the image based on the extracted features. In order to ensure the consistency between the new image and the original image as much as possible, the at least two downsampling modules and the at least one upsampling module correspond one-to-one.

[0133] In practical scenarios, to ensure effective gradient optimization during training, a residual structure can be configured for each module. That is, each sampling module can have a residual structure, thereby achieving effective gradient optimization and improving model training efficiency and stability.

[0134] Figure 6This diagram illustrates the UNet structure in the diffusion model, which includes an encoder network and a decoder network. In this structure, the encoder network has five downsampling modules, and the decoder network has five upsampling modules. A global attention unit is located in the fourth downsampling module, and the corresponding second upsampling module also contains a global attention unit. This effectively addresses the inherent limitations of convolutional neural networks in capturing global contextual information. Understandably, while convolutional neural networks offer good efficiency, speed, and generalization ability in visual tasks, their inductive bias makes it difficult to comprehensively capture global features. Introducing a global attention unit allows the diffusion model to better capture global features, achieving the fusion of local and global features and making the model more robust. Due to the introduction of the attention mechanism, therefore... Figure 6 The UNet structure in this model can also be called a Transformer-based UNet structure. Furthermore, each sampling module is equipped with a residual structure, which ensures effective gradient optimization during training, improving model training efficiency and stability. Additionally, each sampling module receives not only image features as input but also encoded time-step information, enabling the diffusion model to better estimate noise at each step of the diffusion process, improving the quality of extracted features and enhancing the performance of the final tumor prognosis prediction model in prognostic tasks. Moreover, a deep feature extraction module can be added between the last downsampling module and the first upsampling module for further feature extraction.

[0135] This paper combines the advantages of Convolutional Neural Networks (CNNs) and Deep Learning Models (Transformers) in its encoding network. The efficiency, speed, and generalization ability of CNNs are combined with the powerful global feature capture capabilities of Transformers. By equipping the sampling module with a global attention unit, the problem of CNNs' difficulty in capturing global contextual information is solved, enabling the diffusion model to analyze image features more comprehensively.

[0136] Different sampling modules have different feature scales. For example, the first downsampling module outputs image features at a scale of 256×256×64, the second at 128×128×128, the third at 64×64×256, the fourth at 32×32×512, and the fifth at 16×16×1024. The depth feature extraction module includes two convolutional layers, each with a scale of 8×8×1024. The first upsampling module outputs image features at a scale of 16×16×512, the second upsampling module outputs image features at a scale of 32×32×256, the third upsampling module outputs image features at a scale of 64×64×128, the fourth upsampling module outputs image features at a scale of 128×128×64, and the fifth upsampling module outputs image features at a scale of 256×256×64. Image reconstruction is performed based on image features at these scales.

[0137] In real-world scenarios, see Figure 6 The input features of each upsampling module other than the first upsampling module include the image features output by the previous upsampling module and the image features output by the corresponding downsampling module. This can minimize the omission of key information during the decoding process.

[0138] After training, the encoding network is concatenated with the prognostic classification network to obtain the initial tumor prognostic prediction model. See [example example] for details. Figure 7 ,Will Figure 6 The five downsampling modules trained in the middle stage and a prognostic classification network are concatenated to obtain the initial tumor prognostic prediction model. The prognostic classification network can be implemented using various structures; see [link to relevant documentation]. Figure 7 One possible structure is as follows: the prognostic classification network may include a two-dimensional convolutional layer (Conv2D), a ReLU activation function, and a linear function connected in sequence. The two-dimensional convolutional layer (Conv2D) is used to further extract features from the image features output by the encoding network. The ReLU activation function is used to perform a non-linear mapping on the features output by the two-dimensional convolutional layer, and the linear function is used to perform a linear mapping on the non-linear mapping result output by the activation function, thereby obtaining the predicted prognostic classification result. Of course, the activation function is not limited to the ReLU activation function; the linear function is also acceptable.

[0139] To enhance the diffusion model's ability to capture fine-grained features, this application employs a pre-training strategy based on the diffusion model. This strategy differs from traditional pre-training methods by leveraging the advantages of the diffusion model to accurately capture subtle features in images through a gradual process of adding and removing noise. In the forward pass, Gaussian noise or other noise is gradually added to the image according to a designed noise scheduling scheme, simulating the random variation of data and enabling the diffusion model to learn the feature variation patterns of the image under different noise levels. In the reverse pass, the diffusion model learns an approximate inverse process, gradually recovering a clear image from noisy samples. During this process, the diffusion model can deeply mine the detailed information of the image, achieving accurate capture of fine-grained features. In other words, this application's pre-training strategy based on the diffusion model differs from traditional image self-supervised or contrastive learning methods. By reconstructing the image's deep learning image feature encoding and decoding process, it can generate realistic data, allowing the diffusion model to fully grasp prior knowledge of the image's detailed information. As can be seen, the embodiments of this application break through the bottleneck of traditional models in capturing detailed features, significantly improve the model's ability to capture detailed features such as subtle lesions and changes in tissue texture, thereby greatly improving the accuracy and reliability of tumor prognosis prediction models.

[0140] The pre-training strategy based on the diffusion model, using a diffusion probabilistic model for pre-training, offers unique advantages compared to traditional variational autoencoders (VACs) and generative adversarial models (GANs). During pre-training, the diffusion probabilistic model gradually approximates real data through a progressive denoising process, enabling more accurate extraction of image features and better preservation of details and spatial information during image reconstruction. Furthermore, the diffusion probabilistic model eliminates the need for adversarial training, avoiding the instability and mode collapse issues caused by generator-discriminator imbalance in traditional adversarial training. This allows for stable and reliable generation of high-quality reconstructed images, providing a strong guarantee for the model to capture fine-grained features.

[0141] To verify the effectiveness of the tumor prognosis prediction model trained according to the embodiments of this application, a series of experiments can be conducted. For example, during the data collection phase, 485,885 lung CT scans from multiple medical centers were collected and processed. After screening using image quality control and natural language processing techniques, 105,184 high-quality data points were obtained. These data were then divided into training and test sets according to a certain ratio.

[0142] During model training, the diffusion model was trained multiple times, and metrics such as reconstruction error and convergence speed were monitored. Experimental results show that the pre-training strategy based on the diffusion model can converge quickly and has low reconstruction error, demonstrating the effectiveness of the pre-training strategy.

[0143] For the Transformer-based UNet structure, tests were conducted on different datasets to evaluate the diffusion model's ability to capture local and global features and comprehensively analyze image features. Comparative experiments with traditional convolutional neural network models and other Transformer-based networks show that the Transformer-based UNet structure in this embodiment outperforms the model in all evaluation metrics, and the encoding network in the UNet structure can more accurately extract key features from the image.

[0144] In the prognostic prediction task, sample images from the test set are input into the tumor prognostic prediction model. The predicted results are compared and analyzed with the actual tumor prognostic labels. The predictive performance of the tumor prognostic prediction model is evaluated using metrics such as accuracy and recall. Experimental results show that the tumor prognostic prediction model in this embodiment has high accuracy and recall in predicting tumor prognostic outcomes, and can provide valuable decision support for clinicians.

[0145] As an example, the method provided in this application can be used to train a model for predicting the prognostic effects of neoadjuvant therapy for non-small cell carcinoma. This model can be widely applied in clinical medical decision support systems, assisting physicians in conducting more accurate prognostic assessments of non-small cell carcinoma patients, thereby providing a scientific basis for developing personalized neoadjuvant therapy plans. In the field of medical research, it helps to delve deeper into the pathogenesis and imaging characteristics of treatment response in non-small cell carcinoma, promoting the development of related medical research. Furthermore, it can be applied to medical big data analysis platforms to mine potential information in large-scale medical imaging data, improving the efficiency of medical resource utilization and the quality of medical services.

[0146] In summary, the tumor prognosis prediction model training process in this embodiment is divided into two stages: one stage is a diffusion-based pre-training stage, which yields an encoding network capable of feature extraction; the other stage is fine-tuning the tumor prognosis prediction model, which includes both the encoding network and the prognosis classification network. In the pre-training stage, large-scale unlabeled training data and advanced diffusion strategies are used to enable the diffusion model to learn general image features and patterns. The fine-tuning stage, on the other hand, optimizes the model using a small amount of task instance data for specific prognosis prediction tasks, improving the performance of the tumor prognosis prediction model on these tasks. Therefore, this embodiment achieves a novel model training mode through the synergy of the pre-training and fine-tuning stages. This training mode fully utilizes the advantages of large-scale data while allowing for flexible adjustments based on specific tasks, ensuring the generalization ability of the tumor prognosis prediction model while improving its performance in practical applications.

[0147] The above embodiments have provided a detailed description of the training process of the tumor prognosis prediction model. The following will describe in detail the usage process of the tumor prognosis prediction model. It is worth noting that the device used to train the tumor prognosis prediction model can be the same device or different devices, and this application does not impose any limitations on this.

[0148] In some embodiments, see Figure 8 The methods for using the tumor prognosis prediction model shown include:

[0149] S810: Acquire the target user's image of the target organ.

[0150] The target users are those who need to assess the prognosis of tumors.

[0151] The target organ is the same organ targeted by the tumor prognosis prediction model.

[0152] The image to be tested refers to an image of the target organ of the target user, such as a lung CT scan.

[0153] S820, the image to be tested is input into the trained tumor prognosis prediction model to obtain the tumor prognosis prediction result for the target user.

[0154] The trained tumor prognosis prediction model is determined based on the tumor prognosis prediction model training method provided in any of the above embodiments.

[0155] As can be seen, by inputting the image to be tested into the trained tumor prognosis prediction model, the tumor prognosis prediction result can be obtained.

[0156] The tumor prognosis prediction model can be integrated into the computing device that executes the method for using the tumor prognosis prediction model. When prognosis prediction is required, the tumor prognosis prediction model can be used directly. Of course, to reduce the occupation of the storage and computing resources of the aforementioned computing device, the tumor prognosis prediction model can also be stored in the cloud. When prognosis prediction is required, the interface of the tumor prognosis prediction model can be called directly.

[0157] This application embodiment uses the tumor prognosis prediction model trained in the aforementioned embodiments. Since the tumor prognosis prediction model is pre-trained based on a diffusion model, and since the first sample images required during pre-training do not require corresponding label data, the difficulty of obtaining a large number of first sample images is reduced. Pre-training the diffusion model based on unlabeled first sample images allows the encoding network in the diffusion model to gradually learn prior knowledge of image detail information, enabling it to extract more image details and thus improving the feature extraction capability of the encoding network. Furthermore, a tumor prognosis prediction model including a pre-trained encoding network and a prognosis classification network is generated. Then, a second sample image including tumor prognosis labels is used to fine-tune the tumor prognosis prediction model, thereby transferring the encoding network with strong feature extraction capabilities to the tumor prognosis scenario, allowing the trained tumor prognosis prediction model to gradually possess tumor prognosis prediction functionality. Since the fine-tuning stage typically requires only a small number of samples, a small label collection cost can improve the accuracy of prediction results in the tumor prognosis prediction scenario, significantly improving the predictive performance of the tumor prognosis prediction model. It is evident that using tumor prognosis prediction models to predict prognostic outcomes can improve prediction accuracy and has the advantages of being reusable, portable, and universal.

[0158] Based on the same inventive concept, this application also provides a tumor prognosis prediction model training device for implementing the tumor prognosis prediction model training method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the tumor prognosis prediction model training device provided below can be found in the limitations of the tumor prognosis prediction model training method described above, and will not be repeated here.

[0159] In one exemplary embodiment, a tumor prognosis prediction model training device is provided, see [link to relevant documentation]. Figure 9 The device includes: a first acquisition module 910, a pre-training module 920, a model generation module 930, and a model fine-tuning module 940; wherein:

[0160] The first acquisition module 910 is used to acquire first sample images and second sample images of target organs from different sample users, as well as tumor prognosis labels corresponding to the second sample images.

[0161] The pre-training module 920 is used to pre-train the diffusion model based on the first sample image, so that the diffusion model learns the feature extraction capability of the target organ; the diffusion model includes an encoding network for feature extraction;

[0162] The model generation module 930 is used to generate a tumor prognostic prediction model that includes a prognostic classification network and a pre-trained coding network.

[0163] The model fine-tuning module 940 is used to fine-tune the tumor prognosis prediction model based on the second sample image and the corresponding tumor prognosis label to obtain a trained tumor prognosis prediction model.

[0164] In one embodiment, the pre-training module includes:

[0165] The forward unit is used to add noise to the original sample image corresponding to the time step for each time step in the forward process of the diffusion model, to obtain a mixed sample image corresponding to the time step, and use the mixed sample image corresponding to the time step as the original sample image corresponding to the next time step; the original sample image corresponding to the first time step is the first sample image.

[0166] The reverse unit is used to perform denoising processing on the mixed sample image corresponding to each time step during the reverse process of the diffusion model, and to pre-train the diffusion model based on the denoising result and the original sample image corresponding to the time step.

[0167] In one embodiment, the diffusion model further includes a decoding network for image reconstruction; the inverse unit includes:

[0168] The encoding subunit is used to extract features from the mixed sample image corresponding to the time step based on the encoding network for each time step, so as to obtain the features to be reconstructed for the time step.

[0169] The decoding subunit is used to perform image reconstruction on the features to be reconstructed corresponding to the time step based on the decoding network, so as to obtain the reconstructed image corresponding to the time step; the reconstructed image corresponding to the time step is the denoising result corresponding to the time step.

[0170] The pre-training subunit is used to pre-train the diffusion model based on the reconstructed image corresponding to the time step and the original sample image corresponding to the time step.

[0171] In one embodiment, the encoding subunit is specifically used to: perform concatenated encoding of multi-scale features on the mixed sample image corresponding to the time step based on the encoding network to obtain the features to be reconstructed corresponding to the time step; wherein, in at least one feature encoding process, global attention enhancement is performed on the encoding process of the previous feature encoding result to obtain the current feature encoding result.

[0172] In one embodiment, the decoding subunit is specifically used to: perform multi-scale feature concatenation decoding on the features to be reconstructed corresponding to the time step based on the decoding network to obtain the reconstructed image corresponding to the time step; wherein, in at least one feature decoding process, global attention enhancement is performed on the decoding process of the previous feature decoding result to obtain the current feature decoding result.

[0173] Fourthly, based on the same inventive concept, this application also provides a tumor prognosis prediction model application device for implementing the tumor prognosis prediction model application method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more tumor prognosis prediction model application device embodiments provided below can be found in the limitations of the tumor prognosis prediction model application method described above, and will not be repeated here.

[0174] In one exemplary embodiment, a device for using a tumor prognosis prediction model is provided, see [link to relevant documentation]. Figure 10 The device includes: a second acquisition module 1010 and a model prediction module 1020, wherein:

[0175] The second acquisition module 1010 is used to acquire the test image of the target organ by the target user;

[0176] The model prediction module 1020 is used to input the image to be tested into a trained tumor prognosis prediction model to obtain the tumor prognosis prediction result of the target user; wherein, the trained tumor prognosis prediction model is determined based on the tumor prognosis prediction model training method provided in the first aspect.

[0177] The modules in the aforementioned tumor prognosis prediction model training device and / or tumor prognosis prediction model user device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0178] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for training and / or using a tumor prognosis prediction model. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0179] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0180] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0181] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps described in the method embodiments.

[0182] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps described in the method embodiments.

[0183] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0184] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0185] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0186] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for training a tumor prognosis prediction model, characterized in that, The method includes: Obtain first and second sample images of the target organ from different sample users, as well as the tumor prognosis label corresponding to the second sample image; The diffusion model is pre-trained based on the first sample image to enable the diffusion model to learn the feature extraction capability of the target organ; the diffusion model includes an encoding network for feature extraction. Generate a tumor prognostic prediction model that includes a prognostic classification network and a pre-trained coding network; Based on the second sample image and the corresponding tumor prognosis label, the tumor prognosis prediction model is fine-tuned to obtain a trained tumor prognosis prediction model.

2. The method according to claim 1, characterized in that, The step of pre-training the diffusion model based on the first sample image includes: In the forward process of the diffusion model, for each time step, noise is added to the original sample image corresponding to the time step to obtain the mixed sample image corresponding to the time step, and the mixed sample image corresponding to the time step is used as the original sample image corresponding to the next time step; the original sample image corresponding to the first time step is the first sample image. In the reverse process of the diffusion model, for each time step, the mixed sample image corresponding to the time step is denoised, and the diffusion model is pre-trained based on the denoising result and the original sample image corresponding to the time step.

3. The method according to claim 2, characterized in that, The diffusion model further includes a decoding network for image reconstruction; the step of denoising the mixed sample image corresponding to each time step, and pre-training the diffusion model based on the denoising result and the original sample image corresponding to the time step, includes: For each time step, feature extraction is performed on the mixed sample image corresponding to the time step based on the encoding network to obtain the features to be reconstructed for the time step; Based on the decoding network, image reconstruction is performed on the features to be reconstructed at the time step to obtain the reconstructed image at the time step. The diffusion model is pre-trained based on the reconstructed image corresponding to the time step and the original sample image corresponding to the time step.

4. The method according to claim 3, characterized in that, The step of reconstructing the image based on the decoding network for the features to be reconstructed at the time step to obtain the reconstructed image at the time step includes: Based on the coding network, the mixed sample image corresponding to the time step is subjected to concatenated coding of multi-scale features to obtain the features to be reconstructed corresponding to the time step; In at least one feature encoding process, global attention enhancement is applied to the encoding process of the previous feature encoding result to obtain the current feature encoding result.

5. The method according to claim 3, characterized in that, The step of reconstructing the image based on the decoding network for the features to be reconstructed at the time step to obtain the reconstructed image at the time step includes: Based on the decoding network, the features to be reconstructed at the time step are subjected to cascaded decoding of multi-scale features to obtain the reconstructed image corresponding to the time step. In at least one feature decoding process, global attention enhancement is applied to the decoding process of the previous feature decoding result to obtain the current feature decoding result.

6. A method for using a tumor prognosis prediction model, characterized in that, The method includes: Acquire test images of the target organ from the target user; The image to be tested is input into a trained tumor prognosis prediction model to obtain the tumor prognosis prediction result for the target user; wherein the trained tumor prognosis prediction model is determined based on the method described in any one of claims 1-5.

7. A tumor prognosis prediction model training device, characterized in that, The device includes: The first acquisition module is used to acquire first sample images and second sample images of target organs from different sample users, as well as tumor prognosis labels corresponding to the second sample images. A pre-training module is used to pre-train the diffusion model based on the first sample image, so that the diffusion model learns the feature extraction capability of the target organ; the diffusion model includes an encoding network for feature extraction; The model generation module is used to generate tumor prognostic prediction models that include a prognostic classification network and a pre-trained coding network. The model fine-tuning module is used to fine-tune the tumor prognosis prediction model based on the second sample image and the corresponding tumor prognosis label to obtain a trained tumor prognosis prediction model.

8. A device for using a tumor prognosis prediction model, characterized in that, The device includes: The second acquisition module is used to acquire the test image of the target organ from the target user; The model prediction module is used to input the image to be tested into a trained tumor prognosis prediction model to obtain the tumor prognosis prediction result of the target user; wherein the trained tumor prognosis prediction model is determined based on the device described in claim 9.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images