Training method of image processing model, and image processing method and device

Abstract

The invention provides a training method of an image processing model and an image processing method and device, and relates to the technical field of image processing, and the method comprises the steps: obtaining a first sample image and a first preset training label corresponding to the first sample image, and obtaining a second sample image and a second preset training label corresponding to the second sample image, taking a first model parameter of the trained palliative curative effect prediction model as a second model parameter of the image processing model; and iteratively updating a second model parameter of the image processing model according to the first sample image, the second sample image and a second preset training label until a first preset number of iterations is reached, and completing prediction training of the image processing model for the neoadjuvant chemotherapy curative effect. According to the process, the robustness of the image processing model can be improved on the basis of less neoadjuvant chemotherapy training data, and the prediction precision of the neoadjuvant chemotherapy curative effect can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and particularly to a method for training an image processing model, an image processing method, and an apparatus. Background Art

[0002] In the field of modern medicine, a medical model can be used to predict disease progression and evaluate treatment effects based on medical data.

[0003] However, the treatment cycles of many diseases are long, and it is difficult to collect relevant medical data, resulting in a shortage of medical data. Training a medical model using limited medical data not only limits the generalization ability of the model but also reduces the accuracy and reliability of the prediction results. Summary of the Invention

[0004] This application provides a method for training an image processing model, an image processing method, and an apparatus. Based on an incremental learning scheme based on similarity, on the basis of less neoadjuvant chemotherapy training data, after the palliative efficacy prediction model learns the palliative task, it can provide prior knowledge for learning neoadjuvant chemotherapy data, so that the features extracted by the image processing model have a higher signal-to-noise ratio, less redundancy and noise, further improving the robustness of the image processing model and the prediction accuracy of neoadjuvant chemotherapy efficacy.

[0005] In a first aspect, a method for training an image processing model is provided. The method includes: Obtain a first sample image and a first preset training label corresponding to the first sample image, and a second sample image and a second preset training label corresponding to the second sample image. Among them, the first sample image is a CT image before palliative treatment, the first preset training label is palliative efficacy data corresponding to the first sample image, the second sample image is a CT image before neoadjuvant chemotherapy, and the second preset training label is neoadjuvant chemotherapy efficacy data corresponding to the second sample image; Use the first model parameters of the trained palliative efficacy prediction model as the second model parameters of the image processing model, where the palliative efficacy prediction model is trained by the first sample image and the first preset training label; According to the first sample image, the second sample image, and the second preset training label, iteratively update the second model parameters of the image processing model until the first preset number of iterations is reached, and complete the prediction training of the image processing model for neoadjuvant chemotherapy efficacy.

[0006] In a second aspect, an image processing method is provided, including: Obtain a target CT image of a target object before neoadjuvant chemotherapy; Input the target CT image into the trained image processing model of the first aspect to obtain the efficacy prediction result of the target object after neoadjuvant chemotherapy. The image processing model is used to extract the target neoadjuvant chemotherapy features of the target CT image, and predict the efficacy data of the target CT image based on the target neoadjuvant chemotherapy features to obtain the efficacy prediction result.

[0007] In a third aspect, a training device for an image processing model is provided. The device includes: A sample image acquisition module, configured to acquire a first sample image and a first preset training label corresponding to the first sample image, and a second sample image and a second preset training label corresponding to the second sample image. The first sample image is a CT image before palliative treatment, the first preset training label is the palliative treatment efficacy data corresponding to the first sample image, the second sample image is a CT image before neoadjuvant chemotherapy, and the second preset training label is the neoadjuvant chemotherapy efficacy data corresponding to the second sample image; A parameter setting module, configured to use the first model parameters of the trained palliative treatment efficacy prediction model as the second model parameters of the image processing model, where the palliative treatment efficacy prediction model is obtained by training with the first sample image and the first preset training label; A training module, configured to iteratively update the second model parameters of the image processing model according to the first sample image, the second sample image, and the second preset training label until the first preset number of iterations is reached, and complete the prediction training of the image processing model for neoadjuvant chemotherapy efficacy.

[0008] In a fourth aspect, an image processing device is provided, including: An image acquisition module, configured to acquire a target CT image of a target object before neoadjuvant chemotherapy; An efficacy prediction module, configured to input the target CT image into the trained image processing model of the first aspect to obtain the efficacy prediction result of the target object after neoadjuvant chemotherapy. The image processing model is used to extract the target neoadjuvant chemotherapy features of the target CT image, and predict the efficacy data of the target CT image based on the target neoadjuvant chemotherapy features to obtain the efficacy prediction result.

[0009] In a fifth aspect, an electronic device is provided, including: a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method in the first aspect or the second aspect or its various implementation manners.

[0010] In a sixth aspect, a computer-readable storage medium is provided, used to store a computer program, and the computer program enables a computer to execute the method in the first aspect or the second aspect or its various implementation manners.

[0011] Through the technical solution provided by this application, the first sample image and the first preset training label corresponding to the first sample image, as well as the second sample image and the second preset training label corresponding to the second sample image can be obtained. Among them, the first sample image is a CT image before palliative treatment, the first preset training label is the palliative treatment efficacy data corresponding to the first sample image, the second sample image is a CT image before neoadjuvant chemotherapy, and the second preset training label is the neoadjuvant chemotherapy efficacy data corresponding to the second sample image. The first model parameters of the trained palliative treatment efficacy prediction model are used as the second model parameters of the image processing model, where the palliative treatment efficacy prediction model is trained by the first sample image and the first preset training label. According to the first sample image, the second sample image, and the second preset training label, the second model parameters of the image processing model are iteratively updated until the first preset number of iterations is reached, and the prediction training of the image processing model for the neoadjuvant chemotherapy efficacy is completed. This process sets the second model parameters of the image processing model based on the first model parameters of the trained palliative treatment efficacy prediction model. Based on the incremental learning scheme of similarity, on the basis of less neoadjuvant chemotherapy training data, after the palliative treatment efficacy prediction model learns the palliative task, it can provide prior knowledge for learning neoadjuvant chemotherapy data, so that the features extracted by the image processing model have a higher signal-to-noise ratio, less redundancy and noise, further improve the robustness of the image processing model, and improve the prediction accuracy of the neoadjuvant chemotherapy efficacy.

[0012] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit this application. Other features and advantages of this application will be described in detail in the subsequent specific implementation section. Brief Description of the Drawings

[0013] In order to more clearly illustrate the technical solutions in the embodiments of this application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0014] Figure 1 It is an application scenario diagram provided for the embodiments of this application; Figure 2 It is a schematic flowchart of a training method for an image processing model provided for the embodiments of this application; Figure 3 It is a schematic flowchart of the prediction training for the neoadjuvant chemotherapy efficacy of an image processing model provided for the embodiments of this application; Figure 4 It is a schematic principle flowchart of the training of an image processing model provided for the embodiments of this application; Figure 5 A flowchart of an image processing method provided by an embodiment of the present application; Figure 6 A structural schematic diagram of a training device for an image processing model provided by an embodiment of the present application; Figure 7 A structural schematic diagram of an image processing device provided by an embodiment of the present application; Figure 8 A structural schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0015] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0016] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0017] In the field of modern medicine, medical models can be used to predict disease progression and evaluate treatment effects based on medical data. However, the treatment cycles of many diseases are long, and it is difficult to collect relevant medical data, resulting in a shortage of medical data. Training medical models with limited medical data not only limits the generalization ability of the models, but also reduces the accuracy and reliability of the prediction results.

[0018] Approximately 400,000 new gastric cancer cases are added in China every year, accounting for about 40% of the global new gastric cancer cases. The characteristic of gastric cancer is that it is often in the advanced stage when discovered, with a poor prognosis and a relatively high mortality rate. Among the current treatment methods, neoadjuvant chemotherapy is considered to be one of the extremely effective methods for gastric cancer. Therefore, accurately predicting the patient's response to neoadjuvant chemotherapy is crucial for clinicians to formulate more accurate and personalized treatment plans.

[0019] Currently, neoadjuvant chemotherapy can be used to treat gastric cancer. However, the treatment cycle of neoadjuvant chemotherapy is relatively long, and the applicable patient population is relatively limited, resulting in fewer prognostic data for patients undergoing neoadjuvant chemotherapy. The existing evaluation models for treatment efficacy are trained for efficacy prediction based on the prognostic data of a relatively small number of patients undergoing neoadjuvant chemotherapy, suffering from problems such as low accuracy and poor generalization in the efficacy prediction of the models, thus limiting the accuracy of the efficacy evaluation of neoadjuvant chemotherapy.

[0020] To solve the above technical problems, the inventive concept of this application is as follows: based on the first model parameters of the palliative efficacy prediction model after training, set the second model parameters of the image processing model, and perform prediction training on the efficacy of neoadjuvant chemotherapy for the image processing model based on the similarity between the palliative treatment data and the neoadjuvant chemotherapy data, and the neoadjuvant chemotherapy data. Here, since both palliative treatment and neoadjuvant chemotherapy are treatment methods for patients with relatively advanced clinical stages of gastric cancer, the prognostic conditions of patients based on palliative treatment and neoadjuvant chemotherapy both have certain reference values. Therefore, based on the similarity between the efficacy data of palliative treatment and the efficacy data of neoadjuvant chemotherapy, when the image processing model performs incremental learning, it can better focus on the common features of the palliative treatment data and the neoadjuvant chemotherapy data, thereby improving the robustness of the image processing model and the accuracy of the efficacy prediction of neoadjuvant chemotherapy.

[0021] It should be noted that the training method and the image processing method of the image processing model proposed in this application can be used for the clinical auxiliary diagnosis of gastric cancer, and can also be used in the research of pathology, imaging, and evaluation of treatment efficacy of gastric cancer. It can also be combined with the image data of other lesions and used in the research of other diseases in the fields of medical imaging, clinical auxiliary diagnosis, and prediction of treatment efficacy, not limited to the research of gastric cancer.

[0022] It should be understood that the technical solution of this application can be applied to the following scenarios, but is not limited to: In some implementable ways, Figure 1 For an application scenario diagram provided by an embodiment of this application, as Figure 1 shown, this application scenario may include an electronic device 110 and a network device 120. The electronic device 110 can establish a connection with the network device 120 through a wired network or a wireless network.

[0023] Exemplarily, the electronic device 110 may be a desktop computer, a laptop computer, a tablet computer, etc., but is not limited thereto. The network device 120 may be a terminal device or a server, but is not limited thereto. In an embodiment of the present application, the electronic device 110 may send a request message to the network device 120, and the request message may be used to request to obtain the efficacy prediction result of the target object after neoadjuvant chemotherapy. Further, the electronic device 110 may receive a response message sent by the network device 120, and the response message includes the efficacy prediction result of the target object after neoadjuvant chemotherapy.

[0024] In addition, Figure 1 Exemplarily, one electronic device 110 and one network device 120 are given. In fact, other numbers of electronic devices and network devices may be included, and the present application does not limit this.

[0025] In some other implementable manners, the technical solution of the present application may also be executed by the above-mentioned electronic device 110, or the technical solution of the present application may also be executed by the above-mentioned network device 120, and the present application does not limit this.

[0026] After introducing the application scenarios of the embodiments of the present application, the technical solution of the present application will be elaborated in detail below: Figure 2 A flowchart of a method for training an image processing model provided for an embodiment of the present application. This method may be executed by the electronic device 110 as shown in Figure 1 or may be executed by the network device 120 as shown in Figure 1 but is not limited thereto. As shown in Figure 2 the method may include the following steps: Step 210: Obtain a first sample image and a first preset training label corresponding to the first sample image, and a second sample image and a second preset training label corresponding to the second sample image.

[0027] In a specific application scenario, the first sample image is a computed tomography (CT) image of a patient before palliative treatment, which can be used to evaluate the patient's condition, the size, location of the tumor, whether metastasis has occurred, and the involvement of surrounding tissues, etc. The first preset training label corresponding to the first sample image is the palliative treatment efficacy data after the patient undergoes palliative treatment. The palliative treatment efficacy data can be used to represent the treatment efficacy after the patient undergoes palliative treatment. The treatment efficacy of palliative treatment can be determined based on palliative treatment prognosis data such as postoperative symptom control, quality of life assessment, survival period, length of hospital stay, psychological support effect, palliative prognosis index, etc., but is not limited thereto. The second sample image is a CT image of the patient before neoadjuvant chemotherapy. The second preset label corresponding to the second sample image is the neoadjuvant chemotherapy efficacy data after the patient undergoes neoadjuvant chemotherapy. The neoadjuvant chemotherapy efficacy data is used to represent the treatment efficacy after the patient undergoes neoadjuvant chemotherapy and can be determined based on neoadjuvant chemotherapy prognosis data such as pathological complete remission rate, disease-free survival period, overall survival period, distant metastasis rate, local recurrence rate, quality of life indicators, adverse reactions, prognostic factors, etc. Here, both the first sample image and the second sample image include the lesion area of the preset lesion, and the preset lesion can be gastric cancer, but is not limited thereto.

[0028] Step 220: Use the first model parameter of the trained palliative treatment efficacy prediction model as the second model parameter of the image processing model.

[0029] According to an embodiment of the present application, the image processing model includes a first feature extraction layer, a first fully connected layer, and a second fully connected layer. The palliative treatment efficacy prediction model includes a second feature extraction layer and a third fully connected layer. The palliative treatment efficacy prediction model can be trained by the first sample image and the first preset training label. The training steps of the palliative treatment efficacy prediction model may include: inputting the first sample image into the second feature extraction layer to obtain a third palliative feature; inputting the third palliative feature into the third fully connected layer to obtain a third palliative treatment efficacy prediction result, where the third palliative treatment efficacy prediction result is the palliative treatment efficacy prediction data of the first sample image in the palliative treatment efficacy prediction model; with the goal of minimizing the palliative loss function, iteratively update the first model parameter of the palliative treatment efficacy prediction model until the second preset iteration number is reached, and determine that the palliative treatment efficacy prediction model is trained. Here, the palliative loss function is used to reflect the difference between the third palliative treatment efficacy prediction result and the first preset training label.

[0030] In a specific application scenario, the palliative efficacy prediction model can be constructed by a second feature extraction layer based on the ResNet network and a third fully connected layer. The first sample image and the first preset label can be input into the palliative efficacy prediction model for predictive training of palliative efficacy. Specifically, the first sample image can be input into the second feature extraction layer to obtain the third palliative feature in the CT image of the patient before palliative treatment. By inputting the third palliative feature into the third fully connected layer, the palliative efficacy prediction data of the first sample image in the palliative efficacy prediction model can be obtained, which is the third palliative efficacy prediction result. Then, a palliative loss function reflecting the difference between the third palliative efficacy prediction result and the first preset training label is constructed. The palliative loss function can be determined by means such as local error, root mean square error, sum of absolute errors, and cross-entropy loss. This application does not make any limitations. Finally, the first model parameters of the palliative efficacy prediction model are repeatedly adjusted to gradually reduce the deviation between the third palliative efficacy prediction result and the first preset training label. During the training process, the first model parameters are fine-tuned according to the gradient information of the palliative loss function to minimize the palliative loss function until the second preset iteration number is reached, and it is determined that the palliative efficacy prediction model has achieved the best prediction ability under the given iteration constraints. Here, the second preset iteration number can be the same as or different from the first preset iteration number, and this application does not make any restrictions. In addition, the second preset iteration number can be set as needed, such as it can be set to 200, but it is not limited to this.

[0031] According to an embodiment of the present application, the second model parameters include the first feature extraction layer parameters, the first fully connected layer parameters, and the second fully connected layer parameters. The first model parameters include the second feature extraction layer parameters and the third fully connected layer parameters. Using the first model parameters of the trained palliative efficacy prediction model as the second model parameters of the image processing model may include: setting the first feature extraction layer parameters as the second feature extraction layer parameters, setting the first fully connected layer parameters as the third fully connected layer parameters, and setting the initial parameters of the second fully connected layer as the preset fully connected layer parameters.

[0032] In a specific application scenario, the first feature extraction layer and the first fully connected layer of the image processing model are the same as the second feature extraction layer and the third fully connected layer in the palliative efficacy prediction model. Therefore, the second feature extraction layer parameters and the third fully connected layer parameters of the trained palliative efficacy prediction model can be used to set the second model parameters of the image processing model. Specifically, the first feature extraction layer parameters can be set as the second feature extraction layer parameters, the first fully connected layer parameters can be set as the third fully connected layer parameters, and the initial parameters of the second fully connected layer can be set as the preset fully connected layer parameters. In this way, using the first model parameters of the trained palliative efficacy prediction model to set the second model parameters of the image processing model can accelerate the training process of the image processing model and reduce the requirement for the quantity of training sample data of the image processing model.

[0033] Step 230: Iteratively update the second model parameters of the image processing model according to the first sample image, the second sample image, and the second preset training label until the first preset number of iterations is reached, thereby completing the prediction training of the image processing model for the efficacy of neoadjuvant chemotherapy.

[0034] In a specific application scenario, the first sample image, the second sample image, and the second preset training label can be input into the image processing model. According to the similarity between the first sample image and the second sample image, and the second preset training label, the second model parameters of the image processing model are iteratively updated until the first preset number of iterations is reached, so as to perform the prediction training of the image processing model for the efficacy of neoadjuvant chemotherapy. Here, the first preset number of iterations can be set according to the training needs of the image processing model, and can generally be set to 200, but is not limited thereto.

[0035] According to an embodiment of the present application, Figure 3 A flowchart for predicting and training the efficacy of neoadjuvant chemotherapy for an image processing model provided by an embodiment of the present application is shown in Figure 3 As shown, the method includes: Step 310: Input the first sample image into the second feature extraction layer to obtain the first palliative feature.

[0036] In a specific application scenario, the first sample image is input into the second feature extraction layer of the trained palliative efficacy prediction model. The second feature extraction layer extracts the first palliative feature of the first sample image in the palliative treatment prediction model according to the preset lesion area marked in the first sample image. Here, the first palliative feature is the lesion feature of the first sample image corresponding to before palliative treatment.

[0037] Step 320: Input the second sample image into the second feature extraction layer to obtain the second palliative feature, and input the second palliative feature into the third fully connected layer to obtain the first palliative efficacy prediction result.

[0038] In a specific application scenario, the second sample image is input into the second feature extraction layer of the trained palliative efficacy prediction model. The second feature extraction layer extracts the second palliative feature of the second sample image in the palliative treatment prediction model according to the preset lesion area marked in the second sample image. Here, the second palliative feature is the lesion feature of the second sample image corresponding to before palliative treatment. Inputting the second palliative feature into the third fully connected layer of the trained palliative efficacy prediction model can obtain the first palliative efficacy prediction result. Here, the first palliative efficacy prediction result is the palliative efficacy prediction data of the second sample image in the palliative efficacy prediction model.

[0039] Step 330: Input the second sample image into the first feature extraction layer to obtain the neoadjuvant chemotherapy feature.

[0040] In a specific application scenario, the second sample data is input into the first feature extraction layer of the image processing model. The first feature extraction layer extracts the neoadjuvant chemotherapy features of the second sample image in the image processing model according to the preset lesion area marked in the second sample image. Here, the neoadjuvant chemotherapy features are the lesion features of the second sample image corresponding to before neoadjuvant chemotherapy.

[0041] Step 340: Input the neoadjuvant chemotherapy features into the first fully connected layer to obtain the second palliative treatment efficacy prediction result.

[0042] In a specific application scenario, inputting the neoadjuvant features into the first fully connected layer of the image processing model can obtain the second palliative treatment efficacy prediction result of the second sample image in the image processing model. Here, the second palliative treatment efficacy prediction result is the efficacy prediction data of the second sample image in the image processing model for palliative treatment.

[0043] Step 350: Input the neoadjuvant chemotherapy features into the second fully connected layer to obtain the neoadjuvant chemotherapy efficacy prediction result.

[0044] In a specific application scenario, the second fully connected layer is used to output the efficacy prediction data of the second sample image regarding neoadjuvant chemotherapy. Specifically, inputting the neoadjuvant chemotherapy features into the second fully connected layer can obtain the neoadjuvant chemotherapy efficacy prediction result of the second sample image in the image processing model, and the neoadjuvant chemotherapy efficacy prediction result is the efficacy prediction data of the second sample image in the image processing model for neoadjuvant chemotherapy.

[0045] Step 360: Construct a prediction loss function according to the first palliative feature, the neoadjuvant chemotherapy feature, the first palliative treatment efficacy prediction result, the second palliative treatment efficacy prediction result, the neoadjuvant chemotherapy efficacy prediction result, and the second preset training label.

[0046] According to the embodiments of the present application, the steps of constructing the prediction loss function may include: constructing a first loss function according to the cosine similarity between the first palliative feature and the neoadjuvant chemotherapy feature; constructing a second loss function according to the cross-entropy between the first palliative treatment efficacy prediction result and the second palliative treatment efficacy prediction result; constructing a third loss function according to the cross-entropy between the neoadjuvant chemotherapy efficacy prediction result and the second preset training label; weighting the first loss function, the second loss function, and the third loss function respectively based on preset weights, and determining the prediction loss function based on the weighted first loss function, second loss function, and third loss function.

[0047] In a specific application scenario, a first loss function can be constructed based on the cosine similarity between the first palliative features of the first sample image based on the second feature extraction layer and the neoadjuvant features of the second sample image based on the first feature extraction layer. The first loss function based on cosine similarity can be more stable and accurate when dealing with the similarity analysis of feature vectors. A second loss function is constructed using the cross-entropy between the first palliative efficacy prediction result of the second sample image based on the palliative efficacy prediction model and the second palliative efficacy prediction result of the second sample image based on the image processing model; a third loss function is constructed using the cross-entropy between the neoadjuvant chemotherapy efficacy prediction result of the second sample image based on the image processing model and the second preset training label. Finally, the weights of the first loss function, the second loss function, and the third loss function can be set according to actual needs, and the prediction loss function can be determined by summing the weighted loss functions.

[0048] Step 370: With the goal of minimizing the prediction loss function, iteratively update the second model parameters of the image processing model until the first preset number of iterations is reached, and determine that the training of the image processing model is completed.

[0049] In a specific application scenario, the second model parameters of the image processing model can be gradually updated through an iterative algorithm. In each iteration, the image processing model calculates the prediction loss function according to the current model parameters, calculates the gradient of the loss with respect to each parameter through backpropagation, and then adjusts the model parameter values according to the gradient information in the hope of reducing the loss in the next iteration until the first preset number of iterations is reached, and it is determined that the image processing model has achieved the best prediction ability under the given iteration constraints.

[0050] In summary, according to the training method of the image processing model provided in this application, a first sample image and a first preset training label corresponding to the first sample image, as well as a second sample image and a second preset training label corresponding to the second sample image are obtained. Among them, the first sample image is a CT image before palliative treatment, the first preset training label is the palliative treatment efficacy data corresponding to the first sample image, the second sample image is a CT image before neoadjuvant chemotherapy, and the second preset training label is the neoadjuvant chemotherapy efficacy data corresponding to the second sample image; the first model parameters of the trained palliative treatment efficacy prediction model are used as the second model parameters of the image processing model, where the palliative treatment efficacy prediction model is trained by the first sample image and the first preset training label; according to the first sample image, the second sample image, and the second preset training label, the second model parameters of the image processing model are iteratively updated until the first preset number of iterations is reached, and the prediction training of the image processing model for the neoadjuvant chemotherapy efficacy is completed. This process sets the second model parameters of the image processing model based on the first model parameters of the trained palliative treatment efficacy prediction model. Based on the similarity-based incremental learning scheme, on the basis of less neoadjuvant chemotherapy training data, after the palliative treatment efficacy prediction model learns the palliative task, it can provide prior knowledge for learning neoadjuvant chemotherapy data, so that the features extracted by the image processing model have a higher signal-to-noise ratio, less redundancy and noise, further improving the robustness of the image processing model and the prediction accuracy of the neoadjuvant chemotherapy efficacy.

[0051] Figure 4 It is a schematic diagram of the overall process of training an image processing model provided by an embodiment of this application. As Figure 4 shown, first, the first sample image is input into the second feature extraction layer of the palliative treatment efficacy prediction model to obtain the first palliative feature. Then, the second sample image is input into the second feature extraction layer of the palliative treatment efficacy prediction model to obtain the second palliative feature, and the second palliative feature is input into the third fully connected layer to obtain the first palliative treatment prediction result. Next, the second sample image is input into the first feature extraction layer of the image processing model to obtain the neoadjuvant chemotherapy feature. The neoadjuvant chemotherapy feature is input into the first fully connected layer to obtain the second palliative treatment efficacy prediction result, and the neoadjuvant chemotherapy feature is input into the second fully connected layer to obtain the neoadjuvant chemotherapy efficacy prediction result. Finally, the image processing model can be trained according to the cosine similarity between the first palliative feature and the neoadjuvant chemotherapy feature, the cross-entropy between the first palliative treatment efficacy prediction result and the second palliative treatment efficacy prediction result, and the cross-entropy between the neoadjuvant chemotherapy efficacy prediction result and the second preset training label.

[0052] Figure 5 It is a schematic diagram of the process of an image processing method provided by an embodiment of this application. As Figure 5 shown, the steps of this method may include: Step 510: Obtain the target CT image of the target object before neoadjuvant chemotherapy.

[0053] In a specific application scenario, the target CT image of the target object (i.e., the patient) before neoadjuvant chemotherapy can be obtained.

[0054] Step 520: Input the target CT image into the trained image processing model of the first aspect to obtain the efficacy prediction result of the target object after neoadjuvant chemotherapy. The image processing model is used to extract the target neoadjuvant chemotherapy features of the target CT image, and based on the target neoadjuvant chemotherapy features, predict the efficacy data of the target CT image to obtain the efficacy prediction result.

[0055] In a specific application scenario, the image processing model includes a first feature extraction layer, a first fully connected layer, and a second fully connected layer; input the target CT image into the first feature extraction layer of the image processing model to obtain the target neoadjuvant chemotherapy features of the target CT image, and input the target neoadjuvant chemotherapy features into the second fully connected layer to obtain the efficacy prediction result of the target CT image for neoadjuvant chemotherapy. Here, the efficacy prediction result is the predicted efficacy data of the neoadjuvant efficacy. When the predicted efficacy data is greater than the preset value, it is determined that the efficacy effect of the target object based on neoadjuvant chemotherapy is good. When the predicted efficacy data is less than or equal to the preset value, it is determined that the efficacy effect of the target object based on neoadjuvant chemotherapy is poor. The size of the preset value can be set according to actual needs, and this application does not make a limitation.

[0056] In summary, an image processing method provided in this application obtains the target CT image of the target object before neoadjuvant chemotherapy; inputs the target CT image into the trained image processing model of the first aspect to obtain the efficacy prediction result of the target object after neoadjuvant chemotherapy. The image processing model is used to extract the target neoadjuvant chemotherapy features of the target CT image, and based on the target neoadjuvant chemotherapy features, predict the efficacy data of the target CT image to obtain the efficacy prediction result. In view of the fact that the image processing model is obtained by incremental learning based on the trained palliative efficacy prediction model with less neoadjuvant chemotherapy training data, the features extracted by the image processing model have a higher signal-to-noise ratio, less redundancy and noise, and further can improve the accuracy of the efficacy prediction of neoadjuvant chemotherapy.

[0057] Based on the above Figure 2 、 Figure 3 specific description of the training method of the image processing model provided, Figure 6 The following is a schematic structural diagram of an image processing model training device provided by an embodiment of this application. As Figure 6 shown, the device 600 includes: A sample image acquisition module 610 is configured to acquire a first sample image and a first preset training label corresponding to the first sample image, as well as a second sample image and a second preset training label corresponding to the second sample image. Herein, the first sample image is a CT image before palliative care, the first preset training label is palliative care efficacy data corresponding to the first sample image, the second sample image is a CT image before neoadjuvant chemotherapy, and the second preset training label is neoadjuvant chemotherapy efficacy data corresponding to the second sample image; A parameter setting module 620 is configured to use the first model parameters of the trained palliative care efficacy prediction model as the second model parameters of the image processing model, wherein the palliative care efficacy prediction model is obtained by training with the first sample image and the first preset training label; A training module 630 is configured to iteratively update the second model parameters of the image processing model according to the first sample image, the second sample image, and the second preset training label until a first preset number of iterations is reached, thereby completing the prediction training of the image processing model for neoadjuvant chemotherapy efficacy.

[0058] In some embodiments of the present application, the image processing model includes a first feature extraction layer, a first fully connected layer, and a second fully connected layer, and the palliative care efficacy prediction model includes a second feature extraction layer and a third fully connected layer; the training module 630 is further configured to input the first sample image into the second feature extraction layer to obtain a first palliative feature; input the second sample image into the second feature extraction layer to obtain a second palliative feature, input the second palliative feature into the third fully connected layer to obtain a first palliative care efficacy prediction result, wherein the first palliative care efficacy prediction result is palliative care efficacy prediction data of the second sample image in the palliative care efficacy prediction model; input the second sample image into the first feature extraction layer to obtain a neoadjuvant chemotherapy feature; input the neoadjuvant chemotherapy feature into the first fully connected layer to obtain a second palliative care efficacy prediction result, wherein the second palliative care efficacy prediction result is palliative care efficacy prediction data of the second sample image in the image processing model; input the neoadjuvant chemotherapy feature into the second fully connected layer to obtain a neoadjuvant chemotherapy efficacy prediction result, wherein the neoadjuvant chemotherapy efficacy prediction result is neoadjuvant chemotherapy efficacy prediction data of the second sample image in the image processing model; construct a prediction loss function according to the first palliative feature, the neoadjuvant chemotherapy feature, the first palliative care efficacy prediction result, the second palliative care efficacy prediction result, the neoadjuvant chemotherapy efficacy prediction result, and the second preset training label; with the goal of minimizing the prediction loss function, iteratively update the second model parameters of the image processing model until the first preset number of iterations is reached, and determine that the training of the image processing model is completed.

[0059] In some embodiments of the present application, the training module 630 is further configured to construct a first loss function according to the cosine similarity between the first palliative feature and the neoadjuvant chemotherapy feature; construct a second loss function according to the cross-entropy between the first palliative treatment efficacy prediction result and the second palliative treatment efficacy prediction result; construct a third loss function according to the cross-entropy between the neoadjuvant chemotherapy efficacy prediction result and the second preset training label; weight the first loss function, the second loss function, and the third loss function respectively based on a preset weight, and determine a prediction loss function based on the weighted first loss function, second loss function, and third loss function.

[0060] In some embodiments of the present application, the image processing model further includes a second training module, configured to input the first sample image into the second feature extraction layer to obtain a third palliative feature; input the third palliative feature into the third fully connected layer to obtain a third palliative treatment efficacy prediction result, where the third palliative treatment efficacy prediction result is the palliative treatment efficacy prediction data of the first sample image in the palliative treatment efficacy prediction model; with the goal of minimizing the palliative loss function, iteratively update the first model parameters of the palliative treatment efficacy prediction model until a second preset number of iterations is reached, and determine that the training of the palliative treatment efficacy prediction model is completed, where the palliative loss function is used to reflect the difference between the third palliative treatment efficacy prediction result and the first preset training label.

[0061] In some embodiments of the present application, the second model parameters include the first feature extraction layer parameters, the first fully connected layer parameters, and the second fully connected layer parameters, and the first model parameters include the second feature extraction layer parameters and the third fully connected layer parameters; the parameter setting module 620 is further configured to set the first feature extraction layer parameters as the second feature extraction layer parameters, set the first fully connected layer parameters as the third fully connected parameters, and set the initial parameters of the second fully connected layer as the preset fully connected layer parameters.

[0062] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0063] Based on the above Figure 5 specific description of the provided image processing method, Figure 7 FIG. is a schematic structural diagram of an image processing device provided in an embodiment of the present application, as Figure 7 shown, the device 700 includes: An image acquisition module 710, configured to obtain a target CT image of a target object before neoadjuvant chemotherapy; The efficacy prediction module 720 is configured to input the target CT image into the trained image processing model of the first aspect to obtain the efficacy prediction result of the target object after neoadjuvant chemotherapy. The image processing model is used to extract the target neoadjuvant chemotherapy features of the target CT image, and based on the target neoadjuvant chemotherapy features, predict the efficacy data of the target CT image to obtain the efficacy prediction result.

[0064] Regarding the device in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0065] According to the embodiments of the present application, based on the incremental learning of the similarity between the first sample image and the second sample image, the image processing model can, after learning the prediction task of the palliative efficacy prediction model based on the first sample image, provide prior knowledge for learning the neoadjuvant chemotherapy efficacy prediction, so that the features extracted by the image processing model have a higher signal-to-noise ratio, less redundancy, and less noise.

[0066] According to the embodiments of the present application, it is possible to align the features of the first sample image and the second sample image in the feature space according to the feature extraction layer and calculate their cosine similarity, so that the image processing model can, when updating the model parameters, simultaneously refer to the correlation between the first sample image and the second sample image, as well as the common features of the first sample image and the second sample image. Utilizing this common feature can effectively improve the quality of feature extraction of the image processing model. Based on the limited neoadjuvant chemotherapy data (i.e., the second sample image), through the palliative data (i.e., the first sample image), the performance of the image processing model can be further improved.

[0067] According to the embodiments of the present application, incremental learning is used to enable the palliative efficacy prediction model to gradually absorb and learn the second sample image on the basis of the existing training, without having to train from scratch. Furthermore, through the similarity calculation of the first sample image and the second sample image, the image processing model learns the common features between different but similar tasks based on the palliative efficacy prediction model. Under limited data, by increasing the number of similar tasks, the robustness of the features is improved, and the performance of the image processing model is further enhanced. At the same time, in order to balance the importance of each part of the loss function, an adaptive hyperparameter is designed, so that the image processing model can learn how to allocate the weights of the loss function to achieve the best performance. In this way, it is possible to process continuously generated data streams, save computing resources, and continuously optimize the model performance in a limited resource environment.

[0068] In the foregoing, the training apparatus and / or image classification apparatus of the image classification model according to the embodiments of the present application have been described from the perspective of functional modules with reference to the accompanying drawings. It should be understood that the functional modules can be implemented in the form of hardware, or in the form of instructions in software, or in a combination of hardware and software modules. Specifically, the steps of the training method of the image processing model and / or the image processing method according to the embodiments of the present application can be completed by the integrated logic circuit in hardware in the processor and / or instructions in software form. The steps of the training method of the image processing model and / or the image processing method according to the embodiments of the present application in combination with the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art such as a random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps in the training method of the above-mentioned image processing model and / or the image processing method according to the embodiments.

[0069] Figure 8 is a schematic block diagram of an electronic device 800 according to an embodiment provided by the present application.

[0070] As Figure 8 shown, the electronic device 800 may include: A memory 810 and a processor 820. The memory 810 is used to store a computer program and transmit the program code to the processor 820. In other words, the processor 820 can call and run the computer program from the memory 810 to implement the method according to the embodiments of the present application.

[0071] For example, the processor 820 can be used to execute the above method embodiments according to the instructions in the computer program.

[0072] In some embodiments of the present application, the processor 820 may include, but is not limited to: A general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and so on.

[0073] In some embodiments of the present application, the memory 810 includes, but is not limited to: Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be Read-Only Memory (ROM), Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), or flash memory. The volatile memory can be Random Access Memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double DataRate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synch link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0074] In some embodiments of the present application, the computer program can be divided into one or more modules, which are stored in the memory 810 and executed by the processor 820 to complete the method provided by the present application. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the controller.

[0075] As Figure 8 shown, the electronic device 800 may further include: A transceiver 830, which can be connected to the processor 820 or the memory 810.

[0076] Among them, the processor 820 can control the transceiver 830 to communicate with other devices. Specifically, it can send data to other devices or receive data sent by other devices. The transceiver 830 can include a transmitter and a receiver. The transceiver 830 may further include an antenna, and the number of antennas can be one or more.

[0077] It should be understood that the various components in the electronic device are connected through a bus system. Among them, the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus.

[0078] The present application also provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a computer, the computer can execute the methods in the above method embodiments. Or rather, an embodiment provided by the present application also provides a computer program product containing instructions. When the instructions are executed by a computer, the computer executes the methods in the above method embodiments.

[0079] When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a Digital Video Disc (DVD)), or a semiconductor medium (such as a Solid State Disk (SSD)).

[0080] Those of ordinary skill in the art can realize that the modules and algorithm steps of the examples described in combination with the embodiments claimed in this application can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0081] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings, direct couplings, or communication connections shown or discussed with each other can be through some interfaces. The indirect couplings or communication connections of the devices or modules can be in electrical, mechanical, or other forms.

[0082] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules, that is, they can be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. For example, in each embodiment of the present application, the functional modules can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module.

[0083] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A training method for an image processing model, characterized in that, Including: Obtain a first sample image and a first preset training label corresponding to the first sample image, as well as a second sample image and a second preset training label corresponding to the second sample image, where the first sample image is a CT image before palliative care, the first preset training label is palliative care efficacy data corresponding to the first sample image, the second sample image is a CT image before neoadjuvant chemotherapy, and the second preset training label is neoadjuvant chemotherapy efficacy data corresponding to the second sample image; Use the first model parameters of the trained palliative care efficacy prediction model as the second model parameters of the image processing model, where the palliative care efficacy prediction model is obtained by training with the first sample image and the first preset training label; According to the first sample image, the second sample image, and the second preset training label, iteratively update the second model parameters of the image processing model until a first preset number of iterations is reached, and complete the prediction training of the image processing model for neoadjuvant chemotherapy efficacy.

2. The method according to claim 1, characterized in that, The image processing model includes a first feature extraction layer, a first fully connected layer, and a second fully connected layer, and the palliative care efficacy prediction model includes a second feature extraction layer and a third fully connected layer; The step of iteratively updating the second model parameters of the image processing model according to the first sample image, the second sample image, and the second preset training label until a first preset number of iterations is reached and completing the prediction training of the image processing model for neoadjuvant chemotherapy efficacy includes: Input the first sample image into the second feature extraction layer to obtain a first palliative feature; Input the second sample image into the second feature extraction layer to obtain a second palliative feature, and input the second palliative feature into the third fully connected layer to obtain a first palliative care efficacy prediction result, where the first palliative care efficacy prediction result is palliative care efficacy prediction data of the second sample image in the palliative care efficacy prediction model; Input the second sample image into the first feature extraction layer to obtain neoadjuvant chemotherapy features; Input the neoadjuvant chemotherapy features into the first fully connected layer to obtain a second palliative care efficacy prediction result, where the second palliative care efficacy prediction result is palliative care efficacy prediction data of the second sample image in the image processing model; Input the neoadjuvant chemotherapy features into the second fully connected layer to obtain a neoadjuvant chemotherapy efficacy prediction result, where the neoadjuvant chemotherapy efficacy prediction result is neoadjuvant chemotherapy efficacy prediction data of the second sample image in the image processing model; Construct a prediction loss function according to the first palliative feature, the neoadjuvant chemotherapy features, the first palliative care efficacy prediction result, the second palliative care efficacy prediction result, the neoadjuvant chemotherapy efficacy prediction result, and the second preset training label; With the goal of minimizing the prediction loss function, iteratively update the second model parameters of the image processing model until the first preset number of iterations is reached, and determine that the training of the image processing model is completed.

3. The method according to claim 2, characterized in that Constructing a prediction loss function according to the first palliative feature, the neoadjuvant chemotherapy feature, the first palliative efficacy prediction result, the second palliative efficacy prediction result, the neoadjuvant chemotherapy efficacy prediction result, and the second preset training label, includes: Constructing a first loss function according to the cosine similarity between the first palliative feature and the neoadjuvant chemotherapy feature; Constructing a second loss function according to the cross-entropy between the first palliative efficacy prediction result and the second palliative efficacy prediction result; Constructing a third loss function according to the cross-entropy between the neoadjuvant chemotherapy efficacy prediction result and the second preset training label; Based on preset weights, weighting the first loss function, the second loss function, and the third loss function respectively, and determining the prediction loss function based on the weighted first loss function, second loss function, and third loss function.

4. The method according to claim 3, wherein The training steps of the palliative efficacy prediction model include: Inputting the first sample image into the second feature extraction layer to obtain a third palliative feature; Inputting the third palliative feature into the third fully connected layer to obtain a third palliative efficacy prediction result, where the third palliative efficacy prediction result is the palliative efficacy prediction data of the first sample image in the palliative efficacy prediction model; Taking minimizing the palliative loss function as the goal, iteratively updating the first model parameters of the palliative efficacy prediction model until reaching the second preset number of iterations, and determining that the training of the palliative efficacy prediction model is completed, where the palliative loss function is used to reflect the difference between the third palliative efficacy prediction result and the first preset training label.

5. The method according to claim 2, wherein The second model parameters include the first feature extraction layer parameters, the first fully connected layer parameters, and the second fully connected layer parameters, and the first model parameters include the second feature extraction layer parameters and the third fully connected layer parameters; The step of using the first model parameters of the trained palliative efficacy prediction model as the second model parameters of the image processing model includes: Setting the first feature extraction layer parameters as the second feature extraction layer parameters, setting the first fully connected layer parameters as the third fully connected parameters, and setting the initial parameters of the second fully connected layer as the preset fully connected layer parameters.

6. An image processing method, characterized in that, Includes: Obtaining a target CT image of the target object before neoadjuvant chemotherapy; Inputting the target CT image into the trained image processing model according to any one of claims 1 to 5 to obtain the efficacy prediction result of the target object after neoadjuvant chemotherapy, where the image processing model is used to extract the target neoadjuvant chemotherapy feature of the target CT image, and based on the target neoadjuvant chemotherapy feature, predict the efficacy data of the target CT image to obtain the efficacy prediction result.

7. A training device for an image processing model, characterized in that, Includes: A sample image acquisition module, configured to acquire a first sample image and a first preset training label corresponding to the first sample image, as well as a second sample image and a second preset training label corresponding to the second sample image, wherein the first sample image is a CT image before palliative care, the first preset training label is palliative care efficacy data corresponding to the first sample image, the second sample image is a CT image before neoadjuvant chemotherapy, and the second preset training label is neoadjuvant chemotherapy efficacy data corresponding to the second sample image; A parameter setting module, configured to use the first model parameter of the trained palliative care efficacy prediction model as the second model parameter of the image processing model, wherein the palliative care efficacy prediction model is trained by the first sample image and the first preset training label; A training module, configured to iteratively update the second model parameter of the image processing model according to the first sample image, the second sample image, and the second preset training label until a first preset number of iterations is reached, and complete the prediction training of the image processing model for neoadjuvant chemotherapy efficacy.

8. An image processing apparatus, characterized in that, Comprising: An image acquisition module, configured to acquire a target CT image of a target object before neoadjuvant chemotherapy; An efficacy prediction module, configured to input the target CT image into the trained image processing model according to any one of claims 1 to 5, and obtain an efficacy prediction result of the target object after neoadjuvant chemotherapy, wherein the image processing model is configured to extract target neoadjuvant chemotherapy features of the target CT image, and predict efficacy data of the target CT image based on the target neoadjuvant chemotherapy features to obtain the efficacy prediction result.

9. An electronic device, characterized in that, Comprising: A processor and a memory, the memory is configured to store a computer program, and the processor is configured to call and run the computer program stored in the memory to execute the method according to any one of claims 1 - 5, or execute the method according to claim 6.

10. A computer-readable storage medium, characterized in that, For storing a computer program, the computer program causes a computer to execute the method according to any one of claims 1 - 5, or execute the method according to claim 6.

Images