A method, device, equipment and medium for predicting the therapeutic effect of anti-vascular endothelial growth factor (VEGF)
Through the anti-VEGF efficacy prediction model based on OCT images, the deformation field prediction sub-model and efficacy prediction sub-model are used to solve the problem of low prediction accuracy of single OCT images, achieving more accurate VEGF efficacy prediction, reducing labeling and segmentation error.
Patent Information
- Application Number
- CN202210232921.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-09
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-03-09
AI Technical Summary
In the prior art, the anti-VEGF efficacy prediction based on single OCT images has problems such as difficulty in labeling, accumulation of segmentation errors, complex operation and uncertain efficacy, resulting in inaccurate efficacy prediction.
A method for predicting anti-VEGF efficacy is proposed. By obtaining OCT images before anti-VEGF therapy, the anti-VEGF efficacy prediction model has been used to train, including deformation field predictor model, OCT image predictor model after anti-VEGF therapy, and efficacy predictor model , output anti-VEGF treatment to predict OCT images and efficacy results, reduce labeling costs and avoid segmentation errors.
The accuracy of anti-VEGF efficacy prediction is improved, and through deformation field assisted prediction, the changes in the fundus topology before and after treatment are directly output, reducing the error caused by the labeling and segmentation network, enhancing the advantages of OCT imaging, and improving the prediction effect.
Smart Images

Figure CN114565604B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of medical technology, and in particular to a method, device, electronic device and computer-readable storage medium for predicting the efficacy of anti-vascular endothelial growth factor (VEGF). Background Art
[0002] As the global aging problem becomes increasingly serious, wet age-related macular degeneration (AMD) has become one of the major blinding eye diseases. The birth of anti-vascular endothelial growth factor (VEGF) intraocular injection therapy has brought hope to many AMD patients, but this therapy is expensive, the treatment process is painful, and there is generally uncertainty about the efficacy, which makes many patients daunted. Therefore, the industry urgently needs a reliable efficacy prediction method to reduce unnecessary economic and psychological burdens for patients with poor expected efficacy, thereby providing positive encouragement for patients with good expected efficacy.
[0003] Optical Coherence Tomography (OCT) is a conventional eye examination imaging method with the advantages of strong real-time performance, high resolution, non-contact and wide imaging depth, which can extract rich cross-sectional information of fundus structure. Usually, multiple images obtained by cross-shaped scanning or plain scanning can provide a more comprehensive reference. However, due to the high difficulty of operation and low patient cooperation, the scanning sequence images are often seriously missing. Therefore, drug efficacy prediction based on a single OCT will be more valuable.
[0004] In the related technology, the prediction of anti-VEGF efficacy based on a single OCT image is mainly based on the segmentation of wet AMD-related fundus structures and lesion segmentation (effusion, high reflection points, etc.) to extract biological indicators, and add classifier training to obtain an efficacy classification network. Its disadvantages are that the labeling is difficult and intensive, and classification based on segmentation is prone to cumulative errors. Summary of the invention
[0005] The main purpose of the embodiments of the present application is to propose a method, device, electronic device and computer-readable storage medium for predicting the efficacy of anti-vascular endothelial growth factor (VEGF), aiming to improve the accuracy of anti-VEGF efficacy prediction based on a single OCT image.
[0006] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a method for predicting the efficacy of an anti-vascular endothelial growth factor (VEGF) treatment, the method comprising: obtaining an OCT image before anti-VEGF treatment; inputting the OCT image before anti-VEGF treatment into a trained anti-VEGF efficacy prediction model, wherein the anti-VEGF efficacy prediction model comprises a deformation field prediction sub-model, an OCT image prediction sub-model after anti-VEGF treatment, and an efficacy prediction sub-model; the deformation field prediction sub-model outputs a corresponding first predicted deformation field based on the OCT image before anti-VEGF treatment; the OCT image prediction sub-model after anti-VEGF treatment outputs a corresponding first predicted OCT image after anti-VEGF treatment based on the OCT image before anti-VEGF treatment and the first predicted deformation field; the efficacy prediction sub-model outputs a first anti-VEGF efficacy prediction result based on the OCT image before anti-VEGF treatment and the first predicted OCT image after anti-VEGF treatment.
[0007] According to the anti-vascular endothelial growth factor (VEGF) efficacy prediction method provided by some embodiments of the present application, the training process of the deformation field prediction sub-model in the anti-VEGF efficacy prediction model includes: obtaining a training sample set, the training sample set includes multiple training samples, each of the training samples includes a corresponding OCT image of the training sample before anti-VEGF treatment and an OCT image of the training sample after anti-VEGF treatment; for each training sample, according to the corresponding OCT image of the training sample before anti-VEGF treatment and the OCT image of the training sample after anti-VEGF treatment, determining the first deformation field corresponding to the training sample; based on the OCT image of the training sample before anti-VEGF treatment and the first deformation field of each training sample, training the initial deformation field prediction sub-model.
[0008] According to the anti-vascular endothelial growth factor (VEGF) efficacy prediction method provided by some embodiments of the present application, the training process of the anti-VEGF treatment post-treatment OCT image prediction sub-model in the anti-VEGF efficacy prediction model includes: the deformation field prediction sub-model outputs the second predicted deformation field corresponding to each of the anti-VEGF treatment pre-treatment training sample OCT images; based on the anti-VEGF treatment pre-treatment training sample OCT images, the second predicted deformation field and the anti-VEGF treatment post-treatment training sample OCT images corresponding to each training sample, the initial anti-VEGF treatment post-treatment OCT image prediction sub-model is trained.
[0009] According to the anti-vascular endothelial growth factor (VEGF) efficacy prediction method provided by some embodiments of the present application, each of the training samples also includes: a training sample label corresponding to the OCT image of the training sample before anti-VEGF treatment, and the training sample label is an efficacy result label used to indicate whether the treatment is effective or ineffective; correspondingly, the training process of the efficacy prediction sub-model in the anti-VEGF efficacy prediction model includes: the anti-VEGF post-treatment OCT image prediction sub-model outputs the second anti-VEGF post-treatment predicted OCT image corresponding to each of the anti-VEGF pre-treatment training sample OCT images; based on the anti-VEGF pre-treatment training sample OCT image, the second anti-VEGF post-treatment predicted OCT image and the training sample label corresponding to each training sample, the initial efficacy prediction sub-model is trained.
[0010] According to the anti-vascular endothelial growth factor (VEGF) efficacy prediction method provided by some embodiments of the present application, after the initial efficacy prediction sub-model is trained, the method further includes: obtaining a test sample set, the test sample set including multiple test samples, each of the test samples including a corresponding OCT image of the test sample before anti-VEGF treatment and a test sample label; inputting the OCT image of the test sample before anti-VEGF treatment corresponding to each test sample into the anti-VEGF efficacy prediction model to obtain a second anti-VEGF efficacy prediction result corresponding to each test sample; based on the test sample label corresponding to each test sample and the second anti-VEGF efficacy prediction result, determining whether the current anti-VEGF efficacy prediction model meets the training end condition; if the training end condition is met, ending the training process of the anti-VEGF efficacy prediction model to obtain an anti-VEGF efficacy prediction model that has completed training; if the training end condition is not met, adjusting the model parameters of each sub-model in the anti-VEGF efficacy prediction model, and retraining each sub-model in the anti-VEGF efficacy prediction model based on the training sample set.
[0011] According to the anti-vascular endothelial growth factor (VEGF) efficacy prediction method provided by some embodiments of the present application, the determination of whether the current anti-VEGF efficacy prediction model meets the training end condition based on the test sample label corresponding to each of the test samples and the second anti-VEGF efficacy prediction result includes: for each of the test samples, determining whether the corresponding test sample label and the second anti-VEGF efficacy prediction result match; determining the prediction success rate according to the matching results corresponding to each of the test samples; when the prediction success rate is greater than or equal to a preset prediction success rate threshold, determining that the current anti-VEGF efficacy prediction model meets the training end condition; when the prediction success rate is less than the preset prediction success rate threshold, determining that the current anti-VEGF efficacy prediction model does not meet the training end condition.
[0012] According to the anti-vascular endothelial growth factor (VEGF) efficacy prediction method provided by some embodiments of the present application, the efficacy prediction sub-model outputs a first anti-VEGF efficacy prediction result based on the OCT image before anti-VEGF treatment and the predicted OCT image after the first anti-VEGF treatment, including: determining the predicted probability of anti-VEGF treatment based on the OCT image before anti-VEGF treatment and the predicted OCT image after the first anti-VEGF treatment by the efficacy prediction sub-model; when the predicted probability of anti-VEGF treatment is greater than or equal to a preset probability threshold, outputting the first anti-VEGF efficacy prediction result as effective treatment; when the predicted probability of anti-VEGF treatment is less than the preset probability threshold, outputting the first anti-VEGF efficacy prediction result as ineffective treatment.
[0013] To achieve the above-mentioned purpose, the second aspect of the embodiment of the present application proposes an anti-vascular endothelial growth factor (VEGF) efficacy prediction device, the device comprising: an acquisition module, used to acquire an OCT image before anti-VEGF treatment; an input module, used to input the OCT image before anti-VEGF treatment into a trained anti-VEGF efficacy prediction model, wherein the anti-VEGF efficacy prediction model comprises a deformation field prediction sub-model, an OCT image prediction sub-model after anti-VEGF treatment and an efficacy prediction sub-model; a first prediction processing module, used to output a corresponding first predicted deformation field based on the OCT image before anti-VEGF treatment by the deformation field prediction sub-model; a second prediction processing module, used to output a corresponding first predicted OCT image after anti-VEGF treatment by the OCT image prediction sub-model after anti-VEGF treatment based on the OCT image before anti-VEGF treatment and the first predicted deformation field; and a third prediction processing module, used to output a first anti-VEGF efficacy prediction result by the efficacy prediction sub-model based on the OCT image before anti-VEGF treatment and the first predicted OCT image after anti-VEGF treatment.
[0014] To achieve the above-mentioned purpose, the third aspect of the embodiments of the present application proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the anti-vascular endothelial growth factor (VEGF) efficacy prediction method as described in any embodiment of the first aspect above.
[0015] To achieve the above-mentioned purpose, the fourth aspect of the embodiments of the present application proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the anti-vascular endothelial growth factor (VEGF) efficacy prediction method as described in any embodiment of the first aspect above.
[0016] The embodiment of the present application proposes a method, device, electronic device and computer-readable storage medium for predicting the efficacy of anti-vascular endothelial growth factor (VEGF), which obtains a single OCT image before anti-VEGF treatment and inputs the OCT image before anti-VEGF treatment into a trained anti-VEGF efficacy prediction model, so as to output a first anti-VEGF efficacy prediction result through the anti-VEGF efficacy prediction model. The anti-VEGF efficacy prediction model includes a deformation field prediction sub-model, an anti-VEGF post-treatment OCT image prediction sub-model and an efficacy prediction sub-model; the deformation field prediction sub-model is used to output a corresponding first predicted deformation field based on the anti-VEGF pre-treatment OCT image; the anti-VEGF post-treatment OCT image prediction sub-model is used to output a corresponding first anti-VEGF post-treatment predicted OCT image based on the anti-VEGF pre-treatment OCT image and the first predicted deformation field; the efficacy prediction sub-model is used to output a first anti-VEGF efficacy prediction result based on the anti-VEGF pre-treatment OCT image and the first predicted OCT image after the anti-VEGF treatment. The anti-VEGF efficacy prediction model provided in the embodiment of the present application can make efficacy judgments based on the input OCT images before anti-VEGF treatment, and has a wider range of application scenarios; the model provided in the present application is an end-to-end structure, which can directly output the predicted effect after inputting the OCT images before treatment, and does not need to be predicted based on segmentation, thereby reducing the annotation cost and avoiding the cumulative errors caused by the annotation and segmentation networks; the model provided in the present application predicts the deformation field before and after treatment, and uses the deformation field as a sensitive structure to assist in efficacy prediction, thereby giving full play to the advantages of OCT, and guiding the model to make judgments in a more targeted manner by predicting the changes in the fundus topological structure after treatment, and the prediction effect is often more accurate. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a schematic diagram of a flow chart of a method for predicting the efficacy of an anti-vascular endothelial growth factor (VEGF) treatment provided in an embodiment of the present application;
[0018] Figure 2 It is a schematic diagram of the structure of the anti-VEGF efficacy prediction model provided in the embodiments of the present application;
[0019] Figure 3 yes Figure 1 A schematic diagram of the sub-step flow chart of step S150 in FIG.
[0020] Figure 4 It is a schematic diagram of the training process of the deformation field prediction sub-model in the anti-VEGF efficacy prediction model provided in the embodiment of the present application;
[0021] Figure 5It is a schematic diagram of the training process of the OCT image prediction sub-model after anti-VEGF treatment in the anti-VEGF efficacy prediction model provided in the embodiment of the present application;
[0022] Figure 6 It is a schematic diagram of the training process of the efficacy prediction sub-model in the anti-VEGF efficacy prediction model provided in the embodiment of the present application;
[0023] Figure 7 It is a schematic diagram of the structure of an anti-vascular endothelial growth factor (VEGF) therapeutic efficacy prediction device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0025] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0026] First, some nouns involved in this application are analyzed:
[0027] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0028] Artificial Neural Networks (ANNs): Also referred to as Neural Networks (NNs) or Connection Model, it is an algorithmic mathematical model that imitates the behavioral characteristics of animal neural networks and performs distributed parallel information processing. This network relies on the complexity of the system and adjusts the interconnected relationships between a large number of internal nodes to achieve the purpose of processing information.
[0029] Wet age-related macular degeneration (AMD): AMD is a major blinding eye disease, which is an aging change in the structure of the macula. It is mainly manifested by the decreased ability of retinal pigment epithelial cells to phagocytose and digest the outer segment membrane of photoreceptors, resulting in the retention of residual small bodies of incompletely digested membrane in the basal cell plasma, which are then excreted outside the cell and deposited on Bruch's membrane to form drusen.
[0030] Optical Coherence Tomography (OCT): OCT is a non-invasive imaging technology that uses low-coherence light to capture high-resolution information of biological tissues. It has been widely used in the medical field, especially in retinal imaging in the field of ophthalmology. Compared with traditional medical imaging technologies such as fundus cameras and ultrasound, OCT images can obtain more three-dimensional structural information of the physiological tissues of the eye, thus providing doctors with more auxiliary information.
[0031] Vascular endothelial growth factor (VEGF): VEGF, also known as vascular permeability factor (VPF), is a highly specific vascular endothelial cell growth factor that promotes vascular permeability, extracellular matrix degeneration, endothelial cell migration, proliferation and angiogenesis. Intraocular injection of anti-VEGF is an effective treatment for wet AMD, but anti-VEGF injection therapy is expensive and has strict indications, and the efficacy varies for different patients.
[0032] The birth of anti-VEGF intraocular injection therapy has brought hope to many AMD patients, but this therapy is expensive, the treatment process is painful, and there is generally uncertainty about the efficacy, which makes many patients daunted. Therefore, the industry is in urgent need of a reliable efficacy prediction method to reduce unnecessary economic and psychological burdens for patients whose expected efficacy is not obvious, thereby providing positive encouragement for patients with good expected efficacy. In the related technology, the shortcomings of the anti-VEGF efficacy prediction method are:
[0033] (1) Efficacy prediction based on sequential images: Image sequence prediction based on M-shaped scanning or plain scanning often requires high sequence integrity. In the real world, due to the high difficulty of operating the instrument and the poor cooperation of patients, it is easy to have a high degree of missing sequence images. Therefore, based on a single OCT image, the application scenario is more extensive.
[0034] (2) Segmentation-based efficacy prediction: Extract the fundus structures and lesions (effusion, high anti-points, etc.) related to wet AMD for segmentation training, and send the segmentation results to the classifier to predict the efficacy. The disadvantages are high labeling cost and difficulty, and the two-stage operation easily accumulates labeling errors and network segmentation errors, making the prediction results inaccurate.
[0035] The main purpose of the embodiments of the present application is to propose a method, device, electronic device and computer-readable storage medium for predicting anti-VEGF efficacy, aiming to improve the accuracy of anti-VEGF efficacy prediction based on a single OCT.
[0036] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0037] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0038] The anti-VEGF efficacy prediction method provided in the embodiment of the present application relates to the field of artificial intelligence and digital medical technology. The anti-VEGF efficacy prediction method provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or it can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the anti-VEGF efficacy prediction method, etc., but is not limited to the above forms.
[0039] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0040] See also Figure 1 , Figure 1 FIG. 2 is a flow chart showing a method for predicting the therapeutic effect of an anti-vascular endothelial growth factor (VEGF) according to an embodiment of the present application. Figure 1 As shown, the anti-VEGF efficacy prediction method provided in the embodiments of the present application includes but is not limited to the following steps:
[0041] Step S110, obtaining an OCT image before anti-VEGF treatment.
[0042] It is understandable that the embodiment of the present application obtains an OCT image of the patient before anti-VEGF treatment, and predicts the anti-VEGF efficacy based on the OCT image. The OCT image can be obtained by performing a cross-shaped scan or a plain scan of the patient's eyes using an OCT device. The OCT image includes the patient's retinal tissue information.
[0043] Optionally, after obtaining the initial OCT image of the patient, the image portion of the OCT image scanned for the macula area can also be obtained, wherein the macula area is located in the center of the retina and is the location where the central visual cells of the human eye retina are most concentrated. By extracting the area scanned for the macula area, the peripheral area without valid information can be removed, thereby saving calculation time in the subsequent calculation process.
[0044] Optionally, after acquiring the OCT image collected for the macular area, the OCT image may be preprocessed, specifically including: performing image correction on the OCT image, and performing contrast enhancement processing after correction, wherein the image correction includes image tilt correction and / or image brightness correction.
[0045] Step S120, inputting the OCT image before anti-VEGF treatment into the trained anti-VEGF efficacy prediction model, wherein the anti-VEGF efficacy prediction model includes a deformation field prediction sub-model, an OCT image prediction sub-model after anti-VEGF treatment and an efficacy prediction sub-model.
[0046] In the embodiment of the present application, the trained anti-VEGF efficacy prediction model is used to predict the efficacy of the input OCT images before anti-VEGF treatment.
[0047] See also Figure 2 , Figure 2 The schematic diagram of the structure of the anti-VEGF efficacy prediction model provided in the embodiment of the present application is shown. The anti-VEGF efficacy prediction model includes several parts: a deformation field prediction sub-model, an anti-VEGF post-treatment OCT image prediction sub-model and an efficacy prediction sub-model. The deformation field prediction sub-model is used to output the corresponding first predicted deformation field based on the anti-VEGF pre-treatment OCT image; the anti-VEGF post-treatment OCT image prediction sub-model is used to output the corresponding first anti-VEGF post-treatment predicted OCT image based on the anti-VEGF pre-treatment OCT image and the first predicted deformation field; the efficacy prediction sub-model is used to output the first anti-VEGF efficacy prediction result based on the anti-VEGF pre-treatment OCT image and the first anti-VEGF post-treatment predicted OCT image. It should be understood that the training method of the anti-VEGF efficacy prediction model provided in the embodiment of the present application will be described in detail below.
[0048] Step S130: the deformation field prediction sub-model outputs a corresponding first predicted deformation field based on the OCT image before anti-VEGF treatment.
[0049] It can be understood that the deformation field described in the embodiments of the present application refers to the deformation field of the OCT image of the patient after receiving anti-VEGF treatment relative to the OCT image before receiving anti-VEGF treatment.
[0050] When implementing the anti-VEGF efficacy prediction, the present application uses the deformation field prediction sub-model in the anti-VEGF efficacy prediction model to perform deformation field prediction on the patient's OCT image before anti-VEGF treatment to obtain a first predicted deformation field, so as to predict the patient's OCT image after anti-VEGF treatment based on the first predicted deformation field in the subsequent steps.
[0051] It is understandable that the deformation field between the images before and after anti-VEGF treatment can be used as a sensitive structural area in the image to assist in the prediction of therapeutic effect, thereby improving the accuracy of therapeutic effect prediction.
[0052] Step S140 , the post-anti-VEGF treatment OCT image prediction sub-model outputs a corresponding first post-anti-VEGF treatment predicted OCT image based on the pre-anti-VEGF treatment OCT image and the first predicted deformation field.
[0053] It is understandable that the image before treatment can generate the image after treatment after spatial transformation using the deformation field. Therefore, after obtaining the first predicted deformation field, the OCT image before anti-VEGF treatment can be used as the reference image, and then the reference image is spatially transformed according to the first predicted deformation field to generate the deformed first predicted OCT image after anti-VEGF treatment.
[0054] Specifically, the embodiment of the present application inputs the first predicted deformation field and the OCT image before anti-VEGF treatment into the OCT image prediction sub-model after anti-VEGF treatment, and outputs the corresponding first predicted OCT image after anti-VEGF treatment through the OCT image prediction sub-model after anti-VEGF treatment.
[0055] Step S150 , the efficacy prediction sub-model outputs a first anti-VEGF efficacy prediction result based on the OCT image before anti-VEGF treatment and the first predicted OCT image after anti-VEGF treatment.
[0056] It can be understood that after obtaining the predicted OCT image of the patient after the first anti-VEGF treatment, the topological changes of the fundus structure after treatment can be predicted by comparing the OCT image of the patient before anti-VEGF treatment and the predicted OCT image after the first anti-VEGF treatment, thereby predicting the anti-VEGF efficacy of the patient.
[0057] Specifically, in an embodiment of the present application, the predicted OCT image after the first anti-VEGF treatment and the OCT image before the anti-VEGF treatment are input into the efficacy prediction sub-model, and the efficacy prediction sub-model outputs the corresponding first anti-VEGF efficacy prediction result. Here, the first anti-VEGF efficacy prediction result can be any one of effective treatment and ineffective treatment.
[0058] The efficacy prediction sub-model may specifically be a classifier, such as inception, resnet or efficient-net, which is used to forward encode the input image and output two classifications of "effective treatment" or "ineffective treatment".
[0059] For example, see Figure 3 The step of outputting a first anti-VEGF efficacy prediction result by the efficacy prediction sub-model based on the OCT image before anti-VEGF treatment and the first predicted OCT image after anti-VEGF treatment may specifically include the following steps:
[0060] Step S151, determining the predicted probability of anti-VEGF treatment by the efficacy prediction sub-model based on the OCT image before anti-VEGF treatment and the predicted OCT image after the first anti-VEGF treatment;
[0061] Step S152, when the predicted probability of anti-VEGF treatment is greater than or equal to a preset probability threshold, outputting a first anti-VEGF efficacy prediction result as effective treatment;
[0062] Step S153: when the predicted probability of the anti-VEGF treatment is less than the preset probability threshold, outputting the first anti-VEGF efficacy prediction result as ineffective treatment.
[0063] The solution of the embodiment of the present application is to obtain a single OCT image before anti-VEGF treatment, and input the OCT image before anti-VEGF treatment into the trained anti-VEGF efficacy prediction model, so as to output a first anti-VEGF efficacy prediction result through the anti-VEGF efficacy prediction model. Wherein, the anti-VEGF efficacy prediction model includes a deformation field prediction sub-model, an anti-VEGF post-treatment OCT image prediction sub-model and an efficacy prediction sub-model; the deformation field prediction sub-model is used to output the corresponding first predicted deformation field based on the anti-VEGF pre-treatment OCT image; the anti-VEGF post-treatment OCT image prediction sub-model is used to output the corresponding first anti-VEGF post-treatment predicted OCT image based on the anti-VEGF pre-treatment OCT image and the first predicted deformation field; the efficacy prediction sub-model is used to output the first anti-VEGF efficacy prediction result based on the anti-VEGF pre-treatment OCT image and the first anti-VEGF post-treatment predicted OCT image. The anti-VEGF efficacy prediction model provided in the embodiment of the present application can make efficacy judgments based on the input OCT images before anti-VEGF treatment, and has a wider range of application scenarios; the model provided in the present application is an end-to-end structure, which can directly output the predicted effect after inputting the OCT images before treatment, and does not need to be predicted based on segmentation, thereby reducing the annotation cost and avoiding the cumulative errors caused by the annotation and segmentation networks; the model provided in the present application predicts the deformation field before and after treatment, and uses the deformation field as a sensitive structure to assist in efficacy prediction, thereby giving full play to the advantages of OCT, and guiding the model to make judgments in a more targeted manner by predicting the changes in the fundus topological structure after treatment, and the prediction effect is often more accurate.
[0064] The following is a detailed description of the training method of the anti-VEGF efficacy prediction model provided in the embodiments of the present application.
[0065] It is understandable that the training of the anti-VEGF efficacy prediction model in the embodiment of the present application can be divided into a deformation field prediction sub-model training stage, an anti-VEGF treatment OCT image prediction sub-model training stage, and an efficacy prediction sub-model training stage. Optionally, a testing stage can also be included to evaluate the overall prediction effect of the anti-VEGF efficacy prediction model.
[0066] See also Figure 4 The training process of the deformation field prediction sub-model in the anti-VEGF efficacy prediction model includes:
[0067] Step S201 : obtaining a training sample set, wherein the training sample set includes a plurality of training samples, and each of the training samples includes a corresponding OCT image of a training sample before anti-VEGF treatment and an OCT image of a training sample after anti-VEGF treatment.
[0068] It can be understood that the acquired training samples may include OCT images of training samples before anti-VEGF treatment and OCT images of training samples after anti-VEGF treatment corresponding to multiple patients.
[0069] Step S202 : for each training sample, according to the corresponding OCT image of the training sample before anti-VEGF treatment and the OCT image of the training sample after anti-VEGF treatment, determining a first deformation field corresponding to the training sample.
[0070] It can be understood that the first deformation field refers to the deformation field of the training sample OCT image after the anti-VEGF treatment relative to the training sample OCT image before the anti-VEGF treatment.
[0071] Step S203 : training an initial deformation field prediction sub-model based on the OCT images of the training samples before anti-VEGF treatment and the first deformation field of each training sample.
[0072] Specifically, the OCT images of the training samples before anti-VEGF treatment in each training sample are used as input, and the first deformation field is used as the expected output to train the deformation field prediction sub-model. Furthermore, the third predicted deformation field corresponding to each OCT image of the training samples before anti-VEGF treatment can be output by the deformation field prediction sub-model, and then the value of the first loss function can be determined according to the corresponding first deformation field and third predicted deformation field, and then the value of the first loss function can be used to determine whether the current deformation field prediction sub-model meets the training end condition. If it does, the training is terminated; if it does not, the parameters of the deformation field prediction sub-model are adjusted, and the deformation field prediction sub-model is continued to be trained based on the training samples. Here, the first loss function can be a cross entropy loss function.
[0073] See also Figure 5The training process of the anti-VEGF treatment post-treatment OCT image prediction sub-model in the anti-VEGF efficacy prediction model includes:
[0074] Step S301: The deformation field prediction sub-model outputs a second predicted deformation field corresponding to each of the OCT images of the training samples before anti-VEGF treatment.
[0075] It can be understood that each of the anti-VEGF treatment training sample OCT images is input into the deformation field prediction sub-model trained in steps S201-S203, and the deformation field prediction sub-model outputs the second predicted deformation field corresponding to each of the anti-VEGF treatment training sample OCT images.
[0076] Step S302 : training an initial post-anti-VEGF treatment OCT image prediction sub-model based on the pre-anti-VEGF treatment training sample OCT images, the second predicted deformation field, and the post-anti-VEGF treatment training sample OCT images corresponding to each training sample.
[0077] It can be understood that the OCT images of the training samples before anti-VEGF treatment and the second predicted deformation field corresponding to each training sample are used as inputs, and the OCT images of the training samples after anti-VEGF treatment corresponding to each training sample are used as expected outputs to train the OCT image prediction sub-model after anti-VEGF treatment. Further, the OCT image prediction sub-model after anti-VEGF treatment can output the third predicted OCT image after anti-VEGF treatment corresponding to each OCT image of the training samples before anti-VEGF treatment, and then determine the value of the second loss function according to the corresponding OCT image of the training samples after anti-VEGF treatment and the third predicted OCT image after anti-VEGF treatment, and then determine whether the current OCT image prediction sub-model after anti-VEGF treatment meets the training end condition according to the value of the second loss function, and if so, end the training; if not, adjust the parameters of the OCT image prediction sub-model after anti-VEGF treatment, and continue to train the OCT image prediction sub-model after anti-VEGF treatment based on the training samples. Here, the second loss function can be a cross entropy loss function.
[0078] It can be understood that each of the training samples further includes: a training sample label corresponding to the training sample OCT image before anti-VEGF treatment, and the training sample label is an efficacy result label for indicating whether the treatment is effective or ineffective.
[0079] For corresponding Figure 6 The training process of the efficacy prediction sub-model in the anti-VEGF efficacy prediction model includes:
[0080] Step S401: The anti-VEGF treatment OCT image prediction sub-model outputs a second anti-VEGF treatment predicted OCT image corresponding to each of the anti-VEGF treatment training sample OCT images before the anti-VEGF treatment.
[0081] It can be understood that the second predicted OCT image after anti-VEGF treatment corresponding to each of the training sample OCT images before anti-VEGF treatment is obtained by using the OCT image prediction sub-model after anti-VEGF treatment trained in steps S301-S302.
[0082] Step S402: training the initial efficacy prediction sub-model based on the training sample OCT image before anti-VEGF treatment, the second predicted OCT image after anti-VEGF treatment and the training sample label corresponding to each training sample.
[0083] It can be understood that the OCT image of the training sample before anti-VEGF treatment and the predicted OCT image after the second anti-VEGF treatment corresponding to each training sample are used as input, and the training sample label corresponding to each training sample is used as the expected output to train the efficacy prediction sub-model. Furthermore, the prediction label corresponding to each training sample can be output by the efficacy prediction sub-model, and then the value of the third loss function can be determined according to the corresponding training sample label and the prediction label, and then the value of the third loss function can be used to determine whether the current efficacy prediction sub-model meets the training end condition. If it does, the training is terminated; if not, the parameters of the efficacy prediction sub-model are adjusted, and the efficacy prediction sub-model is continued to be trained based on the training sample. Here, the third loss function can be a cross entropy loss function.
[0084] Optionally, after step S402, the training process further includes: obtaining a test sample set, the test sample set including multiple test samples, each of the test samples including a corresponding OCT image of the test sample before anti-VEGF treatment and a test sample label; inputting the OCT image of the test sample before anti-VEGF treatment corresponding to each test sample into the anti-VEGF efficacy prediction model to obtain a second anti-VEGF efficacy prediction result corresponding to each test sample; based on the test sample label corresponding to each test sample and the second anti-VEGF efficacy prediction result, determining whether the current anti-VEGF efficacy prediction model meets a training end condition; if the training end condition is met, ending the training process of the anti-VEGF efficacy prediction model to obtain an anti-VEGF efficacy prediction model that has completed training; if the training end condition is not met, adjusting the model parameters of each sub-model in the anti-VEGF efficacy prediction model, and retraining each sub-model in the anti-VEGF efficacy prediction model based on the training sample set.
[0085] The embodiment of the present application evaluates whether the anti-VEGF efficacy prediction model as a whole achieves the expected training effect through a test set. If so, the training is terminated to obtain the anti-VEGF efficacy prediction model that has completed training; otherwise, the model parameters of each sub-model in the anti-VEGF efficacy prediction model are adjusted, and each sub-model in the anti-VEGF efficacy prediction model is retrained based on the training sample set.
[0086] Specifically, the determination of whether the current anti-VEGF efficacy prediction model meets the training end condition based on the test sample label corresponding to each of the test samples and the second anti-VEGF efficacy prediction result includes: for each of the test samples, determining whether the corresponding test sample label and the second anti-VEGF efficacy prediction result match; determining the prediction success rate according to the matching results corresponding to each of the test samples; when the prediction success rate is greater than or equal to a preset prediction success rate threshold, determining that the current anti-VEGF efficacy prediction model meets the training end condition; when the prediction success rate is less than the preset prediction success rate threshold, determining that the current anti-VEGF efficacy prediction model does not meet the training end condition. Here, based on the prediction success rate of the current anti-VEGF efficacy prediction model, it is evaluated whether the current anti-VEGF efficacy prediction model as a whole has achieved the expected training effect.
[0087] According to the anti-vascular endothelial growth factor (VEGF) efficacy prediction method provided by some embodiments of the present application, the efficacy prediction sub-model outputs a first anti-VEGF efficacy prediction result based on the OCT image before anti-VEGF treatment and the predicted OCT image after the first anti-VEGF treatment, including: determining the predicted probability of anti-VEGF treatment based on the OCT image before anti-VEGF treatment and the predicted OCT image after the first anti-VEGF treatment by the efficacy prediction sub-model; when the predicted probability of anti-VEGF treatment is greater than or equal to a preset probability threshold, outputting the first anti-VEGF efficacy prediction result as effective treatment; when the predicted probability of anti-VEGF treatment is less than the preset probability threshold, outputting the first anti-VEGF efficacy prediction result as ineffective treatment.
[0088] See also Figure 7 The present application embodiment proposes a device for predicting the therapeutic effect of anti-vascular endothelial growth factor (VEGF), the device comprising:
[0089] An acquisition module, used for acquiring OCT images before anti-VEGF treatment;
[0090] An input module, used to input the OCT image before anti-VEGF treatment into the trained anti-VEGF efficacy prediction model, wherein the anti-VEGF efficacy prediction model includes a deformation field prediction sub-model, an OCT image prediction sub-model after anti-VEGF treatment and an efficacy prediction sub-model;
[0091] A first prediction processing module, configured to output a corresponding first predicted deformation field based on the OCT image before anti-VEGF treatment by the deformation field prediction sub-model;
[0092] A second prediction processing module, configured to output a corresponding first predicted OCT image after anti-VEGF treatment based on the OCT image before anti-VEGF treatment and the first predicted deformation field by the OCT image prediction sub-model after anti-VEGF treatment;
[0093] The third prediction processing module is used to output a first anti-VEGF efficacy prediction result based on the OCT image before anti-VEGF treatment and the predicted OCT image after the first anti-VEGF treatment by the efficacy prediction sub-model.
[0094] Optionally, the anti-vascular endothelial growth factor (VEGF) therapeutic efficacy prediction device of the embodiment of the present application further includes a training module, which includes a first training unit, a second training unit and a third training unit.
[0095] Among them, the first training unit is specifically used to: obtain a training sample set, the training sample set includes multiple training samples, each of the training samples includes a corresponding OCT image of the training sample before anti-VEGF treatment and an OCT image of the training sample after anti-VEGF treatment; for each training sample, according to the corresponding OCT image of the training sample before anti-VEGF treatment and the OCT image of the training sample after anti-VEGF treatment, determine the first deformation field corresponding to the training sample; based on the OCT image of the training sample before anti-VEGF treatment and the first deformation field of each training sample, train the initial deformation field prediction sub-model.
[0096] The second training unit is specifically used to: output the second predicted deformation field corresponding to each of the training sample OCT images before anti-VEGF treatment by the deformation field prediction sub-model; and train the initial anti-VEGF treatment OCT image prediction sub-model based on the training sample OCT images before anti-VEGF treatment and the second predicted deformation field corresponding to each training sample.
[0097] The third training unit is specifically used for: training sample labels corresponding to the training sample OCT images before anti-VEGF treatment, wherein the training sample labels are efficacy result labels used to indicate whether the treatment is effective or ineffective; correspondingly, the training process of the efficacy prediction sub-model in the anti-VEGF efficacy prediction model includes: outputting the second anti-VEGF post-treatment predicted OCT image corresponding to each of the training sample OCT images before anti-VEGF treatment by the anti-VEGF post-treatment OCT image prediction sub-model; and training the initial efficacy prediction sub-model based on the anti-VEGF pre-treatment training sample OCT image, the second anti-VEGF post-treatment predicted OCT image and the training sample labels corresponding to each training sample.
[0098] Optionally, the training module also includes a testing unit, which is specifically used to: after training the initial efficacy prediction sub-model, obtain a test sample set, the test sample set including multiple test samples, each of the test samples including a corresponding OCT image of the test sample before anti-VEGF treatment and a test sample label; input the OCT image of the test sample before anti-VEGF treatment corresponding to each test sample into the anti-VEGF efficacy prediction model to obtain a second anti-VEGF efficacy prediction result corresponding to each test sample; based on the test sample label corresponding to each test sample and the second anti-VEGF efficacy prediction result, determine whether the current anti-VEGF efficacy prediction model meets the training end condition; if the training end condition is met, end the training process of the anti-VEGF efficacy prediction model to obtain an anti-VEGF efficacy prediction model that has completed training; if the training end condition is not met, adjust the model parameters of each sub-model in the anti-VEGF efficacy prediction model, and retrain each sub-model in the anti-VEGF efficacy prediction model based on the training sample set.
[0099] The present application also provides an electronic device, including:
[0100] at least one processor;
[0101] and, a memory communicatively coupled to the at least one processor;
[0102] Wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the anti-vascular endothelial growth factor (VEGF) efficacy prediction method as described in any of the above embodiments.
[0103] The embodiment of the present application further proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for predicting the therapeutic effect of anti-vascular endothelial growth factor (VEGF) as described in any of the above embodiments.
[0104] The above embodiments may be used in combination, and modules with the same name in different embodiments may be the same or different.
[0105] The above describes specific embodiments of the present application, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0106] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0107] The apparatus, device, computer-readable storage medium and method provided in the embodiments of the present application correspond to each other. Therefore, the apparatus, device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the corresponding apparatus, device and computer storage medium will not be repeated here.
[0108] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.
[0109] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.
[0110] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0111] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing the embodiments of the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0112] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the embodiments of the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0113] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0114] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0116] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0117] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0118] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0119] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0120] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.
[0121] Embodiments of the present application may be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. Embodiments of the present application may also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.
[0122] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0123] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A method for predicting the therapeutic effect of anti-vascular endothelial growth factor (VEGF), characterized in that: The method comprises: Obtain OCT images before anti-VEGF treatment; Inputting the OCT image before anti-VEGF treatment into a trained anti-VEGF efficacy prediction model, wherein the anti-VEGF efficacy prediction model includes a deformation field prediction sub-model, an OCT image prediction sub-model after anti-VEGF treatment, and an efficacy prediction sub-model; The deformation field prediction sub-model outputs a corresponding first predicted deformation field based on the OCT image before anti-VEGF treatment; The anti-VEGF treatment OCT image prediction sub-model outputs a corresponding first anti-VEGF treatment predicted OCT image based on the anti-VEGF treatment OCT image before the anti-VEGF treatment and the first predicted deformation field; The efficacy prediction sub-model outputs a first anti-VEGF efficacy prediction result based on the OCT image before anti-VEGF treatment and the first predicted OCT image after anti-VEGF treatment.
2. The method according to claim 1, characterized in that: The training process of the deformation field prediction sub-model in the anti-VEGF efficacy prediction model includes: Acquire a training sample set, wherein the training sample set includes a plurality of training samples, and each training sample includes a corresponding OCT image of a training sample before anti-VEGF treatment and an OCT image of a training sample after anti-VEGF treatment; For each training sample, determining a first deformation field corresponding to the training sample according to the corresponding OCT image of the training sample before anti-VEGF treatment and the OCT image of the training sample after anti-VEGF treatment; Based on the OCT images of the training samples before anti-VEGF treatment and the first deformation field of each training sample, an initial deformation field prediction sub-model is trained.
3. The method according to claim 2, characterized in that The training process of the OCT image prediction sub-model after anti-VEGF treatment in the anti-VEGF efficacy prediction model includes: The deformation field prediction sub-model outputs a second predicted deformation field corresponding to each of the OCT images of the training samples before anti-VEGF treatment; Based on the OCT images of the training samples before anti-VEGF treatment, the second predicted deformation field and the OCT images of the training samples after anti-VEGF treatment corresponding to each training sample, the initial OCT image prediction sub-model after anti-VEGF treatment is trained.
4. The method according to claim 3, characterized in that Each of the training samples further includes: a training sample label corresponding to the training sample OCT image before anti-VEGF treatment, wherein the training sample label is an efficacy result label for indicating whether the treatment is effective or ineffective; The training process of the efficacy prediction sub-model in the anti-VEGF efficacy prediction model includes: Outputting, by the anti-VEGF treatment OCT image prediction sub-model, a second anti-VEGF treatment post-treatment predicted OCT image corresponding to each of the anti-VEGF treatment pre-treatment training sample OCT images; The initial efficacy prediction sub-model is trained based on the training sample OCT image before anti-VEGF treatment, the second predicted OCT image after anti-VEGF treatment and the training sample label corresponding to each training sample.
5. The method according to claim 4, characterized in that After the initial efficacy prediction sub-model is trained, the method further includes: Acquire a test sample set, wherein the test sample set includes a plurality of test samples, each of the test samples includes a corresponding OCT image of the test sample before anti-VEGF treatment and a test sample label; Inputting the OCT images of the test samples before anti-VEGF treatment corresponding to each test sample into the anti-VEGF efficacy prediction model to obtain the second anti-VEGF efficacy prediction results corresponding to each test sample; Determining whether the current anti-VEGF efficacy prediction model meets a training end condition based on the test sample labels corresponding to each of the test samples and the second anti-VEGF efficacy prediction result; When the training end condition is met, the training process of the anti-VEGF efficacy prediction model is ended to obtain an anti-VEGF efficacy prediction model that has completed training; When the training end condition is not met, the model parameters of each sub-model in the anti-VEGF efficacy prediction model are adjusted, and each sub-model in the anti-VEGF efficacy prediction model is retrained based on the training sample set.
6. The method according to claim 5, characterized in that The step of determining whether the current anti-VEGF efficacy prediction model meets a training end condition based on the test sample labels corresponding to each of the test samples and the second anti-VEGF efficacy prediction result comprises: For each of the test samples, determining whether the corresponding test sample label and the second anti-VEGF efficacy prediction result match; Determining a prediction success rate according to the matching results corresponding to each of the test samples; When the prediction success rate is greater than or equal to a preset prediction success rate threshold, determining that the current anti-VEGF efficacy prediction model meets the training end condition; When the prediction success rate is less than a preset prediction success rate threshold, it is determined that the current anti-VEGF efficacy prediction model does not meet the training end condition.
7. The method according to claim 1, characterized in that The outputting of the first anti-VEGF efficacy prediction result by the efficacy prediction sub-model based on the OCT image before the anti-VEGF treatment and the first predicted OCT image after the anti-VEGF treatment includes: Determining the predicted probability of anti-VEGF treatment by the efficacy prediction sub-model based on the OCT image before anti-VEGF treatment and the predicted OCT image after the first anti-VEGF treatment; When the predicted probability of anti-VEGF treatment is greater than or equal to a preset probability threshold, outputting a first anti-VEGF efficacy prediction result as effective treatment; When the predicted probability of the anti-VEGF treatment is less than the preset probability threshold, the first anti-VEGF efficacy prediction result is output as ineffective treatment.
8. A device for predicting the therapeutic effect of anti-vascular endothelial growth factor (VEGF), characterized in that: The device comprises: An acquisition module, used for acquiring OCT images before anti-VEGF treatment; An input module, used to input the OCT image before anti-VEGF treatment into the trained anti-VEGF efficacy prediction model, wherein the anti-VEGF efficacy prediction model includes a deformation field prediction sub-model, an OCT image prediction sub-model after anti-VEGF treatment and an efficacy prediction sub-model; A first prediction processing module, configured to output a corresponding first predicted deformation field based on the OCT image before anti-VEGF treatment by the deformation field prediction sub-model; A second prediction processing module, configured to output a corresponding first predicted OCT image after anti-VEGF treatment based on the OCT image before anti-VEGF treatment and the first predicted deformation field by the OCT image prediction sub-model after anti-VEGF treatment; The third prediction processing module is used to output a first anti-VEGF efficacy prediction result based on the OCT image before anti-VEGF treatment and the predicted OCT image after the first anti-VEGF treatment by the efficacy prediction sub-model.
9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program, and the computer program is executed by the at least one processor to enable the at least one processor to perform the anti-vascular endothelial growth factor (VEGF) efficacy prediction method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for predicting the therapeutic effect of an anti-vascular endothelial growth factor (VEGF) according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Anti-vascular endothelial growth factor VEGF curative effect prediction device and method thereof
CN111920375A
Image prediction model training and application method and device, equipment and storage medium
CN113269812A
