Tumor curative effect evaluation method and system based on medical image

Abstract

The invention relates to a tumor curative effect evaluation method and system based on a medical image, and the method comprises the steps: obtaining a first PET-CT image and a second PET-CT image of a blood system tumor patient before and after the treatment process, and determining the focus change information of a focus region based on the first PET-CT image and the second PET-CT image; performing maximum density projection on the first PET-CT image and / or the second PET-CT image along different directions to generate maximum density projection images of a plurality of visual angles; determining global image information of a focus area based on the maximum density projection images of the multiple visual angles; determining a multi-dimensional score corresponding to the curative effect of the tumor patient based on the global image information; and fusing the focus change information and the multi-dimensional score to generate tumor curative effect evaluation information.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of auxiliary medical technology, and in particular to a method and system for evaluating tumor efficacy based on medical imaging. Background Art

[0002] Currently, lymphoma is one of the most common malignant tumors of the blood system. In the clinical treatment of tumors such as lymphoma, regular efficacy evaluation is of great significance. It can not only timely evaluate the effectiveness of the current treatment plan, but also help predict the future development trend of the tumor and provide a basis for subsequent improvement of the treatment plan.

[0003] With the development and popularization of medical imaging, the efficacy evaluation method based on PET / CT images has become the main imaging evaluation method for lymphoma. By observing PET / CT images, imaging physicians can compare and analyze in detail the location, size, morphology and relationship with surrounding tissues of lymphoma lesions before and after treatment, and evaluate the treatment effect of patients. However, the manual evaluation of lymphoma efficacy based on imaging physicians is labor-intensive and time-consuming, and the evaluation consistency of different imaging physicians is poor, which reduces the accuracy and efficiency of lymphoma efficacy evaluation.

[0004] In related technologies, in order to improve the accuracy and efficiency of lymphoma efficacy evaluation, artificial intelligence technology has been introduced into the field of medical imaging, and the intelligent and automated efficacy evaluation methods for medical image analysis through deep learning and other technologies have gradually become mainstream. However, artificial intelligence technology is still progressing slowly in the field of automatic efficacy evaluation of lymphoma. For example, related technologies are mostly limited to single links such as lesion segmentation and property identification in PET / CT images, resulting in the need to further improve the accuracy of efficacy evaluation; in addition, methods based on deep learning often lack good interpretability and cannot provide clear and specific evaluation information for clinicians (different from radiologists) and patients. The clinical practicality is low, which limits its application in clinical diagnosis and treatment. Summary of the invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, an embodiment of the present invention provides a method and system for evaluating tumor efficacy based on medical images.

[0006] In a first aspect, an embodiment of the present invention provides a method for evaluating tumor efficacy based on medical imaging, comprising: Acquire a first PET-CT image and a second PET-CT image twice before and after the treatment of a patient with a hematological tumor, and determine lesion change information of the lesion area based on the first PET-CT image and the second PET-CT image; Performing maximum density projection on the first PET-CT image and / or the second PET-CT image along different directions to generate maximum density projection images from multiple viewing angles; determining global image information of the lesion area based on the maximum density projection images from multiple viewing angles; The Deauville score corresponding to the therapeutic effect of the tumor patient is determined based on the global image information; and the lesion change information and the Deauville score are integrated to generate tumor therapeutic effect evaluation information.

[0007] In one embodiment, determining the global image information of the lesion area based on the maximum density projection images of the multiple viewing angles includes: For the maximum density projection image of each viewing angle, the corresponding feature extraction model is used to extract image feature information; wherein the feature extraction model corresponding to the maximum density projection image of each viewing angle is different and is obtained by pre-training the original machine learning model based on the sample maximum density projection image; The image feature information corresponding to the maximum density projection image of each viewing angle is fused to obtain the fused global image information.

[0008] In one embodiment, the different directions include at least two of a coronal direction, a sagittal direction, a right anterior lateral direction, and a left anterior lateral direction.

[0009] In one embodiment, determining the Deauville score corresponding to the therapeutic effect of the tumor patient based on the global image information includes: The global image information is input into a target regression prediction model to obtain a corresponding Deauville score; wherein the target regression prediction model is obtained by pre-training an original regression prediction model based on the sample global image information.

[0010] In one embodiment, determining the lesion change information of the lesion area based on the first PET-CT image and the second PET-CT image includes: Segmenting and determining a first lesion region image and a second lesion region image from the first PET-CT image and the second PET-CT image respectively; The first lesion area image and the second lesion area image are image registered, and lesion change information of the lesion area is determined based on the registered first lesion area image and the second lesion area image, wherein the lesion change information includes the location distribution, quantity, morphological boundaries, and change information of the maximum standardized uptake ratio SUVmax of the lesions.

[0011] In one embodiment, the fusing of the lesion change information and the Deauville score to generate tumor efficacy evaluation information includes: Combining the lesion change information and the Deauville score to generate input text information; performing feature encoding on the input text information to obtain a semantic vector; Based on the semantic vector, domain knowledge texts whose semantic similarity with the semantic vector is greater than a preset threshold are retrieved from an expert knowledge vector library; wherein the expert knowledge vector library is a pre-constructed knowledge vector library related to the evaluation of therapeutic efficacy of hematological tumors, and contains a plurality of documents related to the evaluation of therapeutic efficacy of hematological tumors that are pre-text-encoded and indexed; The domain knowledge text is fused with the input text information to obtain target text information; the target text information is input into a large language model to obtain tumor efficacy evaluation information.

[0012] In one embodiment, the target text information is used to provide professional prompt information for the large language model, so that the large language model outputs accurate and clinically understandable efficacy evaluation information.

[0013] In a second aspect, an embodiment of the present invention provides a tumor efficacy evaluation system based on medical imaging, comprising: A lesion change determination module, used to obtain a first PET-CT image and a second PET-CT image twice before and after the treatment of a patient with a hematological tumor, and determine lesion change information of a lesion area based on the first PET-CT image and the second PET-CT image; A global information determination module, configured to perform maximum density projection on the first PET-CT image and / or the second PET-CT image along different directions to generate maximum density projection images of multiple viewing angles; and determine global image information of the lesion area based on the maximum density projection images of the multiple viewing angles; The efficacy evaluation generation module is used to determine the Deauville score corresponding to the efficacy of the tumor patient based on the global image information; and to fuse the lesion change information and the Deauville score to generate tumor efficacy evaluation information.

[0014] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned methods for evaluating tumor efficacy based on medical imaging.

[0015] In a fourth aspect, an embodiment of the present invention provides an electronic device, including: Processor; and Memory for storing computer programs; The processor is configured to execute the tumor efficacy evaluation method based on medical images in any of the above-mentioned embodiments by executing the computer program.

[0016] The technical solution provided by the embodiment of the present invention has the following advantages compared with the prior art: The method and system for evaluating the therapeutic effect of tumors based on medical images provided in the embodiment of the present invention obtains the first PET-CT image and the second PET-CT image twice before and after the treatment of a patient with a hematological tumor, determines the lesion change information of the lesion area based on the first PET-CT image and the second PET-CT image; performs maximum density projection (MIP) on the first PET-CT image and / or the second PET-CT image along different directions to generate maximum density projection images of multiple viewing angles; determines the global image information of the lesion area based on the maximum density projection images of the multiple viewing angles; determines the Deauville score corresponding to the therapeutic effect of the tumor patient based on the global image information; and fuses the lesion change information and the Deauville score to generate tumor efficacy evaluation information. In this way, the scheme of this embodiment mines the global lesion information related to the lesion in the PET-CT image based on the multi-view MIP map to realize the accurate evaluation of the Deauville score, and at the same time fuses the lesion change information obtained from the previous and next PET-CT images, that is, the local lesion information, so that the PET-CT image features can be analyzed more comprehensively and accurately, thereby improving the accuracy of the efficacy evaluation of patients with hematological tumors such as lymphoma patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 This is a flow chart of a method for evaluating tumor efficacy based on medical imaging according to an embodiment of the present invention; Figure 2 Schematic diagram of different directions of maximum density projection of an image according to an embodiment of the present invention; Figure 3 A flow chart of a method for obtaining global image information according to an embodiment of the present invention; Figure 4 This is a flow chart of a method for evaluating tumor efficacy based on medical imaging according to another embodiment of the present invention; Figure 5 This is a flow chart of a method for evaluating tumor efficacy based on medical imaging according to another embodiment of the present invention; Figure 6 Schematic diagram of a tumor efficacy evaluation system based on medical imaging according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to more clearly understand the above-mentioned objectives, features and advantages of the present invention, the scheme of the present invention will be further described below. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0021] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present invention, rather than all of the embodiments.

[0022] It should be understood that, in the following, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0023] In the related technology, it is first necessary to automatically locate, segment, identify and quantitatively analyze the lesions in the PET / CT images, and evaluate the Deauville Score (DS) based on the changes in the lesions before and after treatment, and specifically evaluate the efficacy of lymphoma according to the set rules. However, this method is highly complex, especially the accuracy of the Deauville score evaluation is very dependent on the accuracy of the single result of the quantitative analysis of the lesion area, resulting in poor feasibility and stability of the method, which leads to poor accuracy of the Deauville score evaluation, and the accuracy of the efficacy evaluation needs to be further improved.

[0024] Figure 1 The flowchart of a tumor efficacy evaluation method based on medical imaging according to an embodiment of the present invention is shown below. The method can be executed by a computing device such as a computer or a mobile terminal, and specifically may include the following steps: Step S101: Acquire a first PET-CT image and a second PET-CT image twice before and after the treatment of a patient with a hematological tumor, and determine lesion change information of the lesion area based on the first PET-CT image and the second PET-CT image.

[0025] For example, the blood system tumor may be lymphoma, but is not limited thereto. The first PET-CT image and the second PET-CT image may be two adjacent PET-CT images during the patient's treatment, such as the previous examination and the reexamination, but are not limited thereto. These images may be stored in the hospital's information system for retrieval and analysis when used.

[0026] In one embodiment, step S101 determines lesion change information of the lesion area based on the first PET-CT image and the second PET-CT image, including: segmenting and determining the first lesion area image and the second lesion area image from the first PET-CT image and the second PET-CT image, respectively; performing image registration on the first lesion area image and the second lesion area image, and determining lesion change information of the lesion area based on the registered first lesion area image and the second lesion area image, the lesion change information including change information of location distribution, quantity, morphological boundaries, and maximum standardized uptake ratio SUVmax of the lesions.

[0027] As an example, in order to obtain local lesion information related to lymphoma efficacy evaluation in the PET-CT images before and after, that is, lesion change information, the Lugano standard for lymphoma staging and efficacy evaluation, which is currently recognized internationally, can be used in this embodiment to measure the changes in lesions before and after treatment. First, the lesion area can be segmented from the PET-CT image, and the location and benign and malignant nature of different lesions can be further determined, so as to obtain information such as the location distribution, quantity, size, morphological boundaries, and maximum standardized uptake ratio (SUVmax) of lymphoma lesions. Specifically, methods based on feature comparison, deep learning, etc. can be used to achieve automatic segmentation, positioning, and qualitative analysis of the above-mentioned lesions. Next, in order to accurately compare and obtain the changes in lesions in the PET-CT images before and after treatment, the images (i.e., local images) after segmentation of the PET-CT images before and after treatment can be further image registered. Specifically, methods based on key point detection, deep learning, etc. can be used to achieve the above-mentioned image registration process. Finally, by matching the corresponding lesions on the registered PET / CT images, the changes in the position, number, size, morphological boundaries, SUVmax and other information of different lesions before and after treatment are statistically calculated, thereby achieving accurate measurement of the changes in different lesions before and after treatment, that is, the lesion change information, which is conducive to improving the accuracy of the final efficacy evaluation.

[0028] Step S102: Perform maximum intensity projection on the first PET-CT image and / or the second PET-CT image along different directions to generate maximum intensity projection (MIP, Maximum Intensity Projection) images of multiple viewing angles; determine the global image information of the lesion area based on the maximum intensity projection images of the multiple viewing angles.

[0029] Exemplarily, in one embodiment, the different directions include at least two of a coronal direction, a sagittal direction, a right anterior lateral direction, and a left anterior lateral direction. Figure 2 As shown, in a specific example, different directions may include but are not limited to four directions, namely, coronal direction, sagittal direction, right anterior lateral direction, and left anterior lateral direction. In this embodiment, multiple MIP images of different perspectives are generated by PET images before and after treatment, and the subsequent automatic evaluation of Deauville score is realized by combining information from different perspectives to enhance the stability of the evaluation, while avoiding errors caused by lesion quantitative analysis and background metabolism calculation in related technologies, thereby improving the accuracy of the evaluation results.

[0030] Specifically, the three-dimensional PET image is converted into a MIP image by maximum intensity projection. Figure 2 As shown in the figure, in addition to conventional coronal and sagittal projections, the PET image is also projected at maximum density along the right anterior lateral and left anterior lateral directions to obtain multi-view MIP images. This method can present lymphoma lesions from different perspectives, thereby more comprehensively displaying the global distribution of lesions in patients before and after treatment, making the global image information of the determined lesion area more accurate.

[0031] Step S103: determining the Deauville score corresponding to the therapeutic effect of the tumor patient based on the global image information; fusing the lesion change information and the Deauville score to generate tumor therapeutic effect evaluation information.

[0032] Exemplarily, in this embodiment, the evaluation of the efficacy of lymphoma depends not only on the analysis of the changes of the local lesions before and after, but also on the analysis of the global PET-CT image. It is necessary to integrate the local image information, i.e., the lesion change information, and the global image information to effectively evaluate the efficacy. Specifically, based on the multi-view MIP image, accurate global image information related to the lesion in the PET image can be mined, thereby achieving an accurate evaluation of the Deauville score, that is, the Deauville score can be accurately evaluated based on the multi-view MIP image. Finally, the local information and the global information are fused, such as extracting and quantitatively analyzing the lymphoma lesions in the PET-CT image to obtain local information related to the efficacy, and using the multi-view MIP image to mine the global information in the PET image to accurately evaluate the Deauville score. In this way, the fusion of local and global information can more comprehensively and accurately analyze the PET-CT image features and enhance the accuracy and stability of the evaluation of the efficacy of lymphoma.

[0033] The solution of this embodiment mines the global information of lesions related to lesions in PET-CT images based on the multi-view MIP map to achieve accurate evaluation of the Deauville score, and at the same time integrates the lesion change information obtained from the previous and next PET-CT images, that is, the local information of the lesions. In this way, the PET-CT image characteristics can be analyzed more comprehensively and accurately, thereby improving the accuracy of the efficacy evaluation of patients with hematological tumors such as lymphoma.

[0034] In one embodiment, step S102 determines the global image information of the lesion area based on the maximum density projection images of the multiple perspectives, including: for the maximum density projection image of each perspective, using the corresponding feature extraction model to extract image feature information; wherein the feature extraction model corresponding to the maximum density projection image of each perspective is different and is pre-trained based on the original machine learning model of the sample maximum density projection image; and fusing the image feature information corresponding to the maximum density projection image of each perspective to obtain the fused global image information.

[0035] Specifically, as an example, refer to Figure 3 As shown in , the feature extraction model can be an image encoder, which can be obtained by training the original machine learning model with the sample maximum density projection image. The original machine learning model can at least include a convolutional neural network model or a Transformer model, but is not limited thereto. Exemplarily, the present invention extracts global image feature information from multi-view MIP images and fuses image feature information of MIP images of different viewpoints, thereby realizing automatic and accurate prediction of subsequent Deauville scores. Figure 3 As shown in the figure, firstly, the global image features related to the Deauville score are extracted from the MIP images of different perspectives through various image encoders. Then, the image features of different perspectives are fused together, and the fusion process can be achieved by addition, merging along the channel direction, etc. The global image information obtained in this way is more accurate, making the final efficacy evaluation more accurate.

[0036] Based on the above embodiments, in one embodiment, the Deauville score corresponding to the therapeutic effect of a tumor patient determined based on the global image information includes: inputting the global image information into a target regression prediction model to obtain a corresponding Deauville score; wherein the target regression prediction model is obtained by pre-training the original regression prediction model based on the sample global image information.

[0037] Exemplarily, the Deauville score is regressed and predicted using the multi-view fused global image information obtained above. Specifically, the original regression prediction model can use a support vector machine, a multi-layer perceptron, etc., which can be trained in an end-to-end manner to obtain a target regression prediction model, and in the application stage, the model-based Deauville score is automatically evaluated according to the fused global image information corresponding to the multi-view MIP image. In this way, a more accurate Deauville score can be obtained, making the final efficacy evaluation more accurate.

[0038] Based on any one of the above embodiments, in one embodiment, in combination with reference Figure 4 As shown, the step of fusing the lesion change information and the Deauville score in step S103 to generate tumor efficacy evaluation information may specifically include the following steps: Step S301: Combining the lesion change information and the Deauville score to generate input text information; performing feature encoding on the input text information to obtain a semantic vector; Step S302: Based on the semantic vector, retrieve and determine domain knowledge texts whose semantic similarity with the semantic vector is greater than a preset threshold from an expert knowledge vector library; wherein the expert knowledge vector library is a pre-constructed knowledge vector library related to the evaluation of therapeutic efficacy of hematological tumors, and contains a plurality of documents related to the evaluation of therapeutic efficacy of hematological tumors that are pre-text-encoded and indexed; Step S303: Fusing the domain knowledge text with the input text information to obtain target text information; inputting the target text information into a large language model to obtain tumor efficacy evaluation information.

[0039] In one embodiment, the target text information is used to provide professional prompt information for the large language model, so that the large language model outputs accurate and clinically understandable efficacy evaluation information. The large language model can be a large language model currently provided by various manufacturers on the market, such as ChatGPT.

[0040] Exemplary, reference Figure 5 As shown, the present invention further proposes a therapeutic effect evaluation generation method based on RAG (Retrieval-augmented Generation) and a large language model to enhance the accuracy, professionalism and interpretability of the therapeutic effect evaluation of, for example, lymphoma, so that the therapeutic effect evaluation has good interpretability, and can provide clear and specific evaluation information for clinicians (different from radiologists) and patients, which is conducive to its wide application in clinical diagnosis and treatment.

[0041] Specifically, the present invention first uses the local and global information combination that has been obtained to obtain the original input text information, which contains the description of the number, size, location, morphology, SUVmax and other aspects of the local lesions before and after treatment, as well as the Deauville score of the patient. Then, in order to allow the large language model to better understand and analyze the treatment effect of lymphoma, the present invention constructs a lymphoma efficacy evaluation professional knowledge base. Specifically, a large number of field documents related to lymphoma efficacy evaluation are first collected, such as relevant typical cases, international standards, diagnosis and treatment guidelines, etc., and these document information are encoded and indexed by a text encoder, so as to construct an expert knowledge vector library related to lymphoma efficacy evaluation. Then, after the original input text information is feature encoded to obtain a semantic vector, the field knowledge related to efficacy evaluation is retrieved in the expert knowledge vector library according to the semantic similarity, and it is merged with the original input text information to obtain a model prompt text information that integrates professional knowledge, that is, the target text information. Specifically, the prompt text information not only contains the local lesions and global imaging information of the current patient, but also includes professional information such as case records and relevant diagnosis and treatment standards similar to the patient's situation, thereby providing a more comprehensive and professional prompt information for the large language model. Finally, the model prompt text information integrated with professional knowledge is input into the large language model to generate lymphoma efficacy evaluation information. Figure 5 As shown in the figure, the lymphoma efficacy evaluation information not only includes the final evaluation results, but also analyzes and interprets information such as local lesion changes before and after treatment and Deauville score based on the input patient conditions and professional knowledge, thereby providing efficacy evaluation content that is easy for clinical understanding. In addition, by retrieving relevant domain knowledge from the professional knowledge base, it can not only effectively enhance the professionalism, authenticity and stability of the content generated by the large language model, but also facilitate the private domain management of various sensitive data in the medical field, ensuring the security and privacy protection of medical data.

[0042] In this embodiment, the extracted local information and global information are summarized and analyzed through a large language model, and professional domain knowledge is introduced through retrieval enhanced generation, which reduces the generation of irrelevant information to enhance data accuracy and security, and outputs accurate, professional, and highly interpretable lymphoma efficacy evaluation results.

[0043] The large language model in the prior art can analyze and answer medical-related questions, thereby assisting clinical decision-making and improving clinical diagnosis and treatment efficiency. However, in specific professional fields such as lymphoma efficacy evaluation, it is still easy to produce outputs that are irrelevant to the problem or untrue, limiting its application in clinical diagnosis and treatment. The above scheme in this embodiment can output accurate, professional, and highly interpretable lymphoma efficacy evaluation results, so the scheme can be widely used in clinical diagnosis and treatment.

[0044] In a specific example, an embodiment of the present invention aims to achieve automatic efficacy evaluation of, for example, lymphoma based on PET / CT images. To this end, the present invention proposes an automatic efficacy evaluation method that integrates local information and global information of lesions. After obtaining PET / CT images of lymphoma patients before and after treatment, the method first extracts and calculates local information of the lymphoma lesion area from the PET / CT images to obtain measurement of lesion change information before and after treatment. Next, multi-view MIP images are generated using PET images, and global image information related to efficacy evaluation is mined therefrom to achieve automatic and accurate evaluation of the Deauville score. Finally, the Deauville score that integrates the above local information and global information automatically generates lymphoma efficacy evaluation information through a large language model based on RAG.

[0045] It should be noted that, although the steps of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc. In addition, it is also easy to understand that these steps may be, for example, executed synchronously or asynchronously in multiple modules / processes / threads.

[0046] like Figure 6 As shown, an embodiment of the present invention provides a tumor efficacy evaluation system based on medical imaging, comprising: The lesion change determination module 401 is used to obtain a first PET-CT image and a second PET-CT image twice before and after the treatment of a patient with a hematological tumor, and determine lesion change information of a lesion area based on the first PET-CT image and the second PET-CT image; A global information determination module 402 is configured to perform maximum density projection on the first PET-CT image and / or the second PET-CT image along different directions to generate maximum density projection images of multiple viewing angles; and determine global image information of the lesion area based on the maximum density projection images of the multiple viewing angles; The efficacy evaluation generation module 403 is used to determine the Deauville score corresponding to the efficacy of the tumor patient based on the global image information; and fuse the lesion change information and the Deauville score to generate tumor efficacy evaluation information.

[0047] In one embodiment, the global information determination module determines the global image information of the lesion area based on the maximum density projection images of the multiple viewing angles, including: For the maximum density projection image of each viewing angle, the corresponding feature extraction model is used to extract image feature information; wherein the feature extraction model corresponding to the maximum density projection image of each viewing angle is different and is obtained by pre-training the original machine learning model based on the sample maximum density projection image; The image feature information corresponding to the maximum density projection image of each viewing angle is fused to obtain the fused global image information.

[0048] In one embodiment, the different directions include at least two of a coronal direction, a sagittal direction, a right anterior lateral direction, and a left anterior lateral direction.

[0049] In one embodiment, the efficacy evaluation generation module determines the Deauville score corresponding to the efficacy of the tumor patient based on the global image information, including: The global image information is input into a target regression prediction model to obtain a corresponding Deauville score; wherein the target regression prediction model is obtained by pre-training an original regression prediction model based on the sample global image information.

[0050] In one embodiment, the lesion change determination module determines the lesion change information of the lesion area based on the first PET-CT image and the second PET-CT image, including: Segmenting and determining a first lesion region image and a second lesion region image from the first PET-CT image and the second PET-CT image respectively; The first lesion area image and the second lesion area image are image registered, and lesion change information of the lesion area is determined based on the registered first lesion area image and the second lesion area image, wherein the lesion change information includes the location distribution, quantity, morphological boundaries, and change information of the maximum standardized uptake ratio SUVmax of the lesions.

[0051] In one embodiment, the efficacy evaluation generation module fuses the lesion change information and the Deauville score to generate tumor efficacy evaluation information, including: Combining the lesion change information and the Deauville score to generate input text information; performing feature encoding on the input text information to obtain a semantic vector; Based on the semantic vector, domain knowledge texts whose semantic similarity with the semantic vector is greater than a preset threshold are retrieved from an expert knowledge vector library; wherein the expert knowledge vector library is a pre-constructed knowledge vector library related to the evaluation of therapeutic efficacy of hematological tumors, and contains a plurality of documents related to the evaluation of therapeutic efficacy of hematological tumors that are pre-text-encoded and indexed; The domain knowledge text is fused with the input text information to obtain target text information; the target text information is input into a large language model to obtain tumor efficacy evaluation information.

[0052] In one embodiment, the target text information is used to provide professional prompt information for the large language model, so that the large language model outputs accurate and clinically understandable efficacy evaluation information.

[0053] Regarding the system in the above embodiment, the specific manner in which each module performs operations and the corresponding technical effects brought about have been described in detail in the embodiment of the method, and will not be elaborated here.

[0054] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of a module or unit described above can be further divided into multiple modules or units for concretization. The components displayed as modules or units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Those of ordinary skill in the art can understand and implement it without paying creative work.

[0055] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for evaluating tumor efficacy based on medical imaging described in any of the above embodiments is implemented.

[0056] Exemplarily, the readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0057] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0058] An embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory is used to store a computer program, wherein the processor is configured to execute the tumor efficacy evaluation method based on medical images in any of the above embodiments by executing the computer program.

[0059] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0060] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments described herein, but should conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for evaluating tumor efficacy based on medical imaging, characterized in that: include: Acquire a first PET-CT image and a second PET-CT image twice before and after the treatment of a patient with a hematological tumor, and determine lesion change information of the lesion area based on the first PET-CT image and the second PET-CT image; Performing maximum density projection on the first PET-CT image and / or the second PET-CT image along different directions to generate maximum density projection images from multiple viewing angles; determining global image information of the lesion area based on the maximum density projection images from multiple viewing angles; The Deauville score corresponding to the therapeutic effect of the tumor patient is determined based on the global image information; and the lesion change information and the Deauville score are integrated to generate tumor therapeutic effect evaluation information.

2. The method according to claim 1, characterized in that: The determining of global image information of the lesion area based on the maximum density projection images of the multiple viewing angles includes: For the maximum density projection image of each viewing angle, the corresponding feature extraction model is used to extract image feature information; wherein the feature extraction model corresponding to the maximum density projection image of each viewing angle is different and is obtained by pre-training the original machine learning model based on the sample maximum density projection image; The image feature information corresponding to the maximum density projection image of each viewing angle is fused to obtain the fused global image information.

3. The method according to claim 1, characterized in that The different directions include at least two of a coronal direction, a sagittal direction, a right anterior lateral direction, and a left anterior lateral direction.

4. The method according to claim 1, characterized in that: Determining the Deauville score corresponding to the therapeutic effect of the tumor patient based on the global image information includes: The global image information is input into a target regression prediction model to obtain a corresponding Deauville score; wherein the target regression prediction model is obtained by pre-training an original regression prediction model based on the sample global image information.

5. The method according to claim 1, characterized in that The determining of lesion change information of the lesion area based on the first PET-CT image and the second PET-CT image includes: Segmenting and determining a first lesion region image and a second lesion region image from the first PET-CT image and the second PET-CT image respectively; The first lesion area image and the second lesion area image are image registered, and lesion change information of the lesion area is determined based on the registered first lesion area image and the second lesion area image, wherein the lesion change information includes the location distribution, quantity, morphological boundaries, and change information of the maximum standardized uptake ratio SUVmax of the lesions.

6. The method according to any one of claims 1 to 5, characterized in that The fusing of the lesion change information and the Deauville score to generate tumor efficacy evaluation information includes: Combining the lesion change information and the Deauville score to generate input text information; performing feature encoding on the input text information to obtain a semantic vector; Based on the semantic vector, domain knowledge texts whose semantic similarity with the semantic vector is greater than a preset threshold are retrieved from an expert knowledge vector library; wherein the expert knowledge vector library is a pre-constructed knowledge vector library related to the evaluation of therapeutic efficacy of hematological tumors, and contains a plurality of documents related to the evaluation of therapeutic efficacy of hematological tumors that are pre-text-encoded and indexed; The domain knowledge text is fused with the input text information to obtain target text information; the target text information is input into a large language model to obtain tumor efficacy evaluation information.

7. The method according to claim 6, characterized in that The target text information is used to provide professional prompt information for the large language model, so that the large language model outputs accurate and clinically understandable efficacy evaluation information.

8. A tumor efficacy evaluation system based on medical imaging, characterized in that: include: A lesion change determination module, used to obtain a first PET-CT image and a second PET-CT image twice before and after the treatment of a patient with a hematological tumor, and determine lesion change information of a lesion area based on the first PET-CT image and the second PET-CT image; A global information determination module, configured to perform maximum density projection on the first PET-CT image and / or the second PET-CT image along different directions to generate maximum density projection images of multiple viewing angles; and determine global image information of the lesion area based on the maximum density projection images of the multiple viewing angles; The efficacy evaluation generation module is used to determine the Deauville score corresponding to the efficacy of the tumor patient based on the global image information; and to fuse the lesion change information and the Deauville score to generate tumor efficacy evaluation information.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for evaluating tumor efficacy based on medical images according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; as well as Memory for storing computer programs; Wherein, the processor is configured to execute the tumor efficacy evaluation method based on medical imaging according to any one of claims 1 to 7 by executing the computer program.

Images