Complication prediction devices, methods, electronic devices and storage media

Abstract

The present application discloses a complication prediction device, method, electronic device and storage medium, which relate to the field of medical technology. The device includes: an acquisition unit for acquiring medical images, radiotherapy dose information and vital signs information of a target object; a feature extraction unit for extracting features of medical images, radiotherapy dose information and vital signs information respectively, and obtaining medical image features, dose features and vital signs features, wherein the medical image features, dose features and vital signs features belong to time series features; a complication determination unit for determining the complication information of the target object according to the medical image features, dose features and vital signs features. The present application solves the technical problem that the complication information of a patient is predicted only by image information and dose information in the prior art, resulting in inaccurate prediction of the complication information of the patient.

Description

Technical Field

[0001] The present application relates to the field of medical technology, and specifically, to a complication prediction device, method, electronic device and storage medium. Background Art

[0002] Radiotherapy is a commonly used method in the treatment of malignant tumors, but radiotherapy will bring certain biological effects, among which late complications of radiotherapy are a serious problem in the process of radiotherapy. The occurrence of radiotherapy complications can be manifested in a variety of clinical manifestations, such as spinal cord edema after radiotherapy, radiation esophagitis, radiation pneumonia, tumor recurrence after radiotherapy, bone marrow suppression, etc. Radiotherapy complications have a variety of clinical manifestations, and the time and severity of complications, and complications also have mutual influence. In order to reduce the harm of complications to the life and health of patients, it is necessary to predict complications during radiotherapy, so that doctors can intervene in complications during radiotherapy.

[0003] However, in the prior art, the patient's complication information is usually predicted through the patient's medical imaging information or radiotherapy dose information, which leads to the technical problem of inaccurate prediction of the patient's complication information.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The present application provides a complication prediction device, method, electronic device and storage medium to at least solve the technical problem in the prior art of predicting the patient's complication information only through image information and dosage information, resulting in inaccurate prediction of the patient's complication information.

[0006] According to one aspect of the present application, a complication prediction device is provided, including: an acquisition unit, used to acquire medical images, radiotherapy dose information and vital signs information of a target object; a feature extraction unit, used to extract features of the medical images, radiotherapy dose information and vital signs information, respectively, to obtain medical image features, dose features and vital signs features, wherein the medical image features, dose features and vital signs features are time series features; and a complication determination unit, used to determine the complication information of the target object based on the medical image features, dose features and vital signs features.

[0007] Optionally, in the complication prediction device provided in the present application, the acquisition unit includes: a first acquisition subunit, used to acquire original vital signs information of the target object; a first determination subunit, used to determine vital signs variable information of the target object based on the original vital signs information, wherein the vital signs variable information represents the difference between the original vital signs information and the baseline vital signs information of the target object, and the baseline vital signs information represents the average value of historical vital signs information collected by the target object within a historical time period; and a second determination subunit, used to use the original vital signs information and the vital signs variable information as the vital signs information of the target object.

[0008] Optionally, in the complication prediction device provided in the present application, the acquisition unit includes: a second acquisition subunit, used to acquire the width of the target area in the medical image, wherein the target area is the outlined area of ​​the critical organ and / or target area in the medical image; a third determination subunit, used to determine the collection frequency and collection time of vital signs information according to the width of the target area; and an acquisition subunit, used to collect the vital signs information of the target object according to the collection frequency and collection time.

[0009] Optionally, in the complication prediction device provided in the present application, the second determination subunit includes: a first calculation module, used to calculate the difference between the original vital signs information and the baseline vital signs information; a second calculation module, used to calculate the ratio of the difference to the baseline vital signs information to obtain vital signs variable information.

[0010] Optionally, in the complication prediction device provided in the present application, the feature extraction unit includes: a fourth determination subunit, used to determine a first feature and a second feature based on the original vital sign information, wherein the first feature is used to characterize the trend of changes in the vital signs of the target object in at least one time period, and the second feature is used to characterize the duration of time that the indicator value of the vital sign of the target object exceeds a preset interval within a preset time; a fifth determination subunit, used to use the original vital sign information as the third feature and the vital sign variable information as the fourth feature; and a first processing subunit, used to select at least one feature from the first feature, the second feature, the third feature and the fourth feature to generate a vital sign feature.

[0011] Optionally, in the complication prediction device provided in the present application, the first processing subunit is used to perform the following steps: select any one feature from the first feature, the second feature, the third feature and the fourth feature as a vital sign feature; or, select any two features from the first feature, the second feature, the third feature and the fourth feature to fuse into a vital sign feature; or, select any three features from the first feature, the second feature, the third feature and the fourth feature to fuse into a vital sign feature; or, fuse the first feature, the second feature, the third feature and the fourth feature into a vital sign feature.

[0012] Optionally, when the first feature, the second feature, the third feature and the fourth feature are merged into a vital sign feature, the first processing subunit is used to perform the following steps: generate a first feature matrix based on the first feature, the second feature, the third feature and the first correlation coefficient, wherein the first correlation coefficient is used to characterize the correlation between the original vital sign information at different times; generate a second feature matrix based on the first feature, the second feature, the fourth feature and the second correlation coefficient, wherein the second correlation coefficient is used to characterize the correlation between the vital sign variable information at different times; splice the first feature matrix and the second feature matrix into a target feature matrix; and use the target feature matrix as the vital sign feature.

[0013] Optionally, in the complication prediction device provided in the present application, the complication determination unit includes: a feature fusion subunit, used to fuse medical image features, dose features and vital sign features into target features; a complication prediction subunit, used to input the target features into the neural network model, and determine the complication information of the target object based on the target features through the prior knowledge learned by the neural network model in the training phase, wherein the neural network model is a model trained using the medical images, radiotherapy dose information, vital sign information and known complication information of the reference object as training data.

[0014] According to another aspect of the present application, a complication prediction method is also provided, which includes: obtaining medical images, radiotherapy dose information and vital signs information of the target object; performing feature extraction on the medical images, radiotherapy dose information and vital signs information respectively to obtain medical image features, dose features and vital signs features, wherein the medical image features, dose features and vital signs features are time series features; and determining the complication information of the target object based on the medical image features, dose features and vital signs features.

[0015] According to another aspect of the present application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program runs, the device where the computer-readable storage medium is located controls the operation of any one of the above-mentioned complication prediction devices.

[0016] According to another aspect of the present application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors control the operation of any of the above-mentioned complication prediction devices.

[0017] In the present application, a complication prediction device is provided, comprising: an acquisition unit, a feature extraction unit and a complication determination unit. The acquisition unit is used to acquire the medical image, radiotherapy dose information and vital sign information of the target object; the feature extraction unit is used to extract features of the medical image, radiotherapy dose information and vital sign information respectively to obtain medical image features, dose features and vital sign features, wherein the medical image features, dose features and vital sign features belong to time series features; the complication determination unit is used to determine the complication information of the target object according to the medical image features, dose features and vital sign features.

[0018] As can be seen from the above content, the present application determines the complication information of the target object based on medical image features, dosage features and vital signs features. Compared with the prior art, which only predicts the patient's complication information based on image information and dosage information, the present application adds the patient's vital signs information. Since there are more reference dimensions, the prediction accuracy of the patient's complication information is higher. In addition, the present application also defines that the medical image features, dosage features and vital signs features are all time series features, which means that the features themselves also have time attributes, and can further improve the prediction accuracy of complications. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 is a schematic diagram of an optional complication prediction device according to an embodiment of the present application;

[0021] Figure 2 is a flowchart of an optional feature processing according to an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of an optional feature matrix corresponding to respiratory vital sign variable information according to an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of an optional feature matrix corresponding to original respiratory vital sign information according to an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of an optional feature matrix corresponding to heart rate sign variable information according to an embodiment of the present application;

[0025] Figure 6 is a schematic diagram of an optional feature matrix corresponding to original heart rate sign information according to an embodiment of the present application;

[0026] Figure 7 is a schematic diagram of an optional feature matrix corresponding to blood pressure vital sign variable information according to an embodiment of the present application;

[0027] Figure 8 is a schematic diagram of an optional feature matrix corresponding to original blood pressure sign information according to an embodiment of the present application;

[0028] Fig. 9 is a schematic diagram of an optional feature matrix corresponding to body temperature and physical sign variable information according to an embodiment of the present application;

[0029] Fig.10 is a schematic diagram of an optional feature matrix corresponding to original body temperature and physical sign information according to an embodiment of the present application;

[0030] Fig.11 It is a schematic diagram of an optional complication prediction method according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] It should also be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) collected by this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with the relevant laws, regulations and standards of the relevant regions, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation portals are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0034] According to an embodiment of the present application, an optional embodiment of a diagnosis and treatment plan generation device based on multi-source heterogeneous data is provided, wherein: Figure 1 is a schematic diagram of an optional complication prediction device according to an embodiment of the present application, such as Figure 1 As shown, the complication prediction device includes: an acquisition unit 101, a feature extraction unit 102 and a complication determination unit 103.

[0035] Optionally, the complication prediction device can be a software system, an embedded system combining software and hardware, or a device. Those skilled in the art should be aware that the present application does not specifically limit the specific form of the complication prediction device.

[0036] Optionally, in the complication prediction device, the acquisition unit 101 is used to acquire medical images, radiotherapy dose information and vital sign information of the target object.

[0037] Medical images include but are not limited to the following:

[0038] X-ray imaging: Usually used to view bone structure, it can also be used to view chest organs such as the heart and lungs.

[0039] Computed tomography (CT): A scan that uses X-rays and computer technology to create detailed cross-sectional images of the body. It is often used to examine organs, blood vessels, and tumors.

[0040] Magnetic resonance imaging (MRI): Uses a strong magnetic field and radio waves to obtain detailed images of the body's soft tissues. It is often used to examine the brain, muscles, and joints.

[0041] Positron emission tomography (PET): This technique assesses the metabolic activity of organs and tissues by detecting the distribution of radioactive tracers in the body. It is often used in cancer diagnosis and brain disease research.

[0042] Ultrasound imaging: Uses the reflection of high-frequency sound waves to produce real-time images of internal body structures. It is often used to examine the fetus, heart, blood vessels, and other internal organs.

[0043] Digital subtraction angiography (DSA): A special X-ray technique used to visualize blood vessels and blood flow.

[0044] Radiotherapy dose information includes but is not limited to the following forms of dose information:

[0045] Absolute Dose: Indicates the radiation dose at each point in the treatment area, usually in Gray (Gy).

[0046] Relative Biological Effectiveness (RBE): Considering the effects of different types of radiation on biological tissues, the dose is converted into an equivalent radiation dose.

[0047] Fractionated Dose: The total dose is divided into several treatments, and the dose of each treatment is called a fractionated dose.

[0048] Total Dose: The accumulated radiation dose during the entire treatment process.

[0049] Isodose Line: A contour line that indicates the same dose in three-dimensional space. Isodose lines are often used to evaluate the uniformity and accuracy of treatment plans.

[0050] Dose Distribution Map: A two-dimensional or three-dimensional graphic showing the spatial distribution of dose in the treatment area.

[0051] Dose-Volume Histogram (DVH): Displays tissue volume within different dose ranges and is used to evaluate the dose distribution characteristics of the treatment plan.

[0052] Maximum Dose: The highest dose received by the treatment area.

[0053] Minimum Dose: The lowest dose received by the treatment area.

[0054] Mean Dose: The average dose of all points in the treatment area.

[0055] Dose Homogeneity: Describes the consistency of dose distribution within the treatment area.

[0056] Dose Constraint: Dose limitation set to protect normal tissue or achieve specific treatment goals.

[0057] The vital signs information includes at least one of the following vital signs information:

[0058] Respiratory sign information, used to characterize the target object's respiratory information;

[0059] Heart rate sign information, used to represent the heart rate information of the target object;

[0060] Blood pressure sign information, used to represent the blood pressure information of the target object;

[0061] Body temperature and physical sign information is used to characterize the body temperature information of the target object.

[0062] It should be noted that, in addition to obtaining medical images and radiotherapy dose information, the acquisition unit 101 can also comprehensively collect key physiological data of patients during radiotherapy, providing a data basis for subsequent complication prediction. In clinical practice, doctors can use this function to monitor the physiological state of patients during radiotherapy in real time, so as to adjust the treatment plan in time and prevent the occurrence of complications.

[0063] Optionally, the complication prediction device also includes a feature extraction unit, wherein the feature extraction unit is used to extract features of medical images, radiotherapy dose information and vital signs information respectively to obtain medical image features, dose features and vital signs features, wherein the medical image features, dose features and vital signs features are time series features.

[0064] It should be noted that, through feature extraction, the complication prediction device can extract the most valuable information for complication prediction from a large amount of raw data, thereby improving the efficiency and accuracy of the prediction. In practical applications, this function can help doctors quickly identify the physiological changes that may occur in patients during radiotherapy, providing a scientific basis for clinical decision-making.

[0065] It should be noted that medical image features, dose features, and vital signs features all belong to time series features. For example, the feature extraction unit can extract corresponding medical image features from each of multiple medical images (different medical images are collected at different times), and then sort all extracted medical image features by time according to the collection time of the medical images to form a time-medical image feature sequence. For another example, the feature extraction unit can obtain dose features corresponding to different times within a preset time period, and then sort all dose features by time to form a time-dose feature sequence; the feature extraction unit can also obtain vital signs features corresponding to different times within a preset time period, and then sort all vital signs features by time to form a time-vital signs feature sequence.

[0066] Optionally, the complication prediction device further includes a complication determination unit, wherein the complication determination unit is used to determine complication information of the target object according to medical image characteristics, dose characteristics, and vital sign characteristics.

[0067] It should be noted that by determining the complication information of the target object based on medical imaging characteristics, dose characteristics and vital signs characteristics, it is possible to comprehensively analyze multi-source information and predict the complications that patients may encounter during radiotherapy. This can provide clinicians with a comprehensive assessment of the patient's status, help take preventive measures in advance, reduce the occurrence of complications, and improve treatment outcomes.

[0068] Optionally, determining the complication information of the target object according to the medical image features, the dose features, and the vital sign features can be performed through the following steps:

[0069] Step 1: The method of medical image analysis based on deep learning includes: collecting a certain amount of medical image data, including normal and abnormal images; using deep learning algorithms (such as convolutional neural networks) to extract and analyze features of medical images; training classification models based on the extracted features to classify and identify diseases in images; during the diagnosis process, inputting the medical images of the target object into the trained model to obtain information on its disease type and possible complications.

[0070] Step 2: Prediction of drug complications based on dosage characteristics, including: collecting patients' medication records and dosage information; analyzing the relationship between drug dosage and complications, and establishing a dose-complication risk model; assessing the risk of complications based on patients' medication records and dosage characteristics; and providing doctors with complication risk assessment results to provide reference for clinical decision-making.

[0071] Step 3: Analysis of vital signs characteristics based on vital signs, including: real-time monitoring of patients' blood pressure, heart rate, blood oxygen saturation and other vital signs data; analysis of the stability and variability of vital signs data, and identification of abnormal trends; and assessment of patients' possible complication risks based on abnormal trends in vital signs data.

[0072] Finally, the complication risk assessment results obtained in steps one, two, and three above are integrated. For example, assuming that the complication risk assessment results are expressed in the form of risk scores, the obtained risk scores can be weighted and the final complication prediction results of the patients can be determined by the final weighted calculation results.

[0073] Optionally, a complication risk assessment method based on multi-feature fusion can be used, which specifically includes: collecting the patient's medical images, dose characteristics, and vital signs data; extracting and analyzing the features of the medical images, dose characteristics, and vital signs data respectively; fusing the features from different sources to form a comprehensive feature set; using a machine learning algorithm (such as a support vector machine, random forest, etc.) to model the comprehensive feature set, and using the established model to predict the patient's complication risk.

[0074] Optionally, a complication association analysis method based on a knowledge graph can be used, including: constructing a medical knowledge graph that includes entities such as diseases, drugs, complications and their relationships; associating the patient's medical images, dosage characteristics and vital signs data with the entities in the knowledge graph; discovering the patient's possible complication risks through graph reasoning and path analysis; combining the association analysis results with other diagnostic information to provide doctors with more accurate complication predictions and treatment recommendations.

[0075] As can be seen from the above content, the present application determines the complication information of the target object based on medical image features, dosage features and vital signs features. Compared with the prior art, which only predicts the patient's complication information based on image information and dosage information, the present application adds the patient's vital signs information. Since there are more reference dimensions, the prediction accuracy of the patient's complication information is higher. In addition, the present application also defines that the medical image features, dosage features and vital signs features are all time series features, which means that the features themselves also have time attributes, and can further improve the prediction accuracy of complications.

[0076] In an optional embodiment, the acquisition unit 101 includes: a first acquisition subunit, a first determination subunit, and a second determination subunit.

[0077] Optionally, the first acquisition subunit is used to acquire the original vital signs information of the target object; the first determination subunit is used to determine the vital signs variable information of the target object based on the original vital signs information, wherein the vital signs variable information represents the difference between the original vital signs information and the baseline vital signs information of the target object, and the baseline vital signs information represents the average value of the historical vital signs information collected by the target object within a historical time period; the second determination subunit is used to use the original vital signs information and the vital signs variable information as the vital signs information of the target object.

[0078] Optionally, the vital signs information in the present application includes both original vital signs information and vital signs variable information. The original vital signs information can be understood as the actual collected vital signs information without any processing. The vital signs variable information is the variable information extracted based on the original vital signs information and the baseline vital signs information of the target object. The baseline vital signs information represents the average value of the historical vital signs information collected by the target object in the historical time period.

[0079] It should be noted that by acquiring the original vital signs information in real time, the complication prediction device can timely reflect the physiological changes of patients during radiotherapy and provide real-time data support for complication prediction. In clinical practice, this function can help doctors monitor the patient's vital signs in real time to ensure the patient's safety during treatment. By calculating the vital signs variable information, the device can quantify the changes in the patient's physiological state during radiotherapy and provide quantitative indicators for complication prediction. In practical applications, this function can help doctors more accurately evaluate the changes in the patient's physiological state during radiotherapy and provide a quantitative basis for clinical decision-making. By using the original vital signs information and vital signs variable information at the same time, the changes in the patient's physiological state during radiotherapy can be fully reflected, improving the accuracy and reliability of complication prediction.

[0080] In an optional embodiment, the acquisition unit 101 further includes: a second acquisition subunit, a third determination subunit, and a collection subunit.

[0081] Optionally, the second acquisition subunit is used to acquire the width of the target area in the medical image, wherein the target area is the outlined area of ​​the critical organ and / or target area in the medical image; the third determination subunit is used to determine the collection frequency and collection time of vital signs information according to the width of the target area; the collection subunit is used to collect the vital signs information of the target object according to the collection frequency and collection time.

[0082] Optionally, the width of the target area can be recorded as weight, and the collection interval of vital sign information can be recorded as T, wherein the specific calculation can refer to the following formula (1):

[0083] T=24*60 / (weight-2) (1)

[0084] The 24 in the above formula (1) can be understood as 24 hours in a day, and the 60 represents 60 minutes in an hour.

[0085] After obtaining the collection interval T, it can be determined that the collection frequency of the vital sign information is once per interval T, so that the specific collection time can be determined.

[0086] It should be noted that by obtaining the width of the target area, the device can reflect the changes in the critical organs and / or target areas during radiotherapy, and provide key medical imaging data for complication prediction. In clinical practice, this function can help doctors monitor the changes in critical organs and / or target areas in real time to ensure the accuracy and safety of radiotherapy. By determining the acquisition frequency and time, the device can ensure that the collection of vital signs information matches the radiotherapy process, and provide accurate vital signs data for complication prediction. In practical applications, this function can help doctors formulate reasonable data collection plans based on the specific conditions of the patients to ensure the accuracy and effectiveness of data collection.

[0087] In an optional embodiment, the second determining subunit includes: a first calculating module and a second calculating module.

[0088] Optionally, the first calculation module is used to calculate the difference between the original vital sign information and the reference vital sign information; the second calculation module is used to calculate the ratio of the difference to the reference vital sign information to obtain the vital sign variable information.

[0089] Optionally, assuming that the vital sign variable information is recorded as H, the original vital sign information is recorded as G, and the reference vital sign information is recorded as J, the relationship between the three can be expressed by the following formula (2).

[0090] H=(GJ) / J (2)

[0091] It should be noted that by calculating the difference between the original vital signs information and the baseline vital signs information, the changes in the patient's physiological state during radiotherapy can be intuitively reflected, providing intuitive indicators for complication prediction. In clinical practice, this function can help doctors quickly identify changes in the patient's physiological state during radiotherapy and provide an intuitive basis for clinical decision-making. By calculating the ratio of the difference to the baseline vital signs information, the vital signs variable information is obtained. It can more accurately reflect the changes in the patient's physiological state during radiotherapy and provide more accurate quantitative indicators for complication prediction. In practical applications, this function can help doctors more accurately evaluate the changes in the patient's physiological state during radiotherapy and provide more accurate quantitative basis for clinical decision-making.

[0092] In an optional embodiment, the feature extraction unit 102 includes: a fourth determination subunit, a fifth determination subunit, and a first processing subunit.

[0093] Optionally, the fourth determination subunit is used to determine the first feature and the second feature based on the original vital signs information, wherein the first feature is used to characterize the vital signs change trend of the target object in at least one time period, and the second feature is used to characterize the duration that the indicator value of the vital signs of the target object exceeds a preset interval within a preset time; the fifth determination subunit is used to use the original vital signs information as the third feature and the vital signs variable information as the fourth feature; the first processing subunit is used to select at least one feature from the first feature, the second feature, the third feature and the fourth feature to generate a vital signs feature.

[0094] Optionally, the features extracted by the feature extraction unit 102 include at least the following features:

[0095] The first feature is used to characterize the trend of changes in the vital signs of the target object in at least one time period (for example, the breathing rate of the target object continues to increase for 1 hour in a preset time period, or the heart rate of the target object continues to increase for 2 hours in a preset time period).

[0096] The second feature is used to characterize the duration that the indicator value of the target object's vital signs exceeds a preset interval (for example, the total duration that the target object's breathing rate is not within the normal breathing rate range, or the total duration that the target object's heart rate is not within the normal heart rate range).

[0097] The third feature is the original vital signs information itself.

[0098] The fourth feature is the vital sign variable information itself.

[0099] It should be noted that by determining the first feature and the second feature, the device can reflect the changes in the patient's vital signs during radiotherapy from different angles, providing multi-dimensional data support for complication prediction. In addition, the first processing subunit is used to select at least one feature from the first feature, the second feature, the third feature, and the fourth feature to generate a vital sign feature. This design enables the device to flexibly select vital sign features according to actual needs, thereby improving the flexibility and adaptability of complication prediction.

[0100] In an optional embodiment, the first processing subunit is used to perform the following steps:

[0101] Select any one feature from the first feature, the second feature, the third feature and the fourth feature as the vital sign feature; or select any two features from the first feature, the second feature, the third feature and the fourth feature to merge into the vital sign feature; or select any three features from the first feature, the second feature, the third feature and the fourth feature to merge into the vital sign feature; or merge the first feature, the second feature, the third feature and the fourth feature into the vital sign feature.

[0102] It should be noted that any one of the first feature, the second feature, the third feature and the fourth feature can be selected as a vital sign feature, and the vital sign feature can be flexibly selected according to actual needs, thereby improving the flexibility and adaptability of complication prediction. In clinical practice, doctors can select the most relevant vital sign feature for complication prediction according to the specific situation of the patient, thereby improving the pertinence and effectiveness of the prediction.

[0103] By selecting any two features from the first feature, the second feature, the third feature and the fourth feature and fusing them into vital sign features, the changes in the patient's vital signs during radiotherapy can be reflected from different angles, thereby improving the accuracy and reliability of complication prediction.

[0104] Selecting any three features from the first, second, third and fourth features to fuse as vital sign features can more comprehensively reflect the changes in the patient's vital signs during radiotherapy, improving the accuracy and reliability of complication prediction. In clinical practice, this function can help doctors more comprehensively evaluate the changes in patients' vital signs and provide more comprehensive and accurate data support for clinical decision-making.

[0105] By integrating the first, second, third and fourth features into the vital sign feature, the device can most comprehensively reflect the changes in the patient's vital signs during radiotherapy and provide the most comprehensive data support for complication prediction. In clinical practice, this function can help doctors most comprehensively evaluate the changes in the patient's vital signs and provide the most comprehensive and accurate data support for clinical decision-making.

[0106] In an optional embodiment, when the first feature, the second feature, the third feature and the fourth feature are merged into a vital sign feature, the first processing subunit is configured to perform the following steps:

[0107] A first feature matrix is ​​generated according to the first feature, the second feature, the third feature and the first correlation coefficient, wherein the first correlation coefficient is used to characterize the correlation between the original vital sign information at different times; a second feature matrix is ​​generated according to the first feature, the second feature, the fourth feature and the second correlation coefficient, wherein the second correlation coefficient is used to characterize the correlation between the vital sign variable information at different times; the first feature matrix and the second feature matrix are concatenated into a target feature matrix; and the target feature matrix is ​​used as the vital sign feature.

[0108] Optionally, when the first feature, the second feature, the third feature and the fourth feature are merged into a vital sign feature, two feature matrices may be generated first, namely a first feature matrix and a second feature matrix. The first feature matrix includes at least: a first feature, a second feature, a third feature and a first correlation coefficient, and the first correlation coefficient is used to characterize the correlation between the original vital sign information at different times. The second feature matrix includes at least: a first feature, a second feature, a fourth feature and a second correlation coefficient, and the second correlation coefficient is used to characterize the correlation between the vital sign variable information at different times.

[0109] Finally, the first feature matrix and the second feature matrix can be spliced ​​into a target feature matrix by splicing.

[0110] It should be noted that by generating the first feature matrix and the second feature matrix, the trend and correlation of the patient's vital signs during radiotherapy can be reflected, and the accuracy and reliability of complication prediction can be improved. The first feature matrix and the second feature matrix are spliced ​​into the target feature matrix, which can fully reflect the changes in the patient's vital signs during radiotherapy and improve the accuracy and reliability of complication prediction.

[0111] In an optional embodiment, the complication determination unit 103 includes: a feature fusion subunit and a complication prediction subunit.

[0112] Optionally, a feature fusion subunit is used to fuse medical image features, dose features and vital signs features into target features; a complication prediction subunit is used to input the target features into the neural network model, and determine the complication information of the target object based on the target features through the prior knowledge learned by the neural network model during the training phase, wherein the neural network model is a model trained using the medical images, radiotherapy dose information, vital signs information and known complication information of the reference object as training data.

[0113] Optionally, after obtaining the medical image features, dose features and vital sign features, the complication prediction device can use a series of feature fusion methods such as feature vector splicing to fuse the three features into target features, and then use a neural network model to use the prior knowledge learned by the neural network model in the training phase to determine the complication information of the target object based on the target features.

[0114] Optionally, the training process of the neural network model includes the following steps:

[0115] First, the patient’s medical imaging, dose characteristics, and vital signs data can be collected;

[0116] Then, the features of medical images, dose characteristics, and vital signs data were extracted and analyzed respectively;

[0117] The features from different sources are then fused to form a comprehensive feature set;

[0118] Establishing a training label for the patient, wherein the training label represents complication information of the patient;

[0119] A deep neural network learning algorithm is used to train the model on the comprehensive feature set and training labels, and the trained neural network model is used to predict the risk of complications of the target object.

[0120] It should be noted that by using a neural network model to predict complications, the complication prediction device can learn the correlation between changes in the patient's physiological state during radiotherapy and complications based on a large amount of historical data, thereby improving the accuracy and reliability of the prediction.

[0121] According to another aspect of the present application, a complication prediction method is also provided, including: obtaining medical images, radiotherapy dose information and vital signs information of the target object; performing feature extraction on the medical images, radiotherapy dose information and vital signs information respectively to obtain medical image features, dose features and vital signs features, wherein the medical image features, dose features and vital signs features are time series features; and determining the complication information of the target object based on the medical image features, dose features and vital signs features.

[0122] In an optional embodiment, Figure 2 is a flowchart of an optional feature processing according to an embodiment of the present application, such as Figure 2 As shown, the following steps are included:

[0123] Step S201, obtaining medical images, radiotherapy dose information and vital sign information of the target object.

[0124] The vital signs information includes, but is not limited to, respiratory signs information, blood pressure signs information, body temperature signs information, and heart rate signs information.

[0125] Step S202, processing the respiratory vital sign information.

[0126] Optionally, firstly, the average value of respiratory vital signs information collected by the target object in a historical time period (e.g., the first day of hospitalization) is used as the baseline respiratory vital signs information. Then, the width Weight of the delineated area for the critical organ and / or target area in the medical image is calculated. Then, the sampling period T=24*60 / (weight-2) is calculated.

[0127] Optionally, the respiratory sign variable information is calculated once every T time interval, wherein the respiratory sign variable information=(original respiratory sign information-reference respiratory sign information) / reference respiratory sign information.

[0128] Optionally, a hyperparameter related to the respiratory sign information is determined through a time sliding window, including: taking an increase or decrease in respiratory rate for X hours in M ​​hours as a binary variable for the day, wherein M and X are both values ​​greater than 0, and X is less than or equal to M.

[0129] For example: set a label for continuous increased breathing (characterizing that within 24 hours, the patient has a situation of increased breathing rate for X hours). Set a label for continuous slowed breathing (characterizing that within 24 hours, the patient has a situation of slowed breathing rate for X hours).

[0130] It should be noted that the above labels are not diagnostic results, but are only intermediate variables required for model training. Their advantage is that they are easier for clinicians to understand.

[0131] Optionally, the total duration during which the target subject's breathing rate is not within the normal range may also be calculated.

[0132] Optionally, Figure 3 An optional feature matrix corresponding to respiratory vital signs variable information is shown, including: respiratory variation value (i.e., respiratory vital signs variable information), correlation coefficient (used to characterize the correlation between respiratory vital signs variable information at different times), label of fast breathing / slow breathing, abnormal duration (i.e., the total duration that is not within the normal respiratory frequency range). The length of the rows and columns of the feature matrix are both the sampling period T=24*60 / (weight-2).

[0133] Optionally, Figure 4An optional feature matrix corresponding to the original respiratory vital signs information is shown, including: respiratory frequency (i.e., original respiratory vital signs variable information), correlation coefficient (used to characterize the association between the original respiratory vital signs information at different times), labels of fast breathing / slow breathing, and abnormal duration (i.e., the total time that is not within the normal respiratory frequency range).

[0134] Step S203, processing the heart rate and vital signs information.

[0135] Optionally, firstly, an average value of the heart rate sign information collected from the target object in a historical time period is taken as the baseline heart rate sign information.

[0136] Optionally, the heart rate sign variable information is calculated once every T time interval, wherein the heart rate sign variable information = (original heart rate sign information - reference heart rate sign information) / reference heart rate sign information.

[0137] Optionally, the hyperparameters related to the heart rate sign information are determined through a time sliding window, including: taking an increase or decrease in the heart rate frequency for X hours in M ​​hours as a binary variable for the day, wherein M and X are both values ​​greater than 0, and X is less than or equal to M.

[0138] For example, set a label for continuous increased heart rate (indicating that within M hours, the patient has a continuous increased heart rate for X hours). Set a label for continuous decreased heart rate (indicating that within M hours, the patient has a continuous decreased heart rate for X hours).

[0139] It should be noted that the above labels are not diagnostic results, but are only intermediate variables required for model training. Their advantage is that they are easier for clinicians to understand.

[0140] Optionally, the total duration during which the target object is not within the normal heart rate frequency range may also be calculated.

[0141] Optionally, Figure 5 An optional feature matrix corresponding to the heart rate sign variable information is shown, including: heart rate variability (i.e., heart rate sign variable information), correlation coefficient (used to characterize the correlation between heart rate sign variable information at different times), labels of too fast heart rate / too slow heart rate, and abnormal duration (i.e., the total duration that is not within the normal heart rate frequency range). The length of the rows and columns of the feature matrix are both the sampling period T=24*60 / (weight-2).

[0142] Optionally, Figure 6An optional feature matrix corresponding to the original heart rate sign information is shown, including: heart rate frequency (i.e., original heart rate sign variable information), correlation coefficient (used to characterize the association between the original heart rate sign information at different times), labels for too fast heart rate / too slow heart rate, and abnormal duration (i.e., the total time that is not within the normal heart rate frequency range).

[0143] Step S204, processing the blood pressure vital sign information.

[0144] Optionally, firstly, the average value of the blood pressure (including systolic pressure and diastolic pressure) physical sign information collected from the target object in a historical time period is taken as the baseline blood pressure physical sign information.

[0145] Optionally, the blood pressure sign variable information is calculated once every T time interval, wherein the blood pressure sign variable information = (original blood pressure sign information - baseline blood pressure sign information) / baseline blood pressure sign information.

[0146] Optionally, the hyperparameters related to the blood pressure sign information are determined through a time sliding window, including: taking the blood pressure increase or decrease for X hours in M ​​hours as a binary variable for the day, wherein M and X are both values ​​greater than 0, and X is less than or equal to M.

[0147] For example: set a label for continuous high blood pressure (indicating that within M hours, the patient has continued to have high blood pressure for X hours). Set a label for continuous low blood pressure (indicating that within M hours, the patient has continued to have low blood pressure for X hours).

[0148] It should be noted that the above labels are not diagnostic results, but are only intermediate variables required for model training. Their advantage is that they are easier for clinicians to understand.

[0149] Optionally, the total time that the target object is not within the normal blood pressure range may also be calculated.

[0150] Optionally, Figure 7 An optional feature matrix corresponding to blood pressure sign variable information is shown, including: systolic pressure variation value and diastolic pressure variation value (i.e., blood pressure sign variable information), correlation coefficient (used to characterize the association between blood pressure sign variable information at different times), high systolic pressure / low systolic pressure labels (or high diastolic pressure / low diastolic pressure labels), and abnormal duration (i.e., the total time that is not within the normal blood pressure range).

[0151] Optionally, Figure 8An optional feature matrix corresponding to the original blood pressure sign information is shown, including: systolic pressure and diastolic pressure (i.e., original blood pressure sign information), correlation coefficient (used to characterize the association between the original blood pressure sign information at different times), high systolic pressure / low systolic pressure labels (or high diastolic pressure / low diastolic pressure labels), and abnormal duration (i.e., the total time that is not within the normal blood pressure range).

[0152] Step S205, processing the body temperature and vital sign information.

[0153] Optionally, firstly, an average value of the body temperature and physical sign information collected from the target object in a historical time period is taken as the baseline body temperature and physical sign information.

[0154] Optionally, the body temperature and physical sign variable information is calculated once every T time interval, wherein the body temperature and physical sign variable information = (original body temperature and physical sign information - baseline body temperature and physical sign information) / baseline body temperature and physical sign information.

[0155] Optionally, hyperparameters related to body temperature and physical sign information are determined through a time sliding window, including: taking an increase or decrease in body temperature for X hours in M ​​hours as a binary variable for the day, wherein M and X are both values ​​greater than 0, and X is less than or equal to M.

[0156] For example: set a label for continuous temperature rise (indicating that within 24 hours, the patient has a continuous temperature rise for X hours). Set a label for continuous temperature reduction (indicating that within 24 hours, the patient has a continuous temperature reduction for X hours).

[0157] It should be noted that the above labels are not diagnostic results, but are only intermediate variables required for model training. Their advantage is that they are easier for clinicians to understand.

[0158] Optionally, the total time that the target object is not within the normal body temperature range may also be calculated.

[0159] Optionally, Fig. 9 An optional feature matrix corresponding to body temperature and physical sign variable information is shown, including: body temperature variation value (i.e. body temperature and physical sign variable information), correlation coefficient (used to characterize the association between body temperature and physical sign variable information at different times), fever / hypothermia label, abnormal duration (i.e. the total time that is not within the normal body temperature range).

[0160] Optionally, Fig.10 An optional feature matrix corresponding to the original body temperature and physical sign information is shown, including: body temperature (i.e., original body temperature and physical sign information), correlation coefficient (used to characterize the association between the original body temperature and physical sign information at different times), fever / hypothermia labels, and abnormal duration (i.e., the total time that is not within the normal body temperature range).

[0161] In an optional embodiment, Fig.11 is a schematic diagram of an optional complication prediction method according to an embodiment of the present application, such as Fig.11 As shown, firstly, the vital sign variable information is determined based on the baseline vital sign information of each vital sign (including heart rate, blood pressure, body temperature, and respiration), and then the abnormal duration and sliding window label of each vital sign are determined. Then, the feature matrix corresponding to the original vital sign information of each vital sign and the feature matrix corresponding to the vital sign variable information are determined. Finally, the medical image features, radiotherapy dose features, feature matrices corresponding to the original vital sign information of each vital sign, and feature matrices corresponding to the vital sign variable information of each vital sign of the target object are feature fused, and the feature fusion results are input into the neural network model, and the neural network model predicts and outputs the complication information of the target object.

[0162] In summary, the complication prediction device or method proposed in this application can comprehensively evaluate the patient's physiological state during radiotherapy and predict possible complications by integrating medical images, radiotherapy doses, and vital signs information. In practical applications, this device can significantly improve the safety and personalization of radiotherapy and provide strong support for precision medicine. Doctors can use this device to monitor the patient's physiological state during radiotherapy in real time, adjust the treatment plan in a timely manner, reduce the occurrence of complications, and improve the treatment effect and the patient's quality of life.

[0163] According to another aspect of the present application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program runs, the device where the computer-readable storage medium is located controls the operation of any one of the above-mentioned complication prediction devices.

[0164] According to another aspect of the present application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors control the operation of any of the above-mentioned complication prediction devices.

[0165] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0166] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

[0167] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0168] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0170] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0171] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0172] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0173] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A complication prediction device, characterized in that: include: An acquisition unit, used to acquire medical images, radiotherapy dose information, and vital signs information of a target object; a feature extraction unit, used to extract features from the medical image, the radiotherapy dose information and the vital sign information respectively, to obtain medical image features, dose features and vital sign features, wherein the medical image features, the dose features and the vital sign features are time series features; a complication determination unit, configured to determine complication information of the target object according to the medical image characteristics, the dose characteristics, and the vital sign characteristics; Wherein, the acquisition unit includes: a first acquisition subunit, used to acquire the original vital sign information of the target object; a first determination subunit, used to determine the vital sign variable information of the target object according to the original vital sign information, wherein the vital sign variable information represents the difference between the original vital sign information and the baseline vital sign information of the target object, and the baseline vital sign information represents the average value of the historical vital sign information collected by the target object within a historical time period; a second determination subunit, used to use the original vital sign information and the vital sign variable information as the vital sign information of the target object; Wherein, the complication determination unit includes: a feature fusion subunit, used for fusing the medical image feature, the dose feature and the vital sign feature into a target feature; The complication prediction subunit is used to input the target feature into the neural network model, and determine the complication information of the target object based on the target feature through the prior knowledge learned by the neural network model in the training stage, wherein the neural network model is a model trained using the medical images, radiotherapy dose information, vital signs information of the reference object and the known complication information of the reference object as training data.

2. The complication prediction device according to claim 1, characterized in that: The acquisition unit comprises: A second acquisition subunit is used to acquire a width of a target area in the medical image, wherein the target area is a delineated area of ​​a critical organ and / or a target area in the medical image; A third determining subunit is used to determine the collection frequency and collection time of the vital sign information according to the width of the target area; The collection subunit is used to collect the vital sign information of the target object according to the collection frequency and collection time.

3. The complication prediction device according to claim 1, characterized in that: The second determining subunit includes: A first calculation module, used for calculating the difference between the original vital sign information and the reference vital sign information; The second calculation module is used to calculate the ratio of the difference to the reference vital sign information to obtain the vital sign variable information.

4. The complication prediction device according to claim 1, characterized in that: The feature extraction unit comprises: a fourth determination subunit, configured to determine a first feature and a second feature based on the original vital sign information, wherein the first feature is used to characterize a vital sign change trend of the target object within at least one time period, and the second feature is used to characterize a duration during which an indicator value of the vital sign of the target object exceeds a preset interval within a preset time period; a fifth determining subunit, configured to use the original vital sign information as a third feature and the vital sign variable information as a fourth feature; The first processing subunit is used to select at least one feature from the first feature, the second feature, the third feature and the fourth feature to generate the vital sign feature.

5. The complication prediction device according to claim 4, characterized in that: The first processing subunit is used to perform the following steps: Select any one of the first feature, the second feature, the third feature and the fourth feature as the vital sign feature; or, Select any two features from the first feature, the second feature, the third feature and the fourth feature to merge into the vital sign feature; or, Select any three features from the first feature, the second feature, the third feature and the fourth feature to be merged into the vital sign feature; or, The first feature, the second feature, the third feature and the fourth feature are merged into the vital sign feature.

6. The complication prediction device according to claim 5, characterized in that: When the first feature, the second feature, the third feature, and the fourth feature are merged into the vital sign feature, the first processing subunit is used to perform the following steps: A first feature matrix is ​​generated according to the first feature, the second feature, the third feature and the first correlation coefficient, wherein the first correlation coefficient is used to characterize the correlation between the original vital sign information at different times; a second feature matrix is ​​generated according to the first feature, the second feature, the fourth feature and the second correlation coefficient, wherein the second correlation coefficient is used to characterize the correlation between the vital sign variable information at different times; Concatenate the first feature matrix and the second feature matrix into a target feature matrix; The target feature matrix is ​​used as the vital sign feature.

7. A complication prediction method, characterized in that: include: Obtain medical images, radiotherapy dose information, and vital signs information of the target object; Extracting features of the medical image, the radiotherapy dose information and the vital sign information respectively to obtain medical image features, dose features and vital sign features, wherein the medical image features, the dose features and the vital sign features are time series features; Determining complication information of the target object according to the medical image characteristics, the dose characteristics, and the vital sign characteristics; Wherein, obtaining the vital sign information of the target object includes: obtaining the original vital sign information of the target object; determining the vital sign variable information of the target object according to the original vital sign information, wherein the vital sign variable information represents the difference between the original vital sign information and the baseline vital sign information of the target object, and the baseline vital sign information represents the average value of the historical vital sign information collected by the target object in a historical time period; using the original vital sign information and the vital sign variable information as the vital sign information of the target object; Wherein, determining the complication information of the target object according to the medical image features, the dose features and the vital signs features includes: fusing the medical image features, the dose features and the vital signs features into target features; inputting the target features into a neural network model, and determining the complication information of the target object based on the target features through the prior knowledge learned by the neural network model in a training phase, wherein the neural network model is a model trained using the medical images, radiotherapy dose information, vital signs information of a reference object and the known complication information of the reference object as training data.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program runs, the device where the computer-readable storage medium is located controls the operation of the complication prediction device described in any one of claims 1 to 6.

9. An electronic device, characterized in that: It comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors control the operation of the complication prediction device described in any one of claims 1 to 6.

Images