Pre-operative weak state automatic evaluation method based on deep learning
Through the automated evaluation method of preoperative weakness status based on deep learning, the problem of inaccurate preoperative evaluation in the existing technology is solved, accurate assessment and risk prediction of esophageal cancer patients are achieved, and the scientificity and safety of the surgery are improved.
Patent Information
- Application Number
- CN202411002414.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the prior art, the preoperative evaluation method is not accurate enough to evaluate patients with esophageal cancer and cannot effectively obtain preoperative status analysis methods, resulting in low accuracy and reliability of predetermined surgical plans and risk assessment results, making it difficult to provide effective decision support. .
Using deep learning-based automated assessment method for preoperative weakness status, we use deep learning models to calculate the degree of weakness and weakness level of patients, and use clinical risk factors to predict postoperative complications Probability, draw a histogram of risk prediction.
The quantification and standardization of preoperative evaluation has been achieved, and the evaluation results are more scientific and reliable, providing important decision-making references for formulating personalized surgical plans and managing postoperative risks, and improving surgical effectiveness and safety.
Smart Images

Figure CN119028565A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of preoperative status assessment, and more specifically, to an automated preoperative frailty status assessment method based on deep learning. Background Art
[0002] In recent years, preoperative assessment of the patient's physiological state and weakness is crucial for developing an individualized treatment plan and reducing surgical risks. Traditional preoperative evaluation mainly relies on the doctor's clinical experience for subjective judgment, which has a certain degree of randomness and subjectivity, and may lead to inconsistency and deviation in the evaluation results. In the application of multivariate information analysis and processing, deep learning can automatically learn high-level abstract representations or features in multivariate information without the need for manual feature design, and can learn nonlinear relationships in multivariate information. At the same time, it has strong feature expression capabilities and can simultaneously learn spatial and temporal regular dependencies in multivariate information. Therefore, applying multivariate information analysis based on deep learning to preoperative patient status analysis can integrate multi-source clinical data of patients, such as symptoms, images, physiological parameters, etc., and extract complex patient feature representations. It can also identify the high-level rules implicit in the data, grasp the patient's comprehensive status, and provide a basis for accurate assessment of the condition and preoperative diagnosis. In addition, it can identify abnormal and dangerous symptoms in the data, provide reference for clinical decision-making, and improve the level of preoperative management.
[0003] The patent with publication number CN112992355A discloses a preoperative evaluation method and system for esophageal cancer patients; including: obtaining first basic information of a first user; obtaining first basic condition information based on the first basic information; obtaining a first image set of the first user and obtaining a first analysis instruction; analyzing the first image set according to the first analysis instruction to obtain a first analysis result, wherein the first analysis result includes the real-time status information of the first user; integrating the first basic condition information of the first user and the real-time status information to obtain a first integrated assessment result of the first user; obtaining a first predetermined surgical plan of the first user through the preoperative evaluation system; obtaining a first risk assessment result based on the first integrated assessment result and the first predetermined surgical plan; solving the technical problem of inaccurate preoperative evaluation of esophageal cancer patients in the prior art.
[0004] However, the above technology does not explain how to analyze the first image set to obtain the first analysis result, and it is difficult to understand the specific acquisition process of the first integrated assessment result, that is, it is impossible to know the specific preoperative state analysis method; on this basis, the first predetermined surgical plan obtained according to the first integrated assessment result, and the first risk assessment result obtained according to the first integrated assessment result and the first predetermined surgical plan, their accuracy cannot be guaranteed, the reliability is low, and it is difficult to provide effective decision support;
[0005] In view of this, the present invention proposes an automated assessment method for preoperative frailty status based on deep learning to solve the above problems. Summary of the invention
[0006] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for automatic assessment of preoperative frailty state based on deep learning, comprising:
[0007] S1: Collect patients’ electronic medical record data;
[0008] S2: pre-process the electronic medical record data and mark it as processed medical record data;
[0009] S3: Analyze electronic medical record data to obtain imaging diagnosis results;
[0010] S4: Collecting the patient's real-time status data, which includes the patient's facial image and missing data;
[0011] S5: Analyze the patient's first level of weakness by combining the medical record data, imaging diagnosis results and missing data; analyze the patient's second level of weakness by using the patient's facial image;
[0012] S6: Calculate the total weakness degree by combining the patient's first weakness degree and second weakness degree, and analyze the corresponding weakness level;
[0013] S7: Obtain clinical risk factors, predict the probability of postoperative complications in patients based on clinical risk factors and total frailty level, and draw a risk prediction histogram.
[0014] Furthermore, the electronic medical record data includes basic information, physical information, laboratory test information, imaging test information, medication history, surgical history and current symptoms; the patient is a preoperative patient;
[0015] The physical information includes appearance structure and physiological characteristics; the appearance structure includes height and weight; the physiological characteristics include blood pressure and heart rate;
[0016] The laboratory examination information is the examination results obtained after the patient undergoes medical laboratory examination; the imaging examination information is the examination results obtained after the patient undergoes medical imaging examination; the imaging examination information at least includes X-rays, CT scans, magnetic resonance imaging and ultrasound images;
[0017] The medication history is a patient's historical medication record, which includes the name of the drug and the dosage of the drug;
[0018] The surgical history is a patient's historical surgical record, and the surgical history includes the name of the surgery and the date of the surgery;
[0019] The current symptoms are symptoms perceived by the patient, and the current symptoms include symptoms and onset time.
[0020] Furthermore, the step of preprocessing the electronic medical record data includes:
[0021] Step a1: Clean the electronic medical record data, delete the erroneous data in the electronic medical record data, and mark the deleted data as missing data;
[0022] Step a2: Convert the categorical variables in the electronic medical record data into one-hot encoding;
[0023] Categorical variables are feature variables whose value type is classification or category;
[0024] One-hot encoding is an encoding method that converts categorical variables into numerical values.
[0025] Furthermore, the method for analyzing electronic medical record data includes:
[0026] Use the trained result diagnosis model to identify the imaging examination information in turn and output the corresponding recognition results, which include primary weakness, intermediate weakness and advanced weakness;
[0027] The results of the diagnostic model training process include:
[0028] Collect a images of various types from imaging examination information in advance and mark them as training images, where a is an integer greater than 1; annotate each training image, including primary weakness, intermediate weakness and advanced weakness; convert primary weakness, intermediate weakness and advanced weakness into digital annotations respectively; divide the annotated training images into a training set and a test set; use the training set to train the result diagnosis model, and use the test set to test the result diagnosis model; preset an error threshold, and when the mean of the prediction errors of all training images in the test set is less than the error threshold, output the result diagnosis model; the result diagnosis model is a convolutional neural network model.
[0029] Furthermore, the method for obtaining imaging diagnosis results includes:
[0030] According to the output recognition results, count the number of each recognition result and the total number of recognition results;
[0031] Divide the number of each recognition result by the total number of recognition results to obtain the proportion of each recognition result, that is, to obtain the primary weakness proportion ZB q , Intermediate Weakness Percentage ZB d And the proportion of advanced weakness ZB z ;
[0032] like Then the imaging diagnosis result is advanced frailty;
[0033] like and Then the imaging diagnosis result is intermediate frailty;
[0034] like and as well as The imaging diagnosis result is primary frailty.
[0035] Further, the method for analyzing the first weakness level of the patient includes:
[0036] According to the imaging diagnosis results, the corresponding diagnostic value is obtained; the medical record data, the diagnostic value and the missing data are processed as analysis data; the analysis data is input into the trained degree analysis model to predict the corresponding first frailty degree;
[0037] The training process of the degree analysis model includes:
[0038] Pre-collecting the first frailty levels corresponding to b groups of analysis data, converting the analysis data and the corresponding first frailty levels into a corresponding set of feature vectors, where b is an integer greater than 1;
[0039] Each group of feature vectors is used as the input of the degree analysis model, and the degree analysis model takes a group of predicted first frailty degrees corresponding to each group of analysis data as output, and takes the actual first frailty degree corresponding to each group of analysis data as the prediction target, and the actual first frailty degree is the first frailty degree corresponding to the analysis data collected in advance; minimizing the sum of prediction errors of all analysis data is used as the training goal; the degree analysis model is trained until the sum of prediction errors reaches convergence and the training is stopped; the degree analysis model is a deep neural network model.
[0040] Further, the step of analyzing the second weakness level of the patient includes:
[0041] Step b1: Using Viola-Jones face detection, extract the entire facial area from the patient's facial image;
[0042] Step b2: using the active shape model to extract facial key points in the facial area and the coordinates corresponding to the facial key points, the facial key points include upper eye corner points, lower eye corner points, left mouth corner points and right mouth corner points;
[0043] Step b3: Calculate the eye height according to the coordinates of the upper eye corner point and the lower eye corner point, and calculate the lip width according to the coordinates of the left mouth corner point and the right mouth corner point;
[0044] The expression for eye height is:
[0045] Where H' is the eye height, x 2 is the horizontal coordinate of the upper eye corner point, x 1 is the horizontal coordinate of the lower eye corner point, y 2 is the ordinate of the upper eye corner point, y 1 is the ordinate of the lower eye corner point;
[0046] The expression for lip width is:
[0047] Where D is the lip width, x 4 is the horizontal coordinate of the right corner of the mouth, x 3 is the horizontal coordinate of the left corner of the mouth, y 4 is the ordinate of the right corner of the mouth, y 3 is the ordinate of the left mouth corner point;
[0048] Step b4: graying the patient's facial image and marking it as a gray image; segmenting the skin area in the gray image using a region segmentation algorithm;
[0049] Step b5: Sampling the skin area, obtaining the grayscale value corresponding to each pixel, and constructing a grayscale value matrix G;
[0050] Step b6: Preset the construction parameters of the gray-level co-occurrence matrix GLCM, which include direction and distance; according to the construction parameters of the gray-level co-occurrence matrix GLCM, traverse each gray-level value in the gray-level value matrix G to obtain the gray-level value combination of each pair of adjacent pixels; count the occurrence frequency of each gray-level value combination, which is the frequency of occurrence of the gray-level value combination in the skin area, and add it to the corresponding position of the gray-level co-occurrence matrix GLCM;
[0051] Step b7: According to the gray level co-occurrence matrix GLCM, the contrast of the gray level co-occurrence matrix GLCM is calculated and marked as skin roughness;
[0052] The contrast expression of the gray level co-occurrence matrix GLCM is:
[0053] In the formula, DB is the contrast, m is i is a gray value in the i-th gray value combination, n i is another gray value in the i-th gray value combination, S i is the occurrence frequency of the i-th gray value combination, C is the number of gray value combinations, i∈[1,C];
[0054] Step b8: Use eye height, lip width and skin roughness as test data, input the test data into the trained degree prediction model, and predict the corresponding second degree of weakness; wherein the training process of the degree prediction model is consistent with the training process of the degree analysis model, and is also a deep neural network model.
[0055] Furthermore, the expression of total weakness is: R z =ω 1 ×R 1 +ω 2 ×R 2 ;
[0056] In the formula, R z is the overall weakness, R 1 The first level of weakness, R 2 The second level of weakness, ω 1 ,ω 2 All are weight coefficients;
[0057] The range of total weakness is [M 1 , M 2 ], where M 1 、M 2 are all integers greater than 0 and M 2 >M 1 ; The frailty levels include mild frailty, moderate frailty, and severe frailty;
[0058] If the calculated total frailty level is If the patient is within 100 μg / cm2, the weakness level is mild weakness;
[0059] If the calculated total frailty level is If the frailty level is within 100%, the frailty level is moderate;
[0060] If the calculated total frailty level is If the value is within 30%, the weakness level is severe weakness.
[0061] Furthermore, the clinical risk factors include the type of surgery, surgical site, and anesthesia method;
[0062] Different numbers are set for different types of surgery, surgical sites and anesthesia methods; the types of surgery, surgical sites and anesthesia methods in clinical risk factors are converted into corresponding numbers;
[0063] Clinical risk factors and total frailty level are used as evaluation data, and the evaluation data are respectively input into T trained probability prediction models to predict the probability of occurrence of different postoperative complications of patients; one probability prediction model corresponds to one postoperative complication, and T is an integer greater than 1; based on the predicted probability of occurrence of different postoperative complications of patients, a risk prediction histogram is drawn, and the risk prediction histogram includes the probability of occurrence of each postoperative complication;
[0064] Furthermore, the training process of the probability prediction model includes:
[0065] The probability prediction model uses a random forest regression model. First, E groups of historical data are collected in advance. The historical data includes evaluation data and the probability of postoperative complications corresponding to the evaluation data. N tree subsets are sampled from the E groups of historical data. Each tree subset includes e groups of historical data, where e is any value in (1, E]. Each decision tree is trained based on the N tree subsets. Here, both E and N are integers greater than 1, and N < E.
[0066] Initialize the model parameters, including determining the number of decision trees N in the forest, the maximum depth B of each decision tree, the minimum number of samples F for internal nodes, and the minimum number of samples Q for leaf nodes.
[0067] Use the CART algorithm to recursively construct the first decision tree:
[0068] Divide the e groups of historical data in a tree subset into a training set α and a validation set β. α r is the r-th group of historical data in the training set α, r ∈ [1, α'], α' < e, and α' is the number of historical data in the training set α; use the feature t h to represent one of the surgical type number, surgical site number, anesthesia method number, and overall frailty level. Calculate the information gain ratio Z(α, t h ) of the h-th feature t h ). The calculation formula of Z(α, t h ) is as follows:
[0069]
[0070] In the formula, h ∈ [1, 4], af is the decision subset divided by the training set α according to the feature t h , f ∈ [1, 4], H(af) is the information entropy of af, and H(α) is the information entropy of the training set α. The calculation methods of H(α) and H(af) respectively include:
[0071]
[0072]
[0073] In the formula, αf r represents the probability of postoperative complications corresponding to the r-th group of historical data in αf;
[0074] Calculate the information gain ratio Z(α, t h ) corresponding to each feature t h . Select the feature t h corresponding to the maximum information gain ratio Z(α, t h ) as the node, and according to the feature t h corresponding to the maximum information gain ratio Z(α, t hDivide the training set α into decision subsets, each with the same number of historical data;
[0075] For each maximum information gain rate Z(α, t h ) and repeat the algorithm recursive process until the decision subset cannot be divided into smaller decision subsets; the performance of the trained decision tree model is evaluated on the validation set β, and finally the construction of the first decision tree model is completed;
[0076] The remaining N-1 tree subsets are used in turn to train N-1 decision trees; the generation process of each decision tree is independent and based on the CART algorithm; finally, a random forest containing N decision trees is constructed;
[0077] Calculate the loss function value on N decision trees; input the training data into each decision tree and record the decision output, which is the probability of postoperative complications corresponding to the evaluation data; calculate the mean square loss of the error between the average output of N decision trees and the true value; reduce the mean square loss through parameter tuning, and select the model parameters with the smallest mean square loss as the final model.
[0078] Technical effects and advantages of the deep learning-based automated preoperative frailty assessment method of the present invention:
[0079] This embodiment uses deep learning-based technical means to conduct a comprehensive and accurate automated assessment of preoperative frailty status; by collecting and analyzing the patient's electronic medical record data and real-time status data, the deep learning model is used to obtain the patient's first and second frailty levels, and the total frailty level and the corresponding frailty grade are comprehensively calculated; in addition, combined with clinical risk factors, it is possible to make personalized predictions on the probability of occurrence of different postoperative complications in patients, and draw risk prediction histograms to more intuitively display the patient's surgical risks and provide clinicians with more comprehensive decision support; it realizes the quantification and standardization of preoperative evaluation, and the evaluation results are more scientific and reliable, providing important decision-making references for formulating personalized surgical plans and managing postoperative risks, thereby improving surgical effectiveness and safety. BRIEF DESCRIPTION OF THE DRAWINGS
[0080] Figure 1 This is a flow chart of a method for automated assessment of preoperative frailty status based on deep learning according to Example 1 of the present invention;
[0081] Figure 2 This is a schematic diagram of an electronic device according to Embodiment 2 of the present invention;
[0082] Figure 3 This is a schematic diagram of a storage medium according to Embodiment 3 of the present invention. DETAILED DESCRIPTION
[0083] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0084] Example 1
[0085] See also Figure 1 As shown, the method for automatic assessment of preoperative frailty based on deep learning described in this embodiment includes:
[0086] S1: Collect patients’ electronic medical record data;
[0087] Electronic medical record data included basic information, physical information, laboratory test information, imaging test information, medication history, surgical history, and current symptoms; patients were preoperative patients;
[0088] Basic information is the patient's personal information, including age and gender. Patients with different basic information will have different corresponding frailty states and risk levels when the other information is the same. The reason is that with age, the body's physiological functions gradually decline, such as cardiopulmonary function, immune function, etc. Patients of different genders have differences in body composition, physiological functions, etc. For example, female patients have less muscle mass and bone density than male patients, and female patients' livers are smaller than male patients' livers, etc. Therefore, it will affect the patient's frailty state and risk level.
[0089] Physical information includes appearance structure and physiological characteristics; appearance structure includes height and weight; physiological characteristics include blood pressure and heart rate;
[0090] Laboratory test information refers to the test results obtained after the patient undergoes medical laboratory tests; laboratory test information includes at least blood routine information, liver function information, kidney function information, electrolyte information and heart information; blood routine information includes the number of white blood cells, red blood cells and platelets; liver function information includes the number of alanine aminotransferase, aspartate aminotransferase and bilirubin; kidney function information includes blood creatinine and urea nitrogen; electrolyte information includes sodium content, potassium content and calcium content; heart information includes troponin content and phosphoinositide content;
[0091] Imaging examination information refers to the examination results obtained after the patient undergoes medical imaging examination; imaging examination information includes at least X-rays, CT scans, magnetic resonance imaging and ultrasound images;
[0092] Medication history is a record of the patient's medication history, including drug name and drug dosage;
[0093] The surgical history is the patient's historical surgical record, which includes the name and date of surgery;
[0094] Current symptoms are symptoms perceived by the patient; current symptoms include symptoms and onset time;
[0095] Combining physical information, laboratory test information, imaging test information, medication history, surgical history and current symptoms, it can accurately reflect the patient's overall condition, so it can be used to assess the patient's frailty and the risk level of the patient undergoing surgery;
[0096] Electronic medical record data are obtained from the electronic medical record system of medical institutions.
[0097] S2: pre-process the electronic medical record data and mark it as processed medical record data;
[0098] The steps for preprocessing electronic medical record data include:
[0099] Step a1: Clean the electronic medical record data, delete the erroneous data in the electronic medical record data, and mark the deleted data as missing data; erroneous data such as height is too large, white blood cell count is negative, age is non-integer, etc.; to ensure that there is no erroneous data in the electronic medical record data;
[0100] Step a2: Convert the categorical variables in the electronic medical record data into one-hot encoding;
[0101] Categorical variables are feature variables whose value types are categorical or class, such as gender, drug name, surgery name, etc. in electronic medical record data;
[0102] One-hot encoding is a method of converting categorical variables into numerical values. For example, for a categorical variable with A different categories, one-hot encoding creates A binary feature columns, where the corresponding column of each different category is set to 1 and the remaining columns are set to 0. The A binary feature columns are [1, 0, ..., 0], [0, 1, ..., 0], ..., [0, 0, ..., 1].
[0103] S3: Analyze electronic medical record data to obtain imaging diagnosis results;
[0104] Methods for analyzing electronic medical record data include:
[0105] Use the trained result diagnosis model to identify the imaging examination information in turn and output the corresponding recognition results, which include primary weakness, intermediate weakness and advanced weakness;
[0106] The specific training process of the result diagnosis model includes:
[0107] Collect a images of various types from imaging examination information in advance and mark them as training images, where a is an integer greater than 1; annotate each training image, including primary weakness, intermediate weakness and advanced weakness; convert primary weakness, intermediate weakness and advanced weakness into digital annotations, for example, primary weakness is converted to 1, intermediate weakness is converted to 2, and advanced weakness is converted to 3; divide the annotated training images into a training set and a test set, using 70% of the training images as the training set and 30% of the training images as the test set; use the training set to train the result diagnosis model, and use the test set to test the result diagnosis model; preset an error threshold, and when the mean of the prediction errors of all training images in the test set is less than the error threshold, output the result diagnosis model; wherein the calculation formula for the mean prediction error is: Where Z P is the prediction error, P is the number of the training image, α P is the predicted annotation corresponding to the Pth group of training images, μ P is the actual annotation corresponding to the Pth group of training images, U is the number of training images in the test set; the error threshold is preset according to the accuracy required by the result diagnosis model;
[0108] The above result diagnosis model is specifically a convolutional neural network model.
[0109] Methods for obtaining imaging diagnostic results include:
[0110] According to the output recognition results, count the number of each recognition result and the total number of recognition results;
[0111] Divide the number of each recognition result by the total number of recognition results to obtain the proportion of each recognition result, that is, to obtain the primary weakness proportion ZB q , Intermediate Weakness Percentage ZB d And the proportion of advanced weakness ZB z ;
[0112] like The imaging diagnosis result is advanced frailty, which means that the imaging examination information reflects that the patient's physiological function and state have declined severely;
[0113] like and The imaging diagnosis result is intermediate weakness, which means that the imaging examination information reflects that the patient's physiological function and state have declined seriously;
[0114] like and as well as The imaging diagnosis result is primary frailty, which means that the imaging examination information reflects a low decline in the patient's physiological function and status.
[0115] S4: Collect the patient's real-time status data;
[0116] Real-time status data includes patient facial images and missing data;
[0117] The patient's facial image is acquired by an image sensor installed directly opposite the operating table;
[0118] Missing data is obtained by medical staff through corresponding examinations of patients or by asking patients’ families. For example, if the missing data is height, white blood cell count, etc., it is obtained by examining the patients; if the missing data is age, it is obtained by asking patients’ families.
[0119] S5: Analyze the patient's first level of weakness by combining the medical record data, imaging diagnosis results and missing data; analyze the patient's second level of weakness by using the patient's facial image;
[0120] Methods for analyzing a patient's primary frailty level include:
[0121] According to the imaging diagnosis results, the corresponding diagnostic values are obtained; the medical record data, the diagnostic values and the missing data are processed as analysis data; the analysis data are input into the trained degree analysis model to predict the corresponding first degree of frailty; the diagnostic values corresponding to the imaging diagnosis results are assigned different values to different imaging diagnosis results by technicians in this field, wherein the value assigned to advanced frailty is greater than the value assigned to intermediate frailty, and the value assigned to intermediate frailty is greater than the value assigned to primary frailty;
[0122] The specific training process of the degree analysis model includes:
[0123] Pre-collecting the first frailty degree corresponding to b groups of analysis data, converting the analysis data and the corresponding first frailty degree into a corresponding set of feature vectors, where b is an integer greater than 1; wherein the first frailty degree corresponding to the analysis data is obtained by a person skilled in the art during the historical patient frailty status assessment process, by collecting b groups of different analysis data, and analyzing the corresponding first frailty degree respectively under the conditions of each group of analysis data in combination with actual experience;
[0124] Each set of feature vectors is used as the input of the degree analysis model. The degree analysis model uses a set of predicted first frailty levels corresponding to each set of analysis data as output, and uses the actual first frailty level corresponding to each set of analysis data as the prediction target. The actual first frailty level is the first frailty level corresponding to the analysis data collected in advance. The training target is to minimize the sum of the prediction errors of all analysis data. The calculation formula of the prediction error is δ k =(β k -x k ) 2 , where δk is the prediction error, k is the group number of the eigenvector corresponding to the analyzed data, β k is the predicted first frailty level corresponding to the kth group of analysis data, χ k is the actual first frailty degree corresponding to the kth group of analysis data; the degree analysis model is trained until the sum of the prediction errors reaches convergence and the training is stopped;
[0125] The above-mentioned degree analysis model is specifically a deep neural network model, which includes an input layer, a hidden layer and an output layer; each hidden layer includes multiple neurons, each neuron is connected to the neurons in the next layer, and the connection contains weights, which determine the importance and influence of data transmission in the neural network; an activation function is applied to each neuron between the hidden layer and the output layer, and the activation function reflects nonlinearity, allowing the network to learn more complex patterns and features.
[0126] The steps for analyzing a patient's second level of frailty include:
[0127] Step b1: Using Viola-Jones face detection, extract the entire facial area from the patient's facial image;
[0128] Step b2: using the active shape model to extract facial key points in the facial area and the coordinates corresponding to the facial key points, the facial key points include upper eye corner points, lower eye corner points, left mouth corner points and right mouth corner points;
[0129] Step b3: Calculate the eye height according to the coordinates of the upper eye corner point and the lower eye corner point, and calculate the lip width according to the coordinates of the left mouth corner point and the right mouth corner point;
[0130] The expression for eye height is:
[0131] Where H' is the eye height, x 2 is the horizontal coordinate of the upper eye corner point, x 1 is the horizontal coordinate of the lower eye corner point, y 2 is the ordinate of the upper eye corner point, y 1 is the ordinate of the lower eye corner point;
[0132] The expression for lip width is:
[0133] Where D is the lip width, x 4 is the horizontal coordinate of the right corner of the mouth, x 3 is the horizontal coordinate of the left corner of the mouth, y 4 is the ordinate of the right corner of the mouth, y 3 is the ordinate of the left mouth corner point;
[0134] Step b4: graying the patient's facial image and marking it as a gray image; segmenting the skin area in the gray image using a region segmentation algorithm;
[0135] Step b5: Sampling the skin area, obtaining the grayscale value corresponding to each pixel, and constructing a grayscale value matrix G;
[0136] Step b6: Preset the construction parameters of the gray-level co-occurrence matrix GLCM, which include direction and distance. In this embodiment, the preferred direction is 45° and the distance is 1 pixel. According to the construction parameters of the gray-level co-occurrence matrix GLCM, traverse each gray value in the gray value matrix G to obtain the gray value combination of each pair of adjacent pixels. Count the occurrence frequency of each gray value combination, which is the frequency of occurrence of the gray value combination in the skin area, and add it to the corresponding position of the gray-level co-occurrence matrix GLCM.
[0137] Step b7: According to the gray level co-occurrence matrix GLCM, the contrast of the gray level co-occurrence matrix GLCM is calculated and marked as skin roughness;
[0138] The contrast expression of the gray level co-occurrence matrix GLCM is:
[0139] In the formula, DB is the contrast, m is i is a gray value in the i-th gray value combination, n i is another gray value in the i-th gray value combination, S i is the occurrence frequency of the i-th gray value combination, C is the number of gray value combinations, i∈[1,C];
[0140] Step b8: using eye height, lip width and skin roughness as test data, inputting the test data into the trained degree prediction model to predict the corresponding second weakness degree; wherein the specific training process of the degree prediction model is consistent with the training process of the degree analysis model, and is also a deep neural network model;
[0141] It should be noted that Viola-Jones face detection, active shape model and region segmentation algorithm are all existing technologies and will not be described in detail here.
[0142] S6: Calculate the total weakness degree by combining the patient's first weakness degree and second weakness degree, and analyze the corresponding weakness level;
[0143] The expression of total weakness is: R z =ω 1 ×R 1 +ω 2 ×R 2 ;
[0144] In the formula, Rz is the overall weakness, R 1 The first level of weakness, R 2 The second level of weakness, ω 1 ,ω 2 All are weight coefficients;
[0145] The specific value of the weight coefficient in the formula can be set according to actual conditions. The weight coefficient reflects the influence of the first degree of weakness and the second degree of weakness on the patient's overall weakness. Technical personnel in this field can preset the corresponding weight coefficient according to the actual influence of the first degree of weakness and the second degree of weakness on the patient's overall weakness, so as to accurately assess the patient's overall weakness.
[0146] The range of total weakness is [M 1 , M 2 ], where M 1 、M 2 are all integers greater than 0 and M 2 >M 1 ; The frailty levels include mild frailty, moderate frailty, and severe frailty;
[0147] If the calculated total frailty level is If the patient's overall weakness level is lower than 0.01, it means that the patient's overall weakness level is lower, and the weakness level is mild weakness;
[0148] If the calculated total frailty level is If the patient's overall weakness level is higher than 0.01, it means that the patient's overall weakness level is higher, and the weakness level is moderate weakness;
[0149] If the calculated total frailty level is If the patient's total weakness level is too high, the weakness level is severe weakness.
[0150] S7: Obtain clinical risk factors, predict the probability of postoperative complications of patients based on clinical risk factors and total frailty, and draw a risk prediction histogram;
[0151] Clinical risk factors include the type of surgery, surgical site, and anesthesia method; clinical risk factors are also obtained from the electronic medical record system of the medical institution; clinical risk factors are relevant data about the patient's upcoming surgery;
[0152] Different numbers are set for different types of surgery, surgical sites and anesthesia methods; the types of surgery, surgical sites and anesthesia methods in clinical risk factors are converted into corresponding numbers;
[0153] Taking clinical risk factors and the overall frailty level as evaluation data, input the evaluation data into T trained probability prediction models respectively to predict the occurrence probabilities of different postoperative complications for the patient; postoperative complications such as arrhythmia, pulmonary embolism, renal failure, etc.; one probability prediction model corresponds to one postoperative complication, and T is an integer greater than 1; according to the predicted occurrence probabilities of different postoperative complications for the patient, draw a risk prediction histogram, and the risk prediction histogram includes the occurrence probability of each postoperative complication; it can provide decision-making guidance for medical staff.
[0154] The specific training process of the probability prediction model includes:
[0155] The probability prediction model adopts a random forest regression model. Pre-collect E groups of historical data. The historical data includes evaluation data and the occurrence probability of postoperative complications corresponding to the evaluation data; sample N tree subsets from the E groups of historical data, and each tree subset includes e groups of historical data, where e is any value in (1, E], and train each decision tree according to the N tree subsets; among them, both E and N are integers greater than 1, and N < E;
[0156] Initialize the model parameters: determine the number N of decision trees in the forest, the maximum depth B of each decision tree, the minimum number of samples F in the internal nodes, and the minimum number of samples Q in the leaf nodes;
[0157] Use the CART algorithm to recursively construct the first decision tree:
[0158] Divide the e groups of historical data in a tree subset into a training set α and a validation set β, where α r is the r-th group of historical data in the training set α, r ∈ [1, α'], α' < e, and α' is the number of historical data in the training set α; use the feature t h to represent one of the surgical type number, surgical site number, anesthesia method number, and overall frailty level, and calculate the information gain ratio Z(α, t h ) of the h-th feature t h ), and the calculation formula of Z(α, t h ) is as follows:
[0159]
[0160] In the formula, h ∈ [1, 4], af is the decision subset divided by the training set α according to the feature t h , f ∈ [1, 4], H(af) is the information entropy of af, and H(α) is the information entropy of the training set α. The calculation methods of H(α) and H(af) respectively include:
[0161]
[0162]
[0163] In the formula, αf r represents the probability of postoperative complications corresponding to the historical data of group r in αf;
[0164] For each feature t h The corresponding information gain rate Z(α, t h ) is calculated and the maximum information gain rate Z(α, t h ) corresponds to the feature t h As a node, and according to the maximum information gain rate Z(α, t h ) corresponds to the feature t h Divide the training set α into decision subsets so that the number of historical data in each decision subset is the same;
[0165] For each maximum information gain rate Z(α, t h ) and repeat the algorithm recursive process until the decision subset cannot be divided into smaller decision subsets; the performance of the trained decision tree model is evaluated on the validation set β, and finally the construction of the first decision tree model is completed;
[0166] The remaining N-1 tree subsets are used in turn to train N-1 decision trees; the generation process of each decision tree is independent and based on the CART algorithm; finally, a random forest containing N decision trees is constructed;
[0167] Calculate the loss function value on N decision trees; input the training data into each decision tree and record the decision output, which is the probability of postoperative complications corresponding to the evaluation data; calculate the mean square loss of the error between the average output of N decision trees and the true value; reduce the mean square loss through parameter tuning, and select the model parameters with the smallest mean square loss as the final model.
[0168] This embodiment uses deep learning-based technical means to conduct a comprehensive and accurate automated assessment of preoperative frailty status; by collecting and analyzing the patient's electronic medical record data and real-time status data, the deep learning model is used to obtain the patient's first and second frailty levels, and the total frailty level and the corresponding frailty grade are comprehensively calculated; in addition, combined with clinical risk factors, it is possible to make personalized predictions on the probability of occurrence of different postoperative complications in patients, and draw risk prediction histograms to more intuitively display the patient's surgical risks and provide clinicians with more comprehensive decision support; it realizes the quantification and standardization of preoperative evaluation, and the evaluation results are more scientific and reliable, providing important decision-making references for formulating personalized surgical plans and managing postoperative risks, thereby improving surgical effectiveness and safety.
[0169] Example 2
[0170] See also Figure 2As shown, according to another aspect of the present application, an electronic device 500 is also provided. The electronic device 500 may include one or more processors and one or more memories. The memories store computer readable codes, and when the computer readable codes are executed by one or more processors, the method for automatically assessing preoperative frailty based on deep learning as described above may be executed.
[0171] The method or system according to the embodiment of the present application can also be used by Figure 2 The electronic device architecture shown in FIG. Figure 2 As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, ROM 503, RAM 504, a communication port 505 connected to a network, an input / output 506, a hard disk 507, etc. The storage device in the electronic device 500, such as ROM 503 or hard disk 507, may store the preoperative frailty state automated assessment method based on deep learning provided by the present application. Further, the electronic device 500 may also include a user interface 508. Of course, Figure 2 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 2 One or more components of an electronic device are shown.
[0172] Example 3
[0173] See also Figure 3 As shown, a computer-readable storage medium 600 according to an embodiment of the present application is shown. Computer-readable instructions are stored on the computer-readable storage medium 600. When the computer-readable instructions are executed by the processor, the automated assessment method for preoperative frailty state based on deep learning according to the embodiment of the present application described with reference to the above figures can be executed. The storage medium 600 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.
[0174] In addition, according to the embodiments of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided in the present application, for example: a method for automated assessment of preoperative frailty based on deep learning. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed.
[0175] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
[0176] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. An automated assessment method for preoperative frailty based on deep learning, characterized in that: include: S1: Collect patients’ electronic medical record data; S2: pre-process the electronic medical record data and mark it as processed medical record data; S3: Analyze electronic medical record data to obtain imaging diagnosis results; S4: Collecting the patient's real-time status data, including the patient's facial image and missing data; S5: Combine the medical record data, imaging diagnosis results and missing data to analyze the patient's first frailty level; Analyze the patient's second level of weakness through the patient's facial image; S6: Calculate the total weakness degree by combining the patient's first weakness degree and second weakness degree, and analyze the corresponding weakness level; S7: Obtain clinical risk factors, predict the probability of postoperative complications in patients based on clinical risk factors and total frailty level, and draw a risk prediction histogram.
2. The method for automated assessment of preoperative frailty based on deep learning according to claim 1, characterized in that: The electronic medical record data includes basic information, physical information, laboratory test information, imaging test information, medication history, surgical history and current symptoms; the patient is a preoperative patient; The physical information includes appearance structure and physiological characteristics; the appearance structure includes height and weight; the physiological characteristics include blood pressure and heart rate; The laboratory examination information is the examination results obtained after the patient undergoes medical laboratory examination; The imaging examination information is the examination results obtained after the patient undergoes a medical imaging examination; the imaging examination information at least includes X-rays, CT scans, magnetic resonance imaging and ultrasonic images; The medication history is a patient's historical medication record, which includes the name of the drug and the dosage of the drug; The surgical history is a patient's historical surgical record, and the surgical history includes the name of the surgery and the date of the surgery; The current symptoms are symptoms perceived by the patient, and the current symptoms include symptoms and onset time.
3. The method for automated assessment of preoperative frailty based on deep learning according to claim 2, characterized in that: The step of preprocessing the electronic medical record data comprises: Step a1: Clean the electronic medical record data, delete the erroneous data in the electronic medical record data, and mark the deleted data as missing data; Step a2: Convert the categorical variables in the electronic medical record data into one-hot encoding; Categorical variables are feature variables whose value type is classification or category; One-hot encoding is an encoding method that converts categorical variables into numerical values.
4. The method for automated assessment of preoperative frailty based on deep learning according to claim 3, characterized in that: The method for analyzing electronic medical record data comprises: Use the trained result diagnosis model to identify the imaging examination information in turn and output the corresponding recognition results, which include primary weakness, intermediate weakness and advanced weakness; The results of the diagnostic model training process include: Collect a images of various types from imaging examination information in advance and mark them as training images, where a is an integer greater than 1; annotate each training image, including primary weakness, intermediate weakness and advanced weakness; convert primary weakness, intermediate weakness and advanced weakness into digital annotations respectively; divide the annotated training images into a training set and a test set; use the training set to train the result diagnosis model, and use the test set to test the result diagnosis model; preset an error threshold, and when the mean of the prediction errors of all training images in the test set is less than the error threshold, output the result diagnosis model; the result diagnosis model is a convolutional neural network model.
5. The method for automated assessment of preoperative frailty based on deep learning according to claim 4, characterized in that: The method for obtaining imaging diagnosis results comprises: According to the output recognition results, count the number of each recognition result and the total number of recognition results; Divide the number of each recognition result by the total number of recognition results to obtain the proportion of each recognition result, that is, to obtain the primary weakness proportion ZB q , Intermediate Weakness Percentage ZB d And the proportion of advanced weakness ZB z ; like Then the imaging diagnosis result is advanced frailty; like and Then the imaging diagnosis result is intermediate frailty; like and as well as The imaging diagnosis result is primary frailty.
6. The method for automated assessment of preoperative frailty based on deep learning according to claim 5, characterized in that: The method of analyzing a first frailty level of a patient comprises: According to the imaging diagnosis results, the corresponding diagnostic value is obtained; the medical record data, the diagnostic value and the missing data are processed as analysis data; the analysis data is input into the trained degree analysis model to predict the corresponding first frailty degree; The training process of the degree analysis model includes: Pre-collecting the first frailty levels corresponding to b groups of analysis data, converting the analysis data and the corresponding first frailty levels into a corresponding set of feature vectors, where b is an integer greater than 1; Each group of feature vectors is used as the input of the degree analysis model, and the degree analysis model takes a group of predicted first frailty degrees corresponding to each group of analysis data as output, and takes the actual first frailty degree corresponding to each group of analysis data as the prediction target, and the actual first frailty degree is the first frailty degree corresponding to the analysis data collected in advance; minimizing the sum of prediction errors of all analysis data is used as the training goal; the degree analysis model is trained until the sum of prediction errors reaches convergence and the training is stopped; the degree analysis model is a deep neural network model.
7. The method for automated assessment of preoperative frailty based on deep learning according to claim 6, characterized in that: The step of analyzing the second frailty level of the patient comprises: Step b1: Using Viola-Jones face detection, extract the entire facial area from the patient's facial image; Step b2: using the active shape model to extract facial key points in the facial area and the coordinates corresponding to the facial key points, the facial key points include upper eye corner points, lower eye corner points, left mouth corner points and right mouth corner points; Step b3: Calculate the eye height according to the coordinates of the upper eye corner point and the lower eye corner point, and calculate the lip width according to the coordinates of the left mouth corner point and the right mouth corner point; The expression for eye height is: Where H' is the eye height, x2 is the horizontal coordinate of the upper eye corner point, x1 is the horizontal coordinate of the lower eye corner point, y2 is the vertical coordinate of the upper eye corner point, and y1 is the vertical coordinate of the lower eye corner point; The expression for lip width is: Where D is the lip width, x4 is the horizontal coordinate of the right corner of the mouth, x3 is the horizontal coordinate of the left corner of the mouth, y4 is the vertical coordinate of the right corner of the mouth, and y3 is the vertical coordinate of the left corner of the mouth; Step b4: graying the patient's facial image and marking it as a gray image; segmenting the skin area in the gray image using a region segmentation algorithm; Step b5: Sampling the skin area, obtaining the grayscale value corresponding to each pixel, and constructing a grayscale value matrix G; Step b6: Preset the construction parameters of the gray-level co-occurrence matrix GLCM, which include direction and distance; according to the construction parameters of the gray-level co-occurrence matrix GLCM, traverse each gray-level value in the gray-level value matrix G to obtain the gray-level value combination of each pair of adjacent pixels; count the occurrence frequency of each gray-level value combination, which is the frequency of occurrence of the gray-level value combination in the skin area, and add it to the corresponding position of the gray-level co-occurrence matrix GLCM; Step b7: According to the gray level co-occurrence matrix GLCM, the contrast of the gray level co-occurrence matrix GLCM is calculated and marked as skin roughness; The contrast expression of the gray level co-occurrence matrix GLCM is: Where, DB is contrast, m is i is a gray value in the i-th gray value combination, n i is another gray value in the i-th gray value combination, S i is the occurrence frequency of the i-th gray value combination, C is the number of gray value combinations, i∈[1,C]; Step b8: Use eye height, lip width and skin roughness as test data, input the test data into the trained degree prediction model, and predict the corresponding second degree of weakness; wherein the training process of the degree prediction model is consistent with the training process of the degree analysis model, and is also a deep neural network model.
8. The method for automated assessment of preoperative frailty based on deep learning according to claim 7, characterized in that: The expression of total weakness is: R z =ω1×R1+ω2×R2; In the formula, R z is the total frailty level, R1 is the first frailty level, R2 is the second frailty level, ω1 and ω2 are weight coefficients; The range of total frailty is [M1, M2], where M1 and M2 are both integers greater than 0 and M2>M1; frailty levels include mild frailty, moderate frailty, and severe frailty; If the calculated total frailty level is If the weakness level is within 100%, the weakness level is mild weakness; If the calculated total frailty level is If the frailty level is within 100%, the frailty level is moderate; If the calculated total frailty level is If the value is within 30%, the weakness level is severe weakness.
9. The method for automated assessment of preoperative frailty based on deep learning according to claim 8, characterized in that: The clinical risk factors include type of surgery, surgical site, and anesthesia; Different numbers are set for different types of surgery, surgical sites and anesthesia methods; the types of surgery, surgical sites and anesthesia methods in clinical risk factors are converted into corresponding numbers; Clinical risk factors and total frailty level were used as evaluation data, and the evaluation data were respectively input into T trained probability prediction models to predict the probability of different postoperative complications in patients; one of the probability prediction models corresponded to one postoperative complication, and T was an integer greater than 1; based on the predicted probabilities of different postoperative complications in patients, a risk prediction histogram was drawn, and the risk prediction histogram included the probability of each postoperative complication.
10. The method for automated assessment of preoperative frailty based on deep learning according to claim 9, characterized in that: The training process of the probability prediction model includes: The probability prediction model uses a random forest regression model. E groups of historical data are collected in advance. The historical data include evaluation data and the probability of postoperative complications corresponding to the evaluation data. N tree subsets are sampled from the E group of historical data. Each tree subset includes e groups of historical data, where e is any value in (1, E]. Each decision tree is trained based on the N tree subsets. Where E and N are both integers greater than 1, and N is <E; Initialize the model parameters, including determining the number of decision trees in the forest N, the maximum depth of each decision tree B, the minimum number of samples of internal nodes F, and the minimum number of samples of leaf nodes Q; Use the CART algorithm to recursively build the first decision tree: Divide the e sets of historical data in a tree subset into a training set α and a validation set β, α r is the r-th set of historical data in the training set α, r ∈ [l, α′], α′ < e, and α′ is the number of historical data in the training set α; use the feature t h to represent one of the surgical type number, surgical site number, anesthesia method number, and total frailty level, and calculate the information gain ratio Z(α, t h ) of the h-th feature t h ), and the calculation formula of Z(α, t h ) is as follows: In the formula, h∈[1,4], af is the training set α according to the feature t h The decision subsets divided are f∈[l, 4], H(af) is the information entropy of af, and H(α) is the information entropy of the training set α, where the calculation methods of H(α) and H(af) include: In the formula, αf r represents the probability of postoperative complications corresponding to the historical data of group r in af; For each feature t h The corresponding information gain rate Z(α,t h ) is calculated and the maximum information gain rate Z(α, t h ) corresponds to the feature t h As a node, and according to the maximum information gain rate Z(α, t h ) corresponds to the feature t h Divide the training set α into decision subsets, with the same amount of historical data in each decision subset; For each maximum information gain rate Z(α, t h ) and repeat the algorithm recursive process until the decision subset cannot be divided into smaller decision subsets; the performance of the trained decision tree model is evaluated on the validation set β, and finally the construction of the first decision tree model is completed; The remaining N-1 tree subsets are used in turn to train N-1 decision trees; the generation process of each decision tree is independent and based on the CART algorithm; finally, a random forest containing N decision trees is constructed; Calculate the loss function value on N decision trees; input the training data into each decision tree and record the decision output, which is the probability of postoperative complications corresponding to the evaluation data; calculate the mean square loss of the error between the average output of N decision trees and the true value; reduce the mean square loss through parameter tuning, and select the model parameters with the smallest mean square loss as the final model.
Citation Information
Patent Citations
Patient heart failure death risk prediction system based on characteristic rearrangement one-dimensional convolutional neural network
CN111387938A
Preoperative risk assessment method and system for surgical operation
CN113178258A
Method, device and system for predicting postoperative delirium occurrence risk
CN116168840A
Traditional Chinese medicine face diagnosis method, system and equipment based on machine vision and medium
CN116230203A
Nutrition condition prediction device, system and method and electronic equipment
CN117219276A
