Femoral head ischemic necrosis risk prediction method and device based on machine learning

Abstract

The invention provides a femoral head ischemic necrosis risk prediction method and device based on machine learning, and the method comprises the steps: collecting the related information of a patient, screening a mapping relation between an outcome variable and a prediction variable according to standardized text data and a C ox regression analysis method, and building a text machine learning model; carrying out image segmentation and feature labeling on the standardized image data, establishing a focus positioning model, and carrying out training and verification by using sample image data; and fusing the text machine learning model and the image recognition model to obtain a multi-modal femoral head ischemic necrosis risk prediction model, predicting an ischemic necrosis focus position and a staging diagnosis result of a patient, and generating a corresponding risk assessment report. Through a machine learning model, whether the patient is more likely to develop into ANFH or not can be predicted, and a personalized treatment strategy is formulated. In combination with iconography data, lifestyle information and clinical data, the A I can construct an accurate prediction model to help doctors to formulate a more effective treatment scheme.

Description

Technical Field

[0001] This application relates to the field of computer image processing technology, and in particular to a method and device for predicting the risk of avascular necrosis of the femoral head based on machine learning. Background Art

[0002] Avascular necrosis of the femoral head (ANFH) is a disease that is highly prevalent among young and middle-aged people. It is mainly caused by bone blood supply disorders and may lead to loss of joint function and a decline in quality of life. Early diagnosis and intervention are the keys to delaying the progression of the disease, while traditional imaging examinations (such as X-rays and CTs) have low sensitivity in early diagnosis. In recent years, the development of artificial intelligence (AI) technology has provided new possibilities for predicting the risk of avascular necrosis of the femoral head. Through machine learning models, it is possible to combine multimodal information such as the patient's lifestyle, medication history, and imaging data to construct a precise prediction model, thereby enabling early intervention.

[0003] ANFH has a high disability rate, and early intervention can significantly improve the prognosis. Existing diagnoses rely on late-stage imaging manifestations (such as the MRI crescent sign) and lack early prediction methods. Traditional statistical methods (such as Logistic regression) are limited by linear assumptions. The application of deep learning in image analysis (such as ResNet segmentation) requires a large amount of labeled data. The method for predicting the risk of avascular necrosis of the femoral head based on machine learning provides new ideas for early diagnosis and intervention. By combining multimodal data and advanced algorithms, the accuracy and efficiency of prediction can be significantly improved.

[0004] Therefore, in view of the problem of the difficulty in early diagnosis of avascular necrosis of the femoral head (ANFH), it is an important research content for those skilled in the art to propose a risk prediction model that integrates multimodal data and machine learning technology. Summary of the Invention

[0005] In view of this, it is necessary to provide a method and device for predicting the risk of avascular necrosis of the femoral head based on machine learning, which utilizes a modal feature fusion strategy (clinical + radiomics + biochemical indicators) to improve the maintenance and management level of the device. Based on a lightweight model of ensemble learning, both performance and interpretability are considered.

[0006] In a first aspect, an embodiment of the present application provides a method for predicting the risk of avascular necrosis of the femoral head based on machine learning, the method comprising:

[0007] Step 1: Collect patient-related information in a local database, including: personal basic information, electronic medical records, and examination imaging data;

[0008] Step 2: Classify the patient-related information into structured text data and unstructured image data, and store them correspondingly using a graph database;

[0009] Step 3: Preprocess and standardize the structured text data and unstructured image data respectively to obtain standardized text data and standardized image data as the source of sample data;

[0010] Step 4: Screen the mapping relationship between the outcome variable and the predictive variable according to the standardized text data and the Cox regression analysis method, establish a text machine learning model, and use the sample text data for training and verification;

[0011] Step 5: Perform image segmentation and feature annotation on the standardized image data, establish a lesion localization model, and use the sample image data for training and verification;

[0012] Step 6: Integrate the text machine learning model and the image recognition model to obtain a multimodal risk prediction model for avascular necrosis of the femoral head, and use the sample text data and image data for retraining and verification;

[0013] Step 7: Predict the location of the avascular necrosis lesion and the staging diagnosis result of the patient according to the patient's clinical data and the multimodal risk prediction model for avascular necrosis of the femoral head, and generate a corresponding risk assessment report.

[0014] Optionally, in an implementation manner of the first aspect of the present invention, the step 2: classify the patient-related information into structured text data and unstructured image data, and store them correspondingly using a graph database, including:

[0015] S2.1 Store the structured text data in the form of key-value pairs and store the unstructured image data in the form of images or videos;

[0016] S2.2 Establish an entity node set, and the entity node set includes: patient nodes, medical record nodes, test nodes, and image nodes;

[0017] S2.3 Establish a relationship set, and the relationship set includes: patient-medical record relationship, patient-test relationship, patient-image relationship, medical record-test relationship, and medical record-image relationship;

[0018] S2.4 Create an index for the patient ID field, store the image files in the distributed file system object storage, and only store the metadata of the images in the graph database.

[0019] Optionally, in an implementation manner of the first aspect of the present invention, the step 3: preprocess and standardize the structured text data and unstructured image data respectively to obtain standardized text data and standardized image data as the source of sample data, including:

[0020] Preprocessing and standardization of structured text data, including: deduplication of structured text data, removal of stop words, and annotation processing; converting text data into a unified format and filling missing values through interpolation; using the BERT embedding model to generate vector representations of the text for subsequent analysis;

[0021] Preprocessing and standardization of unstructured image data, including: rotation, flipping, and cropping: adjusting the orientation and size of the image to meet the model input requirements; noise reduction processing: removing noise in the image to improve image quality; scale transformation: adjusting the resolution and size of the image to ensure that the image has a unified aspect ratio and number of pixels; grayscale transformation: converting a color image into a grayscale image to simplify subsequent processing; histogram equalization: adjusting the contrast of the image to make it easier to analyze; mean normalization; mixed pixel value calculation: combining the standard deviation and mean of multiple images to generate a mixed enhanced image.

[0022] Optionally, in an implementation manner of the first aspect of the present invention, step 4: screening the mapping relationship between the outcome variable and the predictive variable according to the standardized text data and the Cox regression analysis method, and establishing a text machine learning model, training and validating it using the sample text data, including:

[0023] S4.1 Screening predictive variables through univariate or multivariate Cox regression analysis. Univariate analysis is used to preliminarily screen significant variables, and multivariate analysis further verifies the influence of these variables on the outcome variable;

[0024] S4.2 Using the screened variables to construct a Cox proportional hazards model to estimate the influence of each variable on the outcome variable;

[0025] S4.3 Evaluating the prediction performance of the model through methods such as cross-validation and ROC curves;

[0026] S4.4 After completing the standardization and feature extraction of the text data, using a machine learning algorithm to construct a prediction model;

[0027] S4.5 Using Cox regression to screen out significant predictive variables as the input of the machine learning model;

[0028] S4.6 Taking the results of Cox regression as a part of the machine learning model as feature weights or baseline prediction values;

[0029] S4.7 Optimizing the performance of Cox regression and the machine learning model simultaneously through a multi-objective optimization method.

[0030] Optionally, in an implementation manner of the first aspect of the present invention, step 5: performing image segmentation and feature annotation on the standardized image data, and establishing a lesion localization model, training and validating it using the sample image data, including:

[0031] S5.1 Multi-scale feature extraction step: The input image is first subjected to feature extraction through the ResNetSt network, which performs four downsamplings through four ResNetSt blocks to obtain multi-scale features of the image;

[0032] S5.2 Channel splicing step: The multi-scale features are spliced with the feature maps obtained by upsampling the feature pyramid to obtain fused features. By stacking multiple feature maps in the channel dimension, the depth of the feature maps is increased to retain more information;

[0033] S5.3 Dual-branch feature extraction step: The spliced feature maps are fed into a dual-branch network for further processing. One branch uses the cross-attention mechanism to extract global features, and the other branch uses dilated convolutions with different sampling rates to extract local features to capture lesion information at different scales;

[0034] S5.4 Feature fusion and dependence weight learning step: After channel splicing, the global features and local features at multiple scales are further fused through the dual-branch network, enabling the global features and local features at each scale to learn different dependence weights and improving the model's adaptability to complex scenarios;

[0035] S5.5 Lesion classification prediction step: A Deeplab3+ semantic segmentation model is established, and the Squeeze-and-Excitation module is introduced to enhance the attention to the lesion area. Feature extraction and semantic segmentation are performed on the input image data, and each pixel is classified for lesions through the decoder to obtain the prediction result;

[0036] S5.6 Training and validation step: A lesion localization model based on ResNet and the attention mechanism is established through the above steps, and sample image data is used for training and validation.

[0037] Optionally, in an implementation manner of the first aspect of the present invention, wherein, a combination of Dice loss and cross-entropy loss is used for training, and the loss function is:

[0038]

[0039] Y i represents the class probability that the i-th class is the true label, represents the probability that the model predicts the i-th class, N represents the total number of classes, and α represents the weight adjustment coefficient;

[0040] The model evaluation formula is:

[0041]

[0042] where Sij Indicates the number of samples predicted as the nth class for the mth class of samples.

[0043] Optionally, in an implementation manner of the first aspect of the present invention, step 6: fusing the text machine learning model and the image recognition model to obtain a multi-modal risk prediction model for avascular necrosis of the femoral head, and retraining and validating using sample text data and image data, including:

[0044] S6.1 Feature fusion step: mapping text features and image features to the same feature space through joint embedding and contrast learning methods;

[0045] S6.2 Model fusion step: fusing the outputs of the text model and the image model through a multi-modal fusion network in deep learning;

[0046] S6.3 Training and validation step: retraining using sample text data and image data, and verifying the model performance through ROC curve and AUC value metrics.

[0047] In a second aspect, an embodiment of the present application provides a risk prediction device for avascular necrosis of the femoral head based on machine learning, which is applied to a risk prediction method for avascular necrosis of the femoral head based on machine learning as described in the first aspect, and is characterized in that the system includes:

[0048] A data acquisition module, configured to collect patient-related information in a local database, including: personal basic information, electronic medical records, and examination image data;

[0049] A data division module, configured to classify patient-related information into structured text data and unstructured image data, and store them correspondingly using a graph database;

[0050] A data standardization module, configured to perform preprocessing and standardization processing on the structured text data and the unstructured image data respectively to obtain standardized text data and standardized image data as the source of sample data;

[0051] A feature screening module, configured to screen the mapping relationship between the outcome variable and the prediction variable according to the standardized text data and the Cox regression analysis method, and establish a text machine learning model, and train and validate it using sample text data;

[0052] A feature segmentation module, configured to perform image segmentation and feature annotation on the standardized image data, and establish a lesion localization model, and train and validate it using sample image data;

[0053] A model construction module, configured to fuse a text machine learning model and an image recognition model to obtain a multi-modal risk prediction model for avascular necrosis of the femoral head, and re-train and validate it using sample text data and image data;

[0054] A risk assessment module, configured to predict the location of the ischemic necrosis lesion and the staging diagnosis result of the patient according to the patient's clinical data and the multi-modal risk prediction model for avascular necrosis of the femoral head, and generate a corresponding risk assessment report.

[0055] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0056] A processor;

[0057] A memory for storing instructions executable by the processor;

[0058] Wherein, when the processor is configured to execute the instructions, it implements the method for predicting the risk of avascular necrosis of the femoral head based on machine learning described in the first aspect.

[0059] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a program, and the program instructs the device to execute the method for predicting the risk of avascular necrosis of the femoral head based on machine learning described in the first aspect.

[0060] A method and device for predicting the risk of avascular necrosis of the femoral head based on machine learning provided by the embodiment of the present application, by collecting patient-related information in a local database, including: personal basic information, electronic medical records, and examination image data; classifying the patient-related information into structured text data and unstructured image data, and storing them correspondingly using a graph database; respectively preprocessing and normalizing the structured text data and unstructured image data to obtain normalized text data and normalized image data as the source of sample data; screening the mapping relationship between the outcome variable and the prediction variable according to the normalized text data and the Cox regression analysis method, and establishing a text machine learning model, and training and validating it using sample text data; performing image segmentation and feature annotation on the normalized image data, and establishing a lesion localization model, and training and validating it using sample image data; fusing the text machine learning model and the image recognition model to obtain a multi-modal risk prediction model for avascular necrosis of the femoral head, and re-training and validating it using sample text data and image data; predicting the location of the ischemic necrosis lesion and the staging diagnosis result of the patient according to the patient's clinical data and the multi-modal risk prediction model for avascular necrosis of the femoral head, and generating a corresponding risk assessment report.

[0061] The beneficial effects of this solution specifically include:

[0062] (1) AI technology can denoise, correct, and quantitatively analyze the imaging data of avascular necrosis of the femoral head, improving the positioning accuracy of the lesion area.

[0063] (2) Through a machine learning model, it is possible to predict whether a patient is more likely to develop ANFH and formulate personalized treatment strategies.

[0064] (3) Combining imaging data, lifestyle information, and clinical data, AI can build an accurate prediction model to help doctors develop more effective treatment plans. Description of the Drawings

[0065] Figure 1 Schematic flow chart of a method for predicting the risk of avascular necrosis of the femoral head based on machine learning provided by an embodiment of the present application.

[0066] Figure 2 Schematic flow chart of steps for establishing a lesion localization model provided by an embodiment of the present application.

[0067] Figure 3 Schematic diagram of a multi-modal risk prediction model for avascular necrosis of the femoral head provided by an embodiment of the present application.

[0068] Figure 4 Schematic diagram of a module of a device for predicting the risk of avascular necrosis of the femoral head based on machine learning provided by an embodiment of the present application.

[0069] Figure 5 Schematic diagram of an electronic terminal device provided by an embodiment of the present application. Detailed Embodiments

[0070] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments.

[0071] It should be noted that "at least one" in the embodiments of the present application refers to one or more, and multiple refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present application belongs. The terms used in the description of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0072] It should be noted that, in the embodiments of the present application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order. Features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0073] Based on the implementations in this application, all other implementations obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0074] In view of this, the present application provides a method and device for predicting the risk of avascular necrosis of the femoral head based on machine learning. AI technology can denoise, correct and quantitatively analyze the imaging data of avascular necrosis of the femoral head to improve the positioning accuracy of the lesion area. Through the machine learning model, it is possible to predict whether the patient is more likely to develop ANFH and develop a personalized treatment strategy. Combining imaging data, lifestyle information and clinical data, AI can build an accurate prediction model to help doctors develop more effective treatment plans.

[0075] Figure 1 A flow chart of a method for predicting the risk of avascular necrosis of the femoral head based on machine learning is provided in accordance with one embodiment of the present application.

[0076] Step 1: Collect patient-related information in the local database, including basic personal information, electronic medical records, and examination image data.

[0077] Specifically, in the embodiment of the present application, when constructing a risk prediction model for avascular necrosis of the femoral head (ANFH) in a local database, it is necessary to systematically process multi-source heterogeneous medical data. When collecting patient-related information in a local database, it is necessary to ensure the integrity, accuracy and security of the data.

[0078] Step 2: Classify patient-related information into structured text data and unstructured image data, and use a graph database for corresponding storage.

[0079] Specifically, in the embodiment of the present application, the step 2: classifying the patient-related information into structured text data and unstructured image data, and using a graph database for corresponding storage, includes:

[0080] S2.1 Store structured text data in the form of key-value pairs and store unstructured image data in the form of images or videos;

[0081] S2.2 Establish a set of entity nodes, where the set of entity nodes includes: patient nodes, medical record nodes, examination nodes, and imaging nodes;

[0082] S2.3 Establish a set of relationships, where the set of relationships includes: patient-medical record relationship, patient-examination relationship, patient-imaging relationship, medical record-examination relationship, medical record-imaging relationship;

[0083] S2.4 Create an index for the patient ID field, store the image files in a distributed file system object storage, and only store the metadata of the images in the graph database.

[0084] Specifically, the structured text data is data with a clear format and fields, usually stored in the form of tables or key-value pairs, including: Basic information: name, gender, age, ID number, contact information, etc. Medical record information: diagnosis results, treatment plans, medication records, surgical records, etc. Test reports: text data such as blood routine, urine routine, and imaging examination results. Follow-up records: follow-up visit time, condition changes, patient feedback, etc.

[0085] The unstructured image data is data that cannot be represented in a fixed format and usually exists in the form of images, videos, etc. It includes: Medical images: X-ray films, CT scans, MRI images, ultrasound images, etc. Pathological sections: microscopic tissue section images. Handwritten medical records: doctor's handwritten medical record records (which need to be digitized). Other images: patient skin lesion photos, wound healing process records, etc.

[0086] Step 3: Preprocess and standardize the structured text data and the unstructured image data respectively to obtain standardized text data and standardized image data as the source of sample data.

[0087] Specifically, in the embodiment of the present application, the step 3: Preprocess and standardize the structured text data and the unstructured image data respectively to obtain standardized text data and standardized image data as the source of sample data, including:

[0088] The preprocessing and standardization of the structured text data include: deduplicating, removing stop words, and annotating the structured text data; converting the text data into a unified format and filling in the missing values by interpolation; using the BERT embedding model to generate the vector representation of the text for subsequent analysis;

[0089] Preprocessing and standardization of unstructured image data, including: rotation, flipping, and cropping: adjusting the orientation and size of the image to meet the model input requirements; noise reduction processing: removing noise in the image to improve image quality; scale transformation: adjusting the resolution and size of the image to ensure a unified aspect ratio and number of pixels; grayscale transformation: converting a color image to a grayscale image to simplify subsequent processing; histogram equalization: adjusting the contrast of the image to make it easier to analyze; mean normalization; mixed pixel value calculation: combining the standard deviation and mean of multiple images to generate a mixed enhanced image.

[0090] Step 4: Screen the mapping relationship between the outcome variable and the predictive variables according to the standardized text data and the Cox regression analysis method, and establish a text machine learning model, and use the sample text data for training and verification.

[0091] Specifically, in the embodiment of the present application, the step 4: Screen the mapping relationship between the outcome variable and the predictive variables according to the standardized text data and the Cox regression analysis method, and establish a text machine learning model, and use the sample text data for training and verification, includes:

[0092] S4.1 Screen the predictive variables through univariate or multivariate Cox regression analysis. Univariate analysis is used to preliminarily screen significant variables, and multivariate analysis further verifies the influence of these variables on the outcome variable;

[0093] S4.2 Use the screened variables to construct a Cox proportional hazards model to estimate the influence of each variable on the outcome variable;

[0094] S4.3 Evaluate the predictive performance of the model through methods such as cross-validation and ROC curves;

[0095] S4.4 After completing the standardization and feature extraction of the text data, use machine learning algorithms to construct a predictive model;

[0096] S4.5 Use Cox regression to screen out significant predictive variables as the input of the machine learning model;

[0097] S4.6 Use the results of Cox regression as part of the machine learning model, as feature weights or baseline predicted values;

[0098] S4.7 Optimize the performance of both Cox regression and the machine learning model simultaneously through multi-objective optimization methods.

[0099] Step 5: Perform image segmentation and feature annotation on the standardized image data, and establish a lesion localization model, and use the sample image data for training and verification.

[0100] Specifically, in the embodiment of the present application, as Figure 2As shown in the figure, it is a flowchart of steps for establishing a lesion localization model provided by an embodiment of the present application. The step 5: perform image segmentation and feature annotation on the standardized image data, and establish a lesion localization model, and use the sample image data for training and verification, including:

[0101] S5.1 Multi-scale feature extraction step: The input image is first subjected to feature extraction through the ResNetSt network, and the network performs 4 times of downsampling through 4 ResNetSt blocks to obtain the multi-scale features of the image;

[0102] S5.2 Channel splicing step: The multi-scale features are spliced with the feature map obtained by upsampling the feature pyramid to obtain the fused features, and the depth of the feature map is increased by stacking multiple feature maps in the channel dimension to retain more information;

[0103] S5.3 Dual-branch feature extraction step: The spliced feature map is sent to a dual-branch network for further processing. One branch uses the cross-attention mechanism to extract global features, and the other branch uses dilated convolutions with different sampling rates to extract local features to capture lesion information at different scales;

[0104] S5.4 Feature fusion and dependence weight learning step: After channel splicing, the global features and local features at multiple scales are further fused through the dual-branch network, so that the global features and local features at each scale can learn different dependence weights to improve the adaptability of the model to complex scenarios;

[0105] S5.5 Lesion classification prediction step. Figure 3 It is a schematic diagram of a multi-modal risk prediction model for avascular necrosis of the femoral head provided by an embodiment of the present application. As Figure 3 shown, a Deeplab3+ semantic segmentation model is established, and the Squeeze-and-Excitation module is introduced to enhance the attention to the lesion area. Feature extraction and semantic segmentation are performed on the input image data, and each pixel is classified for the lesion through the decoder to obtain the prediction result;

[0106] S5.6 Training and verification step: Establish a lesion localization model based on ResNet and the attention mechanism through the above steps, and use the sample image data for training and verification.

[0107] Among them, a combination of Dice loss and cross-entropy loss is used for training, and the loss function is:

[0108]

[0109] Y i represents the class probability that the i-th class is the true label, It represents the probability predicted by the model for the i-th category, N represents the total number of categories, and α represents the weight adjustment coefficient;

[0110] The model evaluation formula is:

[0111]

[0112] Among them, S ij represents the number of samples in the m-th category predicted as the n-th category.

[0113] Step 6: Integrate the text machine learning model and the image recognition model to obtain a multi-modal risk prediction model for avascular necrosis of the femoral head, and re-train and validate it using the sample text data and image data.

[0114] Specifically, in the embodiment of the present application, the Step 6: Integrate the text machine learning model and the image recognition model to obtain a multi-modal risk prediction model for avascular necrosis of the femoral head, and re-train and validate it using the sample text data and image data, includes:

[0115] S6.1 Feature fusion step: Map the text features and image features to the same feature space through the methods of joint embedding and contrast learning;

[0116] S6.2 Model fusion step: Integrate the outputs of the text model and the image model through a multi-modal fusion network in deep learning;

[0117] S6.3 Training and validation step: Re-train using the sample text data and image data, and verify the model performance through the ROC curve and AUC value metrics.

[0118] Model verification: (1) ROC curve, used to calculate the true positive rate (TPR) and false positive rate (FPR) of the model at different thresholds. Draw the ROC curve, with the FPR on the horizontal axis and the TPR on the vertical axis. (2) AUC value, calculate the area under the ROC curve (AUC value), and the closer the AUC value is to 1, the better the model performance. (3) Other metrics, including calculating accuracy, precision, recall, and F1 score.

[0119] Step 7: According to the patient's clinical data and the multi-modal risk prediction model for avascular necrosis of the femoral head, predict the location of the avascular necrosis lesion and the staging diagnosis result of the patient, and generate a corresponding risk assessment report.

[0120] Specifically, in the embodiments of the present application, the staging of avascular necrosis of the femoral head usually adopts the staging system of the Association Research Circulation Osseous (ARCO), which is divided into stages 0 to IV: Stage 0: Asymptomatic, no abnormalities on X-ray and MRI, and osteonecrosis can be seen histologically. Stage I: Asymptomatic, normal X-ray, and MRI shows trabecular bone changes without subchondral fracture. Stage II: Symptomatic, X-ray shows trabecular bone changes, MRI shows typical manifestations, and no subchondral fracture. Stage III: Symptomatic, X-ray shows subchondral fracture (semicircular sign), MRI shows typical manifestations, and the shape of the femoral head remains intact. Stage IV: Symptomatic, X-ray shows collapse of the femoral head and narrowing of the joint space.

[0121] Based on the patient's clinical data and imaging examination results, the following features can be combined for prediction: Lesion location: MRI can clearly show the specific location of the lesion, such as whether there are scattered sparse areas, blurred trabecular bone, or stellate spicule signs in the femoral head. Staging: By comparing the imaging manifestations of MRI and CT, combined with the patient's symptoms and signs, the staging of the lesion can be initially judged. For example, if MRI shows abnormal bone marrow edema signal and no subchondral fracture, it may be stage I; if MRI shows a typical double-line sign and there is a subchondral fracture, it may be stage III.

[0122] The risk assessment report should include the following contents: Imaging examination results: Features such as bone marrow edema signal, double-line sign, and subchondral fracture shown by MRI examination. Bone structure changes shown by CT examination (such as blurred trabecular bone, stellate spicule sign). Staging diagnosis: Such as according to the ARCO staging system, combined with imaging manifestations and clinical symptoms, to clarify the staging of the lesion. Treatment suggestions: Early stage (stage I and II): Conservative treatment is recommended, such as physical therapy, non-steroidal anti-inflammatory drugs, etc. Middle and late stage (stage III and IV): Surgical intervention is recommended, such as bone marrow stimulation or artificial joint replacement. Prognosis assessment: According to the staging and treatment plan, predict the patient's prognosis.

[0123] Figure 4 Schematic diagram of a module of a device for predicting the risk of avascular necrosis of the femoral head based on machine learning provided by an embodiment of the present application. As Figure 4 shown, a method and device for predicting the risk of avascular necrosis of the femoral head based on machine learning, the system includes a data acquisition module 11, a data division module 12, a data standardization module 13, a feature screening module 14, a feature segmentation module 15, a model construction module 16, and a risk assessment module 17 connected in sequence, where:

[0124] It can be understood that the data acquisition module 11 is used to collect patient-related information in the local database, including: personal basic information, electronic medical records, and examination image data;

[0125] It can be understood that the data partitioning module 12 is used to classify patient-related information into structured text data and unstructured image data, and store them correspondingly using a graph database;

[0126] It can be understood that the data standardization module 13 is used to perform preprocessing and standardization on the structured text data and unstructured image data respectively to obtain standardized text data and standardized image data as the source of sample data;

[0127] It can be understood that the feature screening module 14 is used to screen the mapping relationship between the outcome variable and the predictive variable according to the standardized text data and the Cox regression analysis method, establish a text machine learning model, and use the sample text data for training and verification;

[0128] It can be understood that the feature segmentation module 15 is used to perform image segmentation and feature annotation on the standardized image data, establish a lesion localization model, and use the sample image data for training and verification;

[0129] It can be understood that the model construction module 16 is used to fuse the text machine learning model and the image recognition model to obtain a multimodal risk prediction model for avascular necrosis of the femoral head, and use the sample text data and image data for retraining and verification;

[0130] It can be understood that the risk assessment module 17 is used to predict the location and staging diagnosis results of the avascular necrosis lesions of the patient according to the patient's clinical data and the multimodal risk prediction model for avascular necrosis of the femoral head, and generate a corresponding risk assessment report.

[0131] See Figure 5 , Figure 5 which is the electronic terminal device 500 provided by an embodiment of the present application. As Figure 5 shown, the electronic terminal device 500 at least includes the following parts: one or more processors 501, one or more input devices 502, one or more output devices 503, and one or more memories 504. The above-mentioned processors 501, input devices 502, output devices 503, and memories 504 communicate with each other through the communication bus 505. The memory 504 is used to store computer programs, and the computer programs include program instructions. The processor 501 is used to execute the program instructions stored in the memory 504. Among them, the processor 501 is configured to call the program instructions to perform the functions of each module / unit in the above-mentioned device embodiments, such as Figure 1 the functions of the modules shown.

[0132] In the embodiment of the present application, a computer-readable storage medium includes instructions that direct the device to execute the system as described in the first aspect. For example, the instructions direct the device to execute asFigure 1 A method and apparatus for predicting the risk of avascular necrosis of the femoral head based on machine learning shown in the steps

[0133] It should be understood that in the embodiments of the present invention, the so-called processor 501 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. It should be noted that a part of the electronic device 500 in the above embodiments may also be implemented by a computer. In this case, the program for implementing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into the computer and executed to implement it.

[0134] The input device 502 may include a touchpad, a fingerprint collection sensor (for collecting the fingerprint information and the direction information of the fingerprint) of a user, a microphone, etc., and the output device 503 may include a display (such as an LCD), a speaker, etc.

[0135] It should be noted that the "computer" mentioned here refers to the computer built in the electronic device 500, which is a computer adopting hardware including an OS, peripheral devices, etc. In addition, the "computer-readable recording medium" refers to removable media such as a floppy disk, a magneto-optical disk, a ROM, a CD-ROM, etc., and storage devices such as a hard disk built in the computer.

[0136] Moreover, the "computer-readable recording medium" may include: a medium such as a communication line that dynamically stores a program in a short time like a communication line in the case of transmitting a program via a network such as the Internet or a communication line such as a telephone line; a medium such as a volatile memory inside a computer of a server or a client in this case that stores a program within a fixed time. In addition, the above program may be a part of the program for implementing the above functions, and may also be a program that can implement the above functions by combining with a program already recorded in the computer.

[0137] In addition, the electronic device 500 in the above-described embodiment can also be implemented as an aggregate (device group) composed of a plurality of devices. Each device constituting the device group may include some or all of the functions or functional blocks of the electronic device 500 in the above-described embodiment. As the device group, it is sufficient to have all the functions or functional blocks of the electronic device 500.

[0138] Those of ordinary skill in the art in this technical field should recognize that the above embodiments are only used to illustrate the present application and are not intended to limit the present application. As long as appropriate changes and variations made to the above embodiments fall within the scope of the spirit of the present application, they fall within the scope of protection required by the present application.

Claims

1. A method for predicting the risk of avascular necrosis of the femoral head based on machine learning, characterized in that, The method includes: Step 1: Collect patient-related information in the local database, including: personal basic information, electronic medical records, and examination image data; Step 2: Classify the patient-related information into structured text data and unstructured image data, and store them correspondingly using a graph database; Step 3: Perform preprocessing and standardization on the structured text data and unstructured image data respectively to obtain standardized text data and standardized image data as the source of sample data; Step 4: Screen the mapping relationship between the outcome variable and the predictive variable according to the standardized text data and the Cox regression analysis method, establish a text machine learning model, and use the sample text data for training and verification; Step 5: Perform image segmentation and feature annotation on the standardized image data, establish a lesion localization model, and use the sample image data for training and verification; Step 6: Integrate the text machine learning model and the image recognition model to obtain a multi-modal risk prediction model for avascular necrosis of the femoral head, and use the sample text data and image data for retraining and verification; Step 7: Predict the location of the avascular necrosis lesion and the staging diagnosis result of the patient according to the patient's clinical data and the multi-modal risk prediction model for avascular necrosis of the femoral head, and generate a corresponding risk assessment report.

2. The method for predicting the risk of avascular necrosis of the femoral head based on machine learning according to claim 1, characterized in that, The said Step 2: Classify the patient-related information into structured text data and unstructured image data, and store them correspondingly using a graph database, including: S2.1 Store the structured text data in the form of key-value pairs, and store the unstructured image data in the form of images or videos; S2.2 Establish an entity node set, and the entity node set includes: patient node, medical record node, test node, and image node; S2.3 Establish a relationship set, and the relationship set includes: patient-medical record relationship, patient-test relationship, patient-image relationship, medical record-test relationship, and medical record-image relationship; S2.4 Create an index for the patient ID field, store the image file in the distributed file system object storage, and only store the metadata of the image in the graph database.

3. The method for predicting the risk of avascular necrosis of the femoral head based on machine learning according to claim 1, wherein, The said Step 3: Perform preprocessing and standardization on the structured text data and unstructured image data respectively to obtain standardized text data and standardized image data as the source of sample data, including: Preprocessing and standardization of structured text data, including: deduplicating, removing stop words, and annotating the structured text data; converting the text data into a unified format, filling in missing values by interpolation method; using the BERT embedding model to generate the vector representation of the text for subsequent analysis; Preprocessing and standardization of unstructured image data, including: rotation, flipping, and cropping: adjusting the orientation and size of the image to meet the model input requirements; noise reduction processing: removing noise in the image to improve image quality; scale transformation: adjusting the resolution and size of the image to ensure that the image has a unified aspect ratio and number of pixels; grayscale transformation: converting a color image to a grayscale image to simplify subsequent processing; histogram equalization: adjusting the contrast of the image to make it easier to analyze; mean normalization; mixed pixel value calculation: combining the standard deviation and mean of multiple images to generate a mixed enhanced image.

4. A method for predicting the risk of avascular necrosis of the femoral head based on machine learning according to claim 3, characterized in that, Step 4: Screen the mapping relationship between the outcome variable and the predictive variable according to the standardized text data and the Cox regression analysis method, and establish a text machine learning model, which is trained and verified using the sample text data, including: S 4.1 Screen the predictive variables through univariate or multivariate Cox regression analysis. Univariate analysis is used to preliminarily screen significant variables, and multivariate analysis further verifies the influence of these variables on the outcome variable; S 4.2 Use the screened variables to construct a Cox proportional hazards model to estimate the influence of each variable on the outcome variable; S 4.3 Evaluate the predictive performance of the model through methods such as cross-validation and ROC curve; S 4.4 After completing the standardization and feature extraction of the text data, use machine learning algorithms to construct a predictive model; S 4.5 Use Cox regression to screen out significant predictive variables as the input of the machine learning model; S 4.6 Use the results of Cox regression as part of the machine learning model, as feature weights or baseline predicted values; S 4.7 Optimize the performance of Cox regression and the machine learning model simultaneously through multi-objective optimization methods.

5. A method for predicting the risk of avascular necrosis of the femoral head based on machine learning according to claim 4, characterized in that Step 5: Perform image segmentation and feature annotation on the standardized image data, and establish a lesion localization model, which is trained and verified using the sample image data, including: S 5.1 Multi-scale feature extraction step: The input image is first subjected to feature extraction through the ResNetSt network, which performs 4 times of downsampling through 4 ResNetSt blocks to obtain the multi-scale features of the image; S 5.2 Channel splicing step: The multi-scale features are spliced with the feature maps obtained by upsampling the feature pyramid to obtain fused features. By stacking multiple feature maps in the channel dimension, the depth of the feature map is increased to retain more information; S 5.3 Dual-branch feature extraction step: The spliced feature map is fed into a dual-branch network for further processing. One branch uses the cross-attention mechanism to extract global features, and the other branch uses dilated convolutions with different sampling rates to extract local features to capture lesion information at different scales; S 5.4 Feature fusion and dependence weight learning step: After channel splicing, the global features and local features at multiple scales are further fused through the dual-branch network, so that the global features and local features at each scale can learn different dependence weights, improving the model's adaptability to complex scenarios; Step S 5.5 for predicting lesion classification: Establish a Deeplab3+ semantic segmentation model, introduce the Squeeze-and-Excitation module to enhance the attention to the lesion area, perform feature extraction and semantic segmentation on the input image data, and classify the lesions for each pixel through the decoder to obtain the prediction result; Step S 5.6 for training and validation: Establish a lesion localization model based on ResNet and the attention mechanism through the above steps, and use the sample image data for training and validation.

6. A method for predicting the risk of avascular necrosis of the femoral head based on machine learning according to claim 5, wherein Among them, Use a combination of Dice loss and cross-entropy loss for training, and the loss function is: Y i represents the class probability that the i-th class is the true label represents the probability that the model predicts the i-th class, N represents the total number of classes, and α represents the weight adjustment coefficient; The model evaluation formula is: Among them, S ij represents the number of samples in which the m-th class of samples is predicted as the n-th class.

7. The method for predicting the risk of avascular necrosis of the femoral head based on machine learning according to claim 6, wherein Step 6: Integrate the text machine learning model and the image recognition model to obtain a multi-modal risk prediction model for avascular necrosis of the femoral head, and use the sample text data and image data for retraining and validation, including: Step S 6.1 for feature integration: Map the text features and image features to the same feature space through the methods of joint embedding and contrast learning; Step S 6.2 for model integration: Integrate the outputs of the text model and the image model through a multi-modal fusion network in deep learning; Step S 6.3 for training and validation: Use the sample text data and image data for retraining, and verify the model performance through the ROC curve and AUC value metrics.

8. A risk prediction device for avascular necrosis of the femoral head based on machine learning, which is applied to a risk prediction method for avascular necrosis of the femoral head based on machine learning according to any one of claims 1 to 7, characterized in that, The system includes: A data acquisition module for collecting patient-related information in the local database, including: personal basic information, electronic medical records, and examination image data; A data partitioning module for classifying the patient-related information into structured text data and unstructured image data, and storing them correspondingly using a graph database; A data standardization module for preprocessing and standardizing the structured text data and unstructured image data respectively to obtain standardized text data and standardized image data as the source of sample data; A feature screening module for screening the mapping relationship between the outcome variable and the prediction variable according to the standardized text data and the Cox regression analysis method, and establishing a text machine learning model, and using the sample text data for training and validation; A feature segmentation module for performing image segmentation and feature annotation on the standardized image data, and establishing a lesion localization model, and using the sample image data for training and validation; A model construction module for integrating the text machine learning model and the image recognition model to obtain a multi-modal risk prediction model for avascular necrosis of the femoral head, and using the sample text data and image data for retraining and validation; A risk assessment module for predicting the location and staging diagnosis results of the avascular necrosis lesions of the patient according to the patient's clinical data and the multi-modal risk prediction model for avascular necrosis of the femoral head, and generating a corresponding risk assessment report.

9. An electronic device, characterized in that, Including: A processor; A memory for storing instructions executable by the processor; Wherein, when the processor is configured to execute the instructions, it implements a method for predicting the risk of avascular necrosis of the femoral head based on machine learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program, and the program instructs the device to execute a method for predicting the risk of avascular necrosis of the femoral head based on machine learning as described in any one of claims 1 to 7.

Images