A clinical decision support system and device for femoral head necrosis

Abstract

The present invention provides a clinical decision support system and device for femoral head necrosis, which relates to the technical field of clinical decision support. The system mainly includes a data receiving module, a data processing module and a result generating module; the data receiving module is used to receive the patient's medical record data; the data processing module includes a medical image segmentation and classification unit, a text encoding unit and a clinical decision support unit; the medical image segmentation and classification unit is used to extract lesion segmentation images from femoral head medical image information and predict femoral head necrosis classification results; the text encoding unit is used to identify the patient's structured features from the femoral head necrosis classification results and text information; the clinical decision support unit predicts and obtains diagnosis and treatment plans and decision evidence based on the structured features and lesion segmentation images; this solution can automatically, quickly and reliably obtain corresponding diagnosis and treatment plans, thereby providing standardized and homogenized clinical decision support for orthopedic surgeons of different years and different hospitals.

Description

Technical Field

[0001] The present invention relates to the technical field of clinical decision support, and in particular to a clinical decision support system and device for femoral head necrosis. Background Art

[0002] Osteonecrosis of the femoral head (ONFH) is a common, intractable and disabling orthopedic disease.

[0003] At present, orthopedic surgeons of different seniority and from different hospitals often lack comprehensive clinical knowledge and rich clinical experience, resulting in cases of femoral head necrosis not receiving high-quality and homogeneous medical services, and even causing medical quality problems such as misdiagnosis and missed diagnosis during consultations.

[0004] Therefore, developing a clinical decision support system (CDSS) for femoral head necrosis to provide relevant medical personnel with reliable and timely diagnosis and treatment references and assistance, thereby improving the treatment quality of femoral head necrosis cases, has become a major issue that needs to be urgently addressed in the industry. Summary of the Invention

[0005] The purpose of the present invention is to provide a clinical decision support system and device for femoral head necrosis to solve at least one of the above-mentioned technical problems existing in the prior art.

[0006] In the first aspect, in order to solve the above technical problems, the present invention provides a clinical decision support system for femoral head necrosis, comprising a data receiving module, a data processing module and a result generating module;

[0007] The data receiving module is used to receive the patient's medical record data; the medical record data includes text information and femoral head medical imaging information; the text information includes demographic information, chief medical history, outpatient diagnosis, test results, drug prescriptions and surgical records; the femoral head medical imaging information includes MRI (magnetic resonance imaging) images and CT (X-ray scan) images; the MRI image is characterized in that it can generate images of human body sections in all directions, can distinguish human tissue details with small differences in density, and has low sensitivity to detecting calcifications and cortical bone lesions; the CT image is characterized in that it mainly generates scanned images of human body cross sections, can distinguish human tissues with large differences in density, and has high sensitivity to detecting calcifications and cortical bone lesions;

[0008] The data processing module includes a medical image segmentation and classification unit, a text encoding unit, and a clinical decision support unit;

[0009] The medical image segmentation and classification unit stores a medical image segmentation and classification model, and is used to extract lesion segmentation images from the medical image information of the femoral head and predict the classification result of femoral head necrosis;

[0010] The text encoding unit stores a natural language processing (NLP) model for identifying the patient's structural features from the femoral head necrosis classification results and the text information;

[0011] The clinical decision support unit stores a clinical decision support model, and predicts a diagnosis and treatment plan and decision evidence based on the structural features and the lesion segmentation image; the decision evidence includes evidence text and evidence image;

[0012] The result generating unit is used to send out diagnosis and treatment plans and decision-making evidence.

[0013] Through the above system, after inputting the medical records of ONFH patients, the corresponding diagnosis and treatment plans can be obtained automatically, quickly and reliably, thereby providing standardized and homogenized clinical decision support for orthopedic surgeons of different years and different hospitals.

[0014] In a feasible embodiment, the medical image segmentation and classification model is a dual-task deep neural network model that can simultaneously handle the two tasks of lesion segmentation and femoral head necrosis classification. The specific model structure includes two groups of convolutional neural networks; each group of convolutional neural networks includes an input layer, a hidden layer, and an output layer arranged in sequence; the hidden layer includes n convolutional layers, a pooling layer, and a fully connected layer;

[0015] The input layer of the first convolutional neural network receives the MRI image; the input layer of the second convolutional neural network receives the CT image;

[0016] The first convolutional layer of the positive number of the first convolutional neural network is connected to the n convolutional layers of the second convolutional neural network; the second convolutional layer of the positive number of the first convolutional neural network is connected to the n-1 convolutional layers of the reciprocal of the second convolutional neural network; the third convolutional layer of the positive number of the first convolutional neural network is connected to the n-2 convolutional layers of the reciprocal of the second convolutional neural network; and so on, until the nth convolutional layer of the positive number of the first convolutional neural network is connected to the first convolutional layer of the reciprocal of the second convolutional neural network; in this way, the intermediate features of the MRI image and the CT image can be fused, so that the shallow convolution and the deep convolution can be fully and orderly fused, effectively retaining the detailed information of the MRI image and the CT image, thereby bringing together the imaging advantages of the MRI image and the CT image, and effectively compressing the feature dimension, thereby avoiding the problem of excessive dimension caused by splicing and fusion in the existing technology.

[0017] In a feasible implementation, the training step of the medical image segmentation and classification model includes:

[0018] Step a1: Based on the medical imaging information of the femoral head from existing cases, the three-column structure (the coronal plane of the femoral head is divided into the lateral column, central column, and medial column in the three-column structure theory) and the range of the necrotic lesion are delineated at each level of the MRI image;

[0019] Step a2: Calculate the first necrosis ratio of each layer. The specific calculation formula can be:

[0020] The first necrosis ratio = (necrotic area / femoral head area) * 100%;

[0021] Step a3: Select the layer with the largest first necrosis ratio and the mid-femoral head layer in each layer, segment the necrotic lesions along the dividing line of the three columns, and calculate the second necrosis ratio in each column in these two layers. The specific calculation formula can be:

[0022] Second necrosis ratio = (necrotic area in each column / femoral head area in each column) * 100%;

[0023] Step a4: extracting lesion segmentation images from the MRI image and the CT image respectively, based on the range of the necrotic lesion in the first layer with the largest necrosis ratio;

[0024] Step a5: Classify and label the MRI images based on the second necrosis ratio according to the CJFH classification criteria using a threshold determination method;

[0025] The CJFH classification standard refers to the China-Japan Friendship Hospital classification standard for femoral head necrosis, which is based on the JIC classification standard. For patients with ARCO stage I to III femoral head necrosis, T1WI (TR / TE=550 / 18) coronal mid-plane MRI images of the femoral head are used to divide femoral head necrosis into five types according to the location of the necrotic lesions involving the three columns: M type (medial type): the necrotic area involves the medial column, leaving the central column and lateral column; C type (central type): the necrotic area involves the central column and medial column, leaving the lateral column; L1 type (sublateral type): the necrotic area involves the three columns, with part of the lateral column remaining; L2 type (extreme lateral type): the necrotic area involves the lateral column and part of the central column, with part of the central column and medial column remaining; L3 type (total femoral head type): the necrotic area penetrates the entire cortex and bone marrow of the lateral, central and medial columns of the femoral head;

[0026] The threshold determination method is as follows: when the second necrosis ratio in each column is greater than or equal to the first preset threshold, it is determined that the necrotic area penetrates the column; when the second necrosis ratio in each column is less than or equal to the second preset threshold, it is determined that the column remains; when the second necrosis ratio in each column is between the first preset threshold and the second preset threshold, it is determined that the column partially remains;

[0027] Preferably, the first preset threshold is greater than the second preset threshold;

[0028] Preferably, the first preset threshold is 95%, and the second preset threshold is 1%;

[0029] Step a6: randomly divide the femoral head medical imaging information into a training set and a test set according to a preset division ratio; use the lesion segmentation image as the image label and the CJFH classification result as the text label to train and test the medical image segmentation and classification model.

[0030] In a feasible implementation, the natural language processing model is a BERT (Bidirectional Encoder Representation from Transformers) model.

[0031] In one feasible implementation, the BERT model training method includes the following steps:

[0032] Step b1: marking the medical terms in the text information, diagnosis and treatment guidelines, and field literature related to femoral head necrosis in the medical record data of existing cases;

[0033] Step b2: randomly divide text information, diagnosis and treatment guidelines, and field literature into training and test sets according to a preset division ratio; train and test the BERT model;

[0034] Step b3: normalize and map the medical term entities identified by the BERT model into standard medical terminology; the standard medical terminology includes gender, age group, symptoms, test items, diseases, drugs, and surgeries;

[0035] Step b4: perform relationship extraction, graph cleaning, and entity sorting on medical standard terms to obtain a medical knowledge graph, which is used to identify structured features related to medicine in text information.

[0036] In a feasible embodiment, the clinical decision support model includes a factor extraction model and a decision tree model; the factor extraction model is used to extract the patient's structural features and diagnostic and treatment factors in the lesion segmentation image; the decision tree model makes a diagnosis and treatment plan decision based on the diagnostic and treatment factors.

[0037] In a feasible implementation, the feature extraction model includes a residual network model, a Transformer, a CRF model (conditional random field), a DETR model (End-to-End Object Detection with Transformers, a Transformer-based target detection model) and a coreference elimination model;

[0038] The residual network model is used to extract image representations of relevant medical knowledge graphs from lesion segmentation images;

[0039] The Transformer is used to interact with structured features and image representations to obtain a fused representation of text modality and image modality. The Transformer belongs to the prior art and is a model based on an encoder-decoder structure. It uses a self-attention mechanism to enable parallel training and has global information.

[0040] The CRF model is used to perform sequence labeling on the text representation in the fusion representation and extract a set of text elements;

[0041] The DETR model is used to perform target detection on the image representation in the fusion representation to obtain an image entity set;

[0042] The co-reference elimination model is based on a set of text elements and a set of image entities. It uses the cosine distance function to calculate the similarity of the elements in the text and the entities in the image one by one to obtain a matching value. When the matching value is greater than the preset matching threshold, the corresponding text elements and image entities are input into the diagnosis and treatment element extraction set.

[0043] In a feasible implementation, the extraction method of the factor extraction model specifically includes:

[0044] Step c1: input the lesion segmentation image into the residual network model and output the image representation;

[0045] Step c2: The patient's structural features and image representation are input into the first linear layer respectively, projected into the same vector space and then spliced. The concatenation is then input into the Transformer for information interaction, and the fusion representation of the text modality and the image modality is output.

[0046] Step c3: Input the text representation in the fusion representation into the second linear layer and convert it into a spectrogram to obtain a representation vector A. The dimension of the representation vector A can be the disease category n. The specific expression can be:

[0047] A=(a1,a2,...,a n );

[0048] Among them, a represents the probability matrix of each disease category label corresponding to each frame of the spectrum graph;

[0049] Through the CRF model, the representation vector A is sequence labeled to extract the text feature set;

[0050] Step c4: Input the image representation in the fused representation into the DETR model for target detection and output a set of image entities; the DETR model adds a feedforward neural network (FFN) and a Hungarian loss function after each decoder layer of the Transformer; the feedforward neural network is used to predict the category and position of each query object in the decoder output, including a shared layer; the shared layer is used to standardize the input value of each feedforward neural network through the shared layer norm, thereby achieving standardization; the Hungarian loss function is used to calculate the minimum loss between the predicted value and the paired label during model training and return the gradient;

[0051] Step c5: Input the text element set and the image entity set into the coreference elimination model. Use the cosine distance function to calculate the similarity of the elements in the text and the entities in the image one by one, and output the matching value Ma. When the matching value Ma is greater than the preset matching threshold, the corresponding text element and image entity are input into the diagnosis and treatment element extraction set. The specific formula can be:

[0052] Ma=(t·i) / (||t||*||i||);

[0053] Among them, t represents the element vector in the text; i represents the entity vector in the image; ||t|| represents the norm of t; ||i|| represents the norm of i; in this way, the coreference information in text elements and image entities can be unified.

[0054] In a feasible implementation, the specific method of performing sequence labeling in step c3 includes:

[0055] Step c31: Add the start frame B and the end frame E at the beginning and end of the characterization vector A respectively to define the standard data frame. The specific expression can be:

[0056] A=(B,a1,a2,...,a n ,E);

[0057] Step c32: define the disease category label sequence G. The specific expression can be:

[0058] G=(B,g1,g2,...,g n ,E);

[0059] Among them, g represents each disease category label;

[0060] Step c33, define the loss function , in order to train the CRF model by gradient descent, the specific formula can be:

[0061] ;

[0062] in, Represents the path score of a sequence, specifically the sum of the emission score (EmissionScore) and the transition score (TransitionScore); Indicates the score conversion value of the sequence path; The score conversion value of the correct sequence path, that is, the score conversion value of the sequence path with the highest score among all sequence paths, corresponds to the prediction result of the CRF model; Represents the total number of paths; as the gradient continues to decrease, the proportion of the correct sequence path score to the sum of all sequence path scores will gradually increase, making the prediction effect of the CRF model more and more accurate;

[0063] Step c34: After concatenating the vector representation of the disease category with the text representation, sequence labeling is performed using a conditional random field to obtain a set of disease element entities and corresponding element trends as a text element set.

[0064] In a feasible embodiment, the decision tree model includes a knowledge graph of femoral head necrosis and a decision logic diagram; the knowledge graph of femoral head necrosis can be obtained by experienced experts in the field of orthopedics based on the diagnosis and treatment guidelines, defining the fields related to clinical decisions on femoral head necrosis in the medical knowledge graph; the decision logic diagram can define each decision node, treatment branch and diagnosis and treatment plan in a mind map manner; the diagnosis and treatment plans include core medullary decompression, impact bone grafting, tantalum rod implantation, pedicled / non-pedicled fibula implantation, rotational osteotomy at the base of the femoral neck and fine needle decompression, etc.; in this way, a decision tree model with clear evidence source, easy to understand and clear diagnosis and treatment plan can be obtained.

[0065] In a feasible embodiment, the decision tree model can also embed a machine learning model at the preset decision nodes in several treatment branches to predict treatment plans, so as to solve complex problems such as new treatment branches not covered by the treatment guidelines, multiple treatment branches appearing at a decision node, and the branch decision of the decision node relying on multiple features; the machine learning model can be a supervised model such as logistic regression, support vector machine or neural network; this can avoid problems such as slow computing speed and high computing bandwidth pressure caused by the global use of large models such as neural networks.

[0066] In one feasible implementation, the training process of the machine learning model includes:

[0067] Step d1: Based on the text information of the medical record data of the existing case, mark the structural features of the current decision node and the corresponding diagnosis and treatment plan;

[0068] Step d2: randomly divide the text information into a training set and a test set according to a preset division ratio; and train and test the machine learning model.

[0069] In one feasible implementation, the system uses Spark big data technology to achieve distributed processing, enabling flexible big data operations and interactions across different hospitals, devices, and business systems. Spark big data technology is a state-of-the-art big data processing framework built around speed, ease of use, and complex analysis. Compared to Hadoop, Storm, and MapReduce, it offers diverse data types, faster execution, and better compatibility.

[0070] Secondly, based on the same inventive concept, the present application also provides a clinical decision support device for femoral head necrosis, including a processor, a memory and a bus, wherein the memory stores instructions and data that can be read by the processor, and the processor is used to call the instructions and data in the memory to implement the system as described above, and the bus connects the functional components for transmitting information.

[0071] By adopting the above technical solution, the present invention has the following beneficial effects:

[0072] The present invention provides a clinical decision support system and device for femoral head necrosis. By integrating text information such as outpatient medical records and test reports with medical imaging information of the femoral head, a clinical decision support system with the characteristics of multiple data sources, multiple data modes and multiple models is constructed. It can collect and learn various aspects of knowledge such as field experts, case databases, diagnosis and treatment guidelines and field literature. Through this solution, corresponding diagnosis and treatment plans can be obtained automatically, quickly and reliably, thereby providing standardized and homogenized clinical decision support for orthopedic surgeons of different years and different hospitals; at the same time, this solution can also reduce the problems of slow system operation speed and high bandwidth pressure. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0074] Figure 1 A diagram of a clinical decision support system for femoral head necrosis provided by an embodiment of the present invention;

[0075] Figure 2 A schematic diagram of the principle of the medical image segmentation and classification model provided by an embodiment of the present invention;

[0076] Figure 3 This is a diagram illustrating the theoretical structure of the femoral head with three columns in the prior art;

[0077] Figure 4 This is an example diagram of the CJFH classification standard in the prior art;

[0078] Figure 5 A schematic diagram of a factor extraction model provided by an embodiment of the present invention;

[0079] Figure 6 A schematic diagram of a decision tree model provided by an embodiment of the present invention;

[0080] Figure 7 A schematic diagram of a decision tree model embedded in a machine learning model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0081] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0082] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0083] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0084] The present invention will be further explained below with reference to specific embodiments.

[0085] It should also be noted that the following specific embodiments or specific implementations are a series of optimized settings listed in the present invention to further explain the specific content of the invention, and these settings can be combined or used in association with each other.

[0086] Example 1:

[0087] like Figure 1 As shown, this embodiment provides a clinical decision support system for femoral head necrosis, including a data receiving module, a data processing module and a result generating module;

[0088] The data receiving module is used to receive the patient's medical record data; the medical record data includes text information and femoral head medical imaging information; the text information includes demographic information, chief medical history, outpatient diagnosis, test results, drug prescriptions and surgical records; the femoral head medical imaging information includes MRI (magnetic resonance imaging) images and CT (X-ray scan) images; the MRI image is characterized in that it can generate images of human body sections in all directions, can distinguish human tissue details with small differences in density, and has low sensitivity to detecting calcifications and cortical bone lesions; the CT image is characterized in that it mainly generates scanned images of human body cross sections, can distinguish human tissues with large differences in density, and has high sensitivity to detecting calcifications and cortical bone lesions;

[0089] The data processing module includes a medical image segmentation and classification unit, a text encoding unit, and a clinical decision support unit;

[0090] The medical image segmentation and classification unit stores a medical image segmentation and classification model, and is used to extract lesion segmentation images from the medical image information of the femoral head and predict the classification result of femoral head necrosis;

[0091] The text encoding unit stores a natural language processing (NLP) model for identifying the patient's structural features from the femoral head necrosis classification results and the text information;

[0092] The clinical decision support unit stores a clinical decision support model, and predicts a diagnosis and treatment plan and decision evidence based on the structural features and the lesion segmentation image; the decision evidence includes evidence text and evidence image;

[0093] The result generating unit is used to send out diagnosis and treatment plans and decision-making evidence.

[0094] Through the above system, after inputting the medical records of ONFH patients, the corresponding diagnosis and treatment plans can be obtained automatically, quickly and reliably, thereby providing standardized and homogenized clinical decision support for orthopedic surgeons of different years and different hospitals.

[0095] Furthermore, if Figure 2 As shown in the figure, the medical image segmentation and classification model is a dual-task deep neural network model that can simultaneously handle two tasks: lesion segmentation and femoral head necrosis classification. The specific model structure includes two groups of convolutional neural networks; each group of convolutional neural networks includes an input layer, a hidden layer, and an output layer arranged in sequence; the hidden layer includes n convolutional layers, a pooling layer, and a fully connected layer;

[0096] The input layer of the first convolutional neural network receives the MRI image; the input layer of the second convolutional neural network receives the CT image;

[0097] The first convolutional layer of the positive number of the first convolutional neural network is connected to the n convolutional layers of the second convolutional neural network; the second convolutional layer of the positive number of the first convolutional neural network is connected to the n-1 convolutional layers of the reciprocal of the second convolutional neural network; the third convolutional layer of the positive number of the first convolutional neural network is connected to the n-2 convolutional layers of the reciprocal of the second convolutional neural network; and so on, until the nth convolutional layer of the positive number of the first convolutional neural network is connected to the first convolutional layer of the reciprocal of the second convolutional neural network; in this way, the intermediate features of the MRI image and the CT image can be fused, so that the shallow convolution and the deep convolution can be fully and orderly fused, effectively retaining the detailed information of the MRI image and the CT image, thereby bringing together the imaging advantages of the MRI image and the CT image, and effectively compressing the feature dimension, thereby avoiding the problem of excessive dimension caused by splicing and fusion in the existing technology.

[0098] Furthermore, the training steps of the medical image segmentation and classification model include:

[0099] Step a1: Based on the medical imaging information of the femoral head of the existing case, the three-column range and the range of the necrotic focus at each level of the MRI image are delineated, such as Figure 3 As shown;

[0100] Step a2: Calculate the first necrosis ratio of each layer. The specific calculation formula can be:

[0101] The first necrosis ratio = (necrotic area / femoral head area) * 100%;

[0102] Step a3: Select the layer with the largest first necrosis ratio and the mid-femoral head layer in each layer, segment the necrotic lesions along the dividing line of the three columns, and calculate the second necrosis ratio in each column in these two layers. The specific calculation formula can be:

[0103] Second necrosis ratio = (necrotic area in each column / femoral head area in each column) * 100%;

[0104] Step a4: extracting lesion segmentation images from the MRI image and the CT image respectively, based on the range of the necrotic lesion in the first layer with the largest necrosis ratio;

[0105] Step a5: Classify and label the MRI images based on the second necrosis ratio according to the CJFH classification criteria using a threshold determination method;

[0106] The CJFH classification standard refers to the classification standard of femoral head necrosis of the China-Japan Friendship Hospital, such as Figure 4 As shown in the figure, this classification standard is based on the JIC classification standard. For patients with ARCO stage I to III femoral head necrosis, T1WI (TR / TE=550 / 18) coronal mid-plane MRI images of the femoral head are used to divide femoral head necrosis into five types according to the location of the necrotic lesions involving the three columns: M type (medial type): the necrotic area involves the medial column, leaving the central column and lateral column; C type (central type): the necrotic area involves the central column and medial column, leaving the lateral column; L1 type (sublateral type): the necrotic area involves the three columns, with part of the lateral column remaining; L2 type (extreme lateral type): the necrotic area involves the lateral column and part of the central column, with part of the central column and medial column remaining; L3 type (total femoral head type): the necrotic area penetrates the entire cortex and bone marrow of the lateral, central and medial columns of the femoral head;

[0107] The threshold determination method is as follows: when the second necrosis ratio in each column is greater than or equal to the first preset threshold, it is determined that the necrotic area penetrates the column; when the second necrosis ratio in each column is less than or equal to the second preset threshold, it is determined that the column remains; when the second necrosis ratio in each column is between the first preset threshold and the second preset threshold, it is determined that the column partially remains;

[0108] Preferably, the first preset threshold is greater than the second preset threshold;

[0109] Preferably, the first preset threshold is 95%, and the second preset threshold is 1%;

[0110] Step a6: randomly divide the femoral head medical imaging information into a training set and a test set according to a preset division ratio; use the lesion segmentation image as the image label and the CJFH classification result as the text label to train and test the medical image segmentation and classification model.

[0111] Furthermore, the natural language processing model is a BERT model.

[0112] Furthermore, the training method of the BERT model includes the following steps:

[0113] Step b1: marking the medical terms in the text information, diagnosis and treatment guidelines, and field literature related to femoral head necrosis in the medical record data of existing cases;

[0114] Step b2: randomly divide text information, diagnosis and treatment guidelines, and field literature into training and test sets according to a preset division ratio; train and test the BERT model;

[0115] Step b3: normalize and map the medical term entities identified by the BERT model into standard medical terminology; the standard medical terminology includes gender, age group, symptoms, test items, diseases, drugs, and surgeries;

[0116] Step b4: perform relationship extraction, graph cleaning, and entity sorting on medical standard terms to obtain a medical knowledge graph, which is used to identify structured features related to medicine in text information.

[0117] Furthermore, the clinical decision support model includes a factor extraction model and a decision tree model; the factor extraction model is used to extract the patient's structural features and diagnostic and treatment factors in the lesion segmentation image; the decision tree model makes diagnostic and treatment plan decisions based on the diagnostic and treatment factors.

[0118] Further, if Figure 5 As shown, the feature extraction model includes a residual network model, a Transformer, a CRF model, a DETR model and a coreference elimination model;

[0119] The residual network model is used to extract image representations of relevant medical knowledge graphs from lesion segmentation images;

[0120] The Transformer is used to interact with structured features and image representations to obtain a fused representation of text modality and image modality. The Transformer belongs to the prior art and is a model based on an encoder-decoder structure. It uses a self-attention mechanism to enable parallel training and has global information.

[0121] The CRF model is used to perform sequence labeling on the text representation in the fusion representation and extract a set of text elements;

[0122] The DETR model is used to perform target detection on the image representation in the fusion representation to obtain an image entity set;

[0123] The co-reference elimination model is based on a set of text elements and a set of image entities. It uses the cosine distance function to calculate the similarity of the elements in the text and the entities in the image one by one to obtain a matching value. When the matching value is greater than the preset matching threshold, the corresponding text elements and image entities are input into the diagnosis and treatment element extraction set.

[0124] Furthermore, the extraction method of the factor extraction model specifically includes:

[0125] Step c1: input the lesion segmentation image into the residual network model and output an image representation; for example, the image representation may include "bone of middle-aged and elderly people", "late stage necrosis of the central column and medial column", etc.;

[0126] Step c2: The patient's structural features and image representation are input into the first linear layer respectively, projected into the same vector space and then spliced. The concatenation is then input into the Transformer for information interaction, and the fusion representation of the text modality and the image modality is output.

[0127] Step c3: Input the text representation in the fusion representation into the second linear layer and convert it into a spectrogram to obtain a representation vector A. The dimension of the representation vector A can be the disease category n. The specific expression can be:

[0128] A=(a1,a2,...,a n );

[0129] Among them, a represents the probability matrix of each disease category label corresponding to each frame of the spectrum graph;

[0130] Using the CRF model, we perform sequence annotation on the representation vector A and extract a set of text elements. For example, in the text "Excessive drinking leads to a rapid expansion of the scope of type C middle-aged femoral head necrosis," the element entities include "alcohol," "femoral head," "middle-aged," and "scope of necrosis." The disease category is "type C middle-aged alcoholic femoral head necrosis," and the element trend is "expansion."

[0131] Step c4: Input the image representation in the fusion representation into the DETR model for target detection and output a set of image entities; the DETR model adds a feedforward neural network and a Hungarian loss function after each decoder layer of the Transformer; the feedforward neural network is used to predict the category and position of each query object in the decoder output result, including a shared layer; the shared layer is used to standardize the input value of each feedforward neural network through the shared layer norm, thereby achieving standardization processing; the Hungarian loss function is used to calculate the minimum loss between the predicted value and the paired label and return the gradient during model training;

[0132] Step c5: Input the text element set and the image entity set into the coreference elimination model. Use the cosine distance function to calculate the similarity of the elements in the text and the entities in the image one by one, and output the matching value Ma. When the matching value Ma is greater than the preset matching threshold, the corresponding text element and image entity are input into the diagnosis and treatment element extraction set. The specific formula can be:

[0133] Ma=(t·i) / (||t||*||i||);

[0134] Where t represents the element vector in the text; i represents the entity vector in the image; ||t|| represents the norm of t; ||i|| represents the norm of i; for example, after coreference elimination, we get "Type C middle-aged alcoholic skeletal head necrosis extending to the late stage"

[0135] Furthermore, the specific method of performing sequence labeling in step c3 includes:

[0136] Step c31: Add the start frame B and the end frame E at the beginning and end of the characterization vector A respectively to define the standard data frame. The specific expression can be:

[0137] A=(B,a1,a2,...,a n ,E);

[0138] Step c32: define the disease category label sequence G. The specific expression can be:

[0139] G=(B,g1,g2,...,g n ,E);

[0140] Among them, g represents each disease category label;

[0141] Step c33, define the loss function , in order to train the CRF model by gradient descent, the specific formula can be:

[0142] ;

[0143] in, Represents the score of a sequence path, specifically the sum of the emission score and the transfer score; Indicates the score conversion value of the sequence path; The score conversion value of the correct sequence path, that is, the score conversion value of the sequence path with the highest score among all sequence paths, corresponds to the prediction result of the CRF model; Represents the total number of paths; as the gradient continues to decrease, the proportion of the correct sequence path score to the sum of all sequence path scores will gradually increase, making the prediction effect of the CRF model more and more accurate;

[0144] Step c34: After concatenating the vector representation of the disease category with the text representation, sequence labeling is performed using a conditional random field to obtain a set of disease element entities and corresponding element trends as a text element set.

[0145] Further, if Figure 6As shown, the decision tree model includes a knowledge graph of femoral head necrosis and a decision logic diagram; the knowledge graph of femoral head necrosis can be obtained by experienced experts in the field of orthopedics according to the diagnosis and treatment guidelines, defining the fields related to clinical decision-making of femoral head necrosis in the medical knowledge graph; the decision logic diagram can define each judgment node, treatment branch and diagnosis and treatment plan in a mind map manner; the diagnosis and treatment plan includes core medullary decompression, impact bone grafting, tantalum rod implantation, pedicled / non-pedicled fibula implantation, rotational osteotomy at the base of the femoral neck and fine needle decompression; in this way, a decision tree model with clear evidence source, easy to understand and clear diagnosis and treatment plan can be obtained.

[0146] Furthermore, if Figure 7 As shown, the decision tree model can also embed a machine learning model at the preset decision nodes in several treatment branches to predict the diagnosis and treatment plan, so as to solve complex problems such as new treatment branches not covered by the diagnosis and treatment guidelines, multiple treatment branches appearing at a decision node, and the branch decision of the decision node relying on multiple features; the machine learning model can be a supervised model such as logistic regression, support vector machine or neural network; this can avoid the problems of slow computing speed and high computing bandwidth pressure caused by the global use of large models such as neural networks.

[0147] Furthermore, the training process of the machine learning model includes:

[0148] Step d1: Based on the text information of the medical record data of the existing case, mark the structural features of the current decision node and the corresponding diagnosis and treatment plan;

[0149] Step d2: randomly divide the text information into a training set and a test set according to a preset division ratio; and train and test the machine learning model.

[0150] Furthermore, the system uses Spark big data technology to achieve distributed processing, enabling flexible big data operations and interactions across different hospitals, devices, and business systems. Spark big data technology is a state-of-the-art big data processing framework built around speed, ease of use, and complex analysis. Compared to Hadoop, Storm, and MapReduce, it offers diverse data types, fast execution, and improved compatibility.

[0151] Example 2:

[0152] This embodiment provides a clinical decision support device for femoral head necrosis, including a processor, a memory and a bus. The memory stores instructions and data that can be read by the processor. The processor is used to call the instructions and data in the memory to implement the system as described above. The bus connects the functional components for transmitting information.

[0153] In another embodiment, this solution can be implemented by a device, which may include corresponding modules for performing each or several steps in each of the above embodiments. The modules may be one or more hardware modules specifically configured to perform the corresponding steps, or implemented by a processor configured to perform the corresponding steps, or stored in a computer-readable medium for implementation by the processor, or implemented by some combination thereof.

[0154] The processor performs the various methods and processes described above. For example, the method implementation in this solution can be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via a memory and / or a communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps in the method described above can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods by any other appropriate means (e.g., by means of firmware).

[0155] The device can be implemented using a bus architecture. The bus architecture can include any number of interconnecting buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus connects various circuits including one or more processors, memories, and / or hardware modules. The bus can also connect various other circuits such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0156] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A clinical decision support system for femoral head necrosis, characterized in that: It includes a data receiving module, a data processing module and a result generating module; The data receiving module is used to receive the patient's medical record data; the medical record data includes text information and femoral head medical imaging information; the text information includes demographic information, chief medical history, outpatient diagnosis, test results, drug prescriptions and surgical records; the femoral head medical imaging information includes MRI images and CT images; The data processing module includes a medical image segmentation and classification unit, a text encoding unit, and a clinical decision support unit; The medical image segmentation and classification unit stores a medical image segmentation and classification model, and is used to extract lesion segmentation images from the medical image information of the femoral head and predict the classification result of femoral head necrosis; The medical image segmentation and classification model is a dual-task deep neural network model. The specific model structure includes two groups of convolutional neural networks; each group of convolutional neural networks includes an input layer, a hidden layer, and an output layer arranged in sequence; the hidden layer includes n convolutional layers, a pooling layer, and a fully connected layer; The input layer of the first convolutional neural network receives the MRI image; the input layer of the second convolutional neural network receives the CT image; The first convolutional layer of the positive number of the first convolutional neural network is connected to the n convolutional layers of the second convolutional neural network; the second convolutional layer of the positive number of the first convolutional neural network is connected to the n-1 convolutional layers of the reciprocal number of the second convolutional neural network; the third convolutional layer of the positive number of the first convolutional neural network is connected to the n-2 convolutional layers of the reciprocal number of the second convolutional neural network; and so on, until the nth convolutional layer of the positive number of the first convolutional neural network is connected to the first convolutional layer of the reciprocal number of the second convolutional neural network; The training steps of the medical image segmentation and classification model include: Step a1: based on the femoral head medical imaging information of the existing case, delineate the three-column range and the necrotic focus range of each layer of the MRI image; Step a2: Calculate the first necrosis ratio of each layer. The specific calculation formula is: The first necrosis ratio = (necrotic area / femoral head area) * 100%; Step a3: Select the plane with the largest first necrosis ratio and the mid-femoral head plane in each plane, segment the necrotic lesions along the dividing line of the three cylinders, and calculate the second necrosis ratio in each cylinder in these two planes. The specific calculation formula is: Second necrosis ratio = (necrotic area in each column / femoral head area in each column) * 100%; Step a4: extracting lesion segmentation images from the MRI image and the CT image respectively, based on the range of the necrotic lesion in the first layer with the largest necrosis ratio; Step a5: According to the CJFH classification standard, based on the second necrosis ratio, the MRI image is classified and marked by a threshold determination method; the threshold determination method is as follows: when the second necrosis ratio in each column is greater than or equal to a first preset threshold, the necrotic area is determined to have penetrated the column; when the second necrosis ratio in each column is less than or equal to a second preset threshold, the column is determined to have remained; when the second necrosis ratio in each column is between the first preset threshold and the second preset threshold, the column is determined to have partially remained; the CJFH classification standard refers to the China-Japan Friendship Hospital Femoral Head Necrosis Classification Standard; Step a6: randomly dividing the femoral head medical imaging information into a training set and a test set according to a preset division ratio; using the lesion segmentation image as the image label and the CJFH classification result as the text label, and training and testing the medical image segmentation and classification model; The text encoding unit stores a natural language processing model for identifying the patient's structural features from the femoral head necrosis classification results and the text information; The clinical decision support unit stores a clinical decision support model, and predicts a diagnosis and treatment plan and decision evidence based on the structural features and the lesion segmentation image; the decision evidence includes evidence text and evidence image; The result generation module is used to send out diagnosis and treatment plans and decision-making evidence.

2. The system according to claim 1, wherein: The first preset threshold is 95%, and the second preset threshold is 1%.

3. The system according to claim 1, wherein: The natural language processing model is a BERT model; the training method of the BERT model includes: Step b1: Based on the text information, diagnosis and treatment guidelines, and field literature related to femoral head necrosis in the medical record data of existing cases, mark the medical terms therein; Step b2: randomly dividing the text information, the diagnosis and treatment guidelines, and the field literature into a training set and a test set according to a preset division ratio; and training and testing the BERT model; Step b3: normalize and map the medical term entities identified by the BERT model into standard medical terminology; the standard medical terminology includes gender, age group, symptoms, test items, diseases, drugs, and surgeries; Step b4: perform relationship extraction, graph cleaning, and entity sorting on medical standard terms to obtain a medical knowledge graph, which is used to identify structured features related to medicine in text information.

4. The system according to claim 1, wherein: The clinical decision-making support model includes a factor extraction model and a decision tree model; the factor extraction model is used to extract the patient's structural features and the diagnosis and treatment factors in the lesion segmentation image; the decision tree model makes diagnosis and treatment plan decisions based on the diagnosis and treatment factors.

5. The system according to claim 4, characterized in that The feature extraction model includes a residual network model, a Transformer, a CRF model, a DETR model and a coreference elimination model; The residual network model is used to extract image representations of relevant medical knowledge graphs from lesion segmentation images; The Transformer is used to interact with the structured features and the image representation to obtain a fusion representation of the text modality and the image modality; The CRF model is used to perform sequence labeling on the text representation in the fusion representation and extract a set of text elements; The DETR model is used to perform target detection on the image representation in the fusion representation to obtain an image entity set; The co-reference elimination model is based on a set of text elements and a set of image entities. It uses the cosine distance function to calculate the similarity of the elements in the text and the entities in the image one by one to obtain a matching value. When the matching value is greater than the preset matching threshold, the corresponding text elements and image entities are input into the diagnosis and treatment element extraction set.

6. The system according to claim 4, characterized in that The decision tree model includes a femoral head necrosis knowledge graph and a decision logic diagram; the femoral head necrosis knowledge graph is obtained by experienced experts in the field of orthopedics based on diagnosis and treatment guidelines, defining fields related to femoral head necrosis clinical decision-making in the medical knowledge graph; The decision logic diagram defines each decision node, treatment branch and diagnosis and treatment plan in a mind map manner.

7. The system according to claim 6, characterized in that The decision tree model embeds a machine learning model at preset decision nodes in several treatment branches to predict diagnosis and treatment plans; The training process of the machine learning model includes: Step d1: Based on the text information of the medical record data of the existing case, mark the structural features of the current decision node and the corresponding diagnosis and treatment plan; Step d2: randomly divide the text information into a training set and a test set according to a preset division ratio; and train and test the machine learning model.

8. A clinical decision support device for femoral head necrosis, characterized in that: The system comprises a processor, a memory and a bus, wherein the memory stores instructions and data read by the processor, the processor is used to call the instructions and data in the memory to implement the system as described in any one of claims 1 to 7, and the bus connects the functional components to transmit information.

Images