Femoral head necrosis clinical decision support system and device

Through the clinical decision support system for femoral head necrosis, medical imaging and natural language processing technology are used to automatically generate diagnosis and treatment plans, which solves the problem of misdiagnosis and missed diagnosis caused by insufficient knowledge of orthopedics and improves the treatment quality of femoral head necrosis cases.

CN118983058AActive Publication Date: 2024-11-19CHINA JAPAN FRIENDSHIP HOSPITAL

Patent Information

Application Number
CN202410995043.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-19
Estimated Expiration
2044-07-24

AI Technical Summary

Technical Problem

Due to the lack of comprehensive clinical knowledge and rich experience orthopedics from different seniority and hospitals, the cases of femoral head necrosis cannot receive high-quality medical services, and are prone to misdiagnosis and missed diagnosis.

Method used

Develop a clinical decision support system for femoral head necrosis, including data reception module, data processing module and result generation module. Through medical imaging segmentation and classification unit, natural language processing model and clinical decision support unit, it can automatically extract from patient medical record data. Lesion segmentation images and structured features, generate diagnostic and treatment plans and decision-making evidence, and provide standardized clinical decision-making support.

Benefits of technology

It has achieved automatic, rapid and reliable diagnosis and treatment plans for femoral head necrosis cases, and provided standardized and homogenized clinical decision support for orthopedics of different seniority and hospitals, reducing the occurrence of misdiagnosis and missed diagnosis and improving the quality of treatment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118983058A_ABST
    Figure CN118983058A_ABST
Patent Text Reader

Abstract

The invention provides a clinical decision support system and device for femoral head necrosis, and relates to the technical field of clinical decision support, the system mainly comprises a data receiving module, a data processing module and a result generation module; the data receiving module is used for receiving medical record data of the patient; the data processing module comprises a medical image segmentation and typing unit, a character coding unit and a clinical decision support unit; the medical image segmentation and typing unit is used for extracting a focus segmentation image from the femoral head medical image information and predicting a femoral head necrosis typing result; the character coding unit is used for identifying the structural features of the patient from the femoral head necrosis typing result and the character information; the clinical decision support unit is used for predicting a diagnosis and treatment scheme and decision evidence based on the structured features and the focus segmentation image; according to the scheme, the corresponding diagnosis and treatment scheme can be automatically, quickly and reliably obtained, so that standardized and homogenized clinical decision support is provided for orthopedists in different years and different hospitals.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of clinical decision support, and in particular to a clinical decision support system and device for femoral head necrosis. Background Art

[0002] Osteonecrosis of the femoral head (ONFH) is a common, intractable and disabling orthopedic disease.

[0003] At present, orthopedic doctors of different seniority and from different hospitals often lack comprehensive clinical knowledge and rich clinical experience, resulting in the lack of high-quality and homogeneous medical services for cases of femoral head necrosis, and even medical quality problems such as misdiagnosis and missed diagnosis during consultations.

[0004] Therefore, developing a clinical decision support system (CDSS) for femoral head necrosis to provide reliable and timely diagnosis and treatment reference and assistance to relevant medical personnel, thereby improving the treatment quality of femoral head necrosis cases, has become a major issue that needs to be urgently addressed in the industry. Summary of the invention

[0005] The purpose of the present invention is to provide a clinical decision support system and device for femoral head necrosis to solve at least one of the above-mentioned technical problems existing in the prior art.

[0006] In the first aspect, in order to solve the above technical problems, the present invention provides a clinical decision support system for femoral head necrosis, including a data receiving module, a data processing module and a result generating module; The data receiving module is used to receive the patient's medical record data; the medical record data includes text information and femoral head medical image information; the text information includes demographic information, medical history, outpatient diagnosis, test results, drug prescriptions and operation records, etc.; the femoral head medical image information includes MRI (magnetic resonance imaging) images and CT (X-ray scan) images; the MRI image is characterized in that it can generate images of sections of the human body in all directions, can distinguish the details of human tissues with small differences in density, and has a low sensitivity to detecting calcification foci and cortical bone lesions, etc.; the CT image is characterized in that it mainly generates scanned images of human cross-sections, can distinguish human tissues with large differences in density, and has a high sensitivity to detecting calcification foci and cortical bone lesions, etc.; The data processing module includes a medical image segmentation and classification unit, a text encoding unit and a clinical decision support unit; The medical image segmentation and classification unit stores a medical image segmentation and classification model, and is used to extract lesion segmentation images from the medical image information of the femoral head and predict the classification result of femoral head necrosis; The text encoding unit stores a natural language processing (NLP) model for identifying the structural features of the patient from the femoral head necrosis classification result and the text information; The clinical decision support unit stores a clinical decision support model, and predicts a diagnosis and treatment plan and decision evidence based on the structural features and the lesion segmentation image; the decision evidence includes evidence text and evidence image; The result generating unit is used to send out diagnosis and treatment plans and decision-making evidence.

[0007] Through the above system, after inputting the medical records of ONFH patients, the corresponding diagnosis and treatment plans can be obtained automatically, quickly and reliably, thereby providing standardized and homogeneous clinical decision support for orthopedic surgeons of different years and different hospitals.

[0008] In a feasible implementation, the medical image segmentation and classification model is a dual-task deep neural network model that can simultaneously handle the two tasks of lesion segmentation and femoral head necrosis classification. The specific model structure includes two groups of convolutional neural networks; each group of convolutional neural networks includes an input layer, a hidden layer and an output layer arranged in sequence; the hidden layer includes n convolutional layers, a pooling layer and a fully connected layer; The input layer of the first convolutional neural network receives the MRI image; the input layer of the second convolutional neural network receives the CT image; The first convolutional layer of the positive number of the first convolutional neural network is connected to the n convolutional layers of the second convolutional neural network; the second convolutional layer of the positive number of the first convolutional neural network is connected to the n-1 convolutional layers of the reciprocal of the second convolutional neural network; the third convolutional layer of the positive number of the first convolutional neural network is connected to the n-2 convolutional layers of the reciprocal of the second convolutional neural network; and so on, until the nth convolutional layer of the positive number of the first convolutional neural network is connected to the first convolutional layer of the reciprocal of the second convolutional neural network; in this way, the intermediate features of the MRI image and the CT image can be fused, so that the shallow convolution and the deep convolution can be fully and orderly fused, and the detailed information of the MRI image and the CT image can be effectively retained, thereby bringing together the imaging advantages of the MRI image and the CT image, and effectively compressing the feature dimension, thereby avoiding the problem of excessive dimension caused by splicing and fusion in the prior art.

[0009] In a feasible implementation, the training step of the medical image segmentation and typing model includes: Step a1: based on the medical imaging information of the femoral head of the existing cases, the range of the three columns (the coronal plane of the femoral head is divided into the lateral column, the central column and the medial column in the three-column structure theory) and the range of the necrotic focus at each level of the MRI image are delineated; Step a2: Calculate the first necrosis ratio of each layer. The specific calculation formula can be: The first necrosis ratio = (necrotic area / femoral head area) * 100%; Step a3, select the plane with the largest first necrosis ratio in each plane and the mid-plane of the femoral head, divide the necrotic lesions by the dividing line of the three cylinders, and calculate the second necrosis ratio in each cylinder in these two planes. The specific calculation formula can be: The second necrosis ratio = (necrotic area in each column / femoral head area in each column) * 100%; Step a4, taking the necrotic focus range in the first layer with the largest necrosis ratio as a reference, extracting lesion segmentation images from the MRI image and the CT image respectively; Step a5, according to the CJFH classification standard, based on the second necrosis ratio, the MRI image is classified and marked by a threshold determination method; The CJFH classification standard refers to the classification standard of femoral head necrosis of the China-Japan Friendship Hospital. The classification standard is based on the JIC classification standard. For patients with femoral head necrosis of ARCO stage I to III, T1WI (TR / TE=550 / 18) coronal femoral head mid-plane MRI images are used to divide femoral head necrosis into five types according to the location of the necrosis involving the three columns: M type (medial type): the necrotic area involves the medial column, and the central column and lateral column are retained; C type (central type): the necrotic area involves the central column and the medial column, and the lateral column is retained; L1 type (sublateral type): the necrotic area involves the three columns, and the lateral column is partially retained; L2 type (extreme lateral type): the necrotic area involves the lateral column and part of the central column, and part of the central column and the medial column are retained; L3 type (entire femoral head type): the necrotic area penetrates the cortex and bone marrow of the entire lateral, central and medial columns of the femoral head; The threshold determination method means: when the second necrosis ratio in each column is greater than or equal to the first preset threshold, it is determined that the necrotic area penetrates the column; when the second necrosis ratio in each column is less than or equal to the second preset threshold, it is determined that the column remains; when the second necrosis ratio in each column is between the first preset threshold and the second preset threshold, it is determined that part of the column remains; Preferably, the first preset threshold is greater than the second preset threshold; Preferably, the first preset threshold is 95%, and the second preset threshold is 1%; Step a6: randomly divide the femoral head medical imaging information into a training set and a test set according to a preset ratio; use the lesion segmentation image as the image label and the CJFH classification result as the text label to train and test the medical imaging segmentation and classification model.

[0010] In a feasible implementation, the natural language processing model is a BERT (Bidirectional Encoder Representation from Transformers) model.

[0011] In a feasible implementation, the training method of the BERT model includes the following steps: Step b1, based on the text information, diagnosis and treatment guidelines and field literature related to femoral head necrosis in the medical record data of existing cases, mark the medical terms therein; Step b2: randomly divide text information, diagnosis and treatment guidelines, and field literature into training sets and test sets according to a preset division ratio; train and test the BERT model; Step b3: normalize and map the medical term entities identified by the BERT model into standard medical terms; the standard medical terms include gender, age group, symptoms, test items, diseases, drugs, and surgeries; Step b4: extract relations, clean graphs and sort entities of standard medical terms to obtain a medical knowledge graph for identifying structured features related to medicine in text information.

[0012] In a feasible embodiment, the clinical decision support model includes a factor extraction model and a decision tree model; the factor extraction model is used to extract the patient's structural features and diagnostic and treatment factors in the lesion segmentation image; the decision tree model makes a diagnosis and treatment plan decision based on the diagnostic and treatment factors.

[0013] In a feasible implementation, the feature extraction model includes a residual network model, a Transformer, a CRF model (conditional random field), a DETR model (End-to-End Object Detection with Transformers, a Transformer-based target detection model) and a coreference elimination model; The residual network model is used to extract image representations related to the medical knowledge graph from the lesion segmentation image; The Transformer is used to exchange information between structured features and image representations to obtain a fusion representation of text modality and image modality; the Transformer belongs to the prior art and is a model based on an encoder-decoder structure. Through a self-attention mechanism, it can be trained in parallel and has global information; The CRF model is used to perform sequence labeling on the text representation in the fusion representation and extract a set of text elements; The DETR model is used to perform target detection on the image representation in the fusion representation to obtain an image entity set; The co-reference elimination model is based on a set of text elements and a set of image entities. It uses a cosine distance function to calculate the similarity of elements in the text and entities in the image one by one to obtain a matching value. When the matching value is greater than a preset matching threshold, the corresponding text elements and image entities are input into the diagnosis and treatment element extraction set.

[0014] In a feasible implementation manner, the extraction method of the factor extraction model specifically includes: Step c1, input the lesion segmentation image into the residual network model and output the image representation; Step c2: Input the patient's structural features and image representation into the first linear layer respectively, project them into the same vector space for concatenation, and then input them into the Transformer for information interaction, and output the fusion representation of text modality and image modality; Step c3: Input the text representation in the fusion representation into the second linear layer and convert it into a spectrum graph to obtain a representation vector A. The dimension of the representation vector A can be the category n of the disease. The specific expression can be: A=(a 1 ,a 2 ,...,a n ); Among them, a represents the probability matrix of each disease category label corresponding to each frame of the spectrum graph; Through the CRF model, the representation vector A is sequence labeled to extract the text element set; Step c4, input the image representation in the fusion representation into the DETR model for target detection, and output the image entity set; the DETR model adds a feedforward neural network (FFN) and a Hungarian loss function after each decoder layer of the Transformer; the feedforward neural network is used to predict the category and position of each query object in the decoder output result, including a shared layer; the shared layer is used to standardize the input value of each feedforward neural network through the shared layer norm, thereby achieving standardization processing; the Hungarian loss function is used to calculate the minimum loss between the predicted value and the paired label and return the gradient during model training; Step c5: Input the text element set and the image entity set into the coreference elimination model, calculate the similarity of the elements in the text and the entities in the image one by one through the cosine distance function, and output the matching value Ma. When the matching value Ma is greater than the preset matching threshold, the corresponding text element and image entity are input into the diagnosis and treatment element extraction set. The specific formula can be: Ma=(t·i) / (||t||*||i||); Among them, t represents the element vector in the text; i represents the entity vector in the image; ||t|| represents the norm of t; ||i|| represents the norm of i; in this way, the co-reference information in the text elements and image entities can be unified.

[0015] In a feasible implementation manner, the specific method of performing sequence labeling in step c3 includes: Step c31: Add a start frame B and an end frame E at the beginning and the end of the characterization vector A, respectively, so as to define a standard data frame. The specific expression can be: A=(B,a 1 ,a 2 ,...,a n ,E); Step c32: define the disease category label sequence G, the specific expression can be: G=(B,g 1 ,g 2 ,...,g n ,E); Among them, g represents each disease category label; Step c33, define the loss function , so as to train the CRF model by gradient descent, the specific formula can be: ; in, Represents the path score of a sequence, specifically the sum of the emission score (EmissionScore) and the transition score (TransitionScore); Indicates the score conversion value of the sequence path; The score conversion value of the correct sequence path, that is, the score conversion value of the sequence path with the highest score among all sequence paths, corresponds to the prediction result of the CRF model; Represents the total number of paths; as the gradient continues to decrease, the proportion of the correct sequence path score to the sum of all sequence path scores will gradually increase, making the prediction effect of the CRF model more and more accurate; Step c34: After concatenating the vector representation of the disease category with the text representation, sequence labeling is performed through conditional random fields to obtain a set of disease element entities and corresponding element trends as a text element set.

[0016] In a feasible implementation, the decision tree model includes a knowledge graph of femoral head necrosis and a decision logic diagram; the knowledge graph of femoral head necrosis can be obtained by experienced experts in the field of orthopedics according to the diagnosis and treatment guidelines, defining the fields related to clinical decisions on femoral head necrosis in the medical knowledge graph; the decision logic diagram can define each decision node, treatment branch and diagnosis and treatment plan in a mind map manner; the diagnosis and treatment plans include core medullary decompression, impact bone grafting, tantalum rod implantation, pedicled / non-pedicled fibula implantation, rotational osteotomy at the base of the femoral neck and fine needle decompression, etc.; in this way, a decision tree model with clear evidence source, easy to understand and clear diagnosis and treatment plans can be obtained.

[0017] In a feasible implementation, the decision tree model can also embed a machine learning model at preset decision nodes in several treatment branches to predict treatment plans, so as to solve complex problems such as new treatment branches not covered by the treatment guidelines, multiple treatment branches appearing at a decision node, and branch decisions of the decision nodes relying on multiple features; the machine learning model can be a supervised model such as logistic regression, support vector machine or neural network; this can avoid problems such as slow computing speed and high computing bandwidth pressure caused by the global use of large models such as neural networks.

[0018] In a feasible implementation, the training process of the machine learning model includes: Step d1: Based on the text information of the medical record data of the existing case, mark the structural features of the current decision node and the corresponding diagnosis and treatment plan; Step d2: randomly divide the text information into a training set and a test set according to a preset division ratio; and train and test the machine learning model.

[0019] In a feasible implementation, the system implements distributed processing through Spark big data technology, so as to flexibly realize big data computing and interaction between different hospitals, different equipment and different business systems. The Spark big data technology belongs to the prior art and is a big data processing framework built around speed, ease of use and complex analysis. Compared with Hadoop, Storm and MapReduce technologies, it has the characteristics of diverse data types, fast running speed and good compatibility.

[0020] On the second aspect, based on the same inventive concept, the present application also provides a clinical decision support device for femoral head necrosis, including a processor, a memory and a bus, the memory stores instructions and data that can be read by the processor, the processor is used to call the instructions and data in the memory to implement the system as described above, and the bus connects the functional components for transmitting information.

[0021] By adopting the above technical solution, the present invention has the following beneficial effects: The present invention provides a clinical decision support system and device for femoral head necrosis. By integrating text information such as outpatient medical records and test reports and medical imaging information of the femoral head, a clinical decision support system with the characteristics of multiple data sources, multiple data modes and multiple models is constructed. It can collect and learn various aspects of knowledge such as field experts, case databases, diagnosis and treatment guidelines and field literature. Through this solution, corresponding diagnosis and treatment plans can be obtained automatically, quickly and reliably, thereby providing standardized and homogenized clinical decision support for orthopedic doctors of different years and different hospitals; at the same time, this solution can also reduce the problems of slow system operation speed and high bandwidth pressure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A diagram of a clinical decision support system for femoral head necrosis provided by an embodiment of the present invention; Figure 2 A schematic diagram of the principle of a medical image segmentation and typing model provided by an embodiment of the present invention; Figure 3 It is a theoretical illustration of the three-column structure of the femoral head in the prior art; Figure 4 This is an example diagram of the CJFH classification standard in the prior art; Figure 5 A schematic diagram of a factor extraction model provided by an embodiment of the present invention; Figure 6 A schematic diagram of a decision tree model provided by an embodiment of the present invention; Figure 7 A schematic diagram of a decision tree model embedded in a machine learning model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0026] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0027] The present invention is further explained below in conjunction with specific implementation modes.

[0028] It should also be noted that the following specific embodiments or specific implementations are a series of optimized settings listed in the present invention to further explain the specific content of the invention, and these settings can be used in combination or in association with each other.

[0029] Embodiment 1: like Figure 1 As shown, this embodiment provides a clinical decision support system for femoral head necrosis, including a data receiving module, a data processing module and a result generating module; The data receiving module is used to receive the patient's medical record data; the medical record data includes text information and femoral head medical image information; the text information includes demographic information, medical history, outpatient diagnosis, test results, drug prescriptions and operation records; the femoral head medical image information includes MRI (magnetic resonance imaging) images and CT (X-ray scan) images; the MRI image is characterized in that it can generate images of sections of the human body in all directions, can distinguish the details of human tissues with small differences in density, and has a low sensitivity to detecting calcification foci and cortical bone lesions; the CT image is characterized in that it mainly generates scanned images of human cross-sections, can distinguish human tissues with large differences in density, and has a high sensitivity to detecting calcification foci and cortical bone lesions; The data processing module includes a medical image segmentation and classification unit, a text encoding unit and a clinical decision support unit; The medical image segmentation and classification unit stores a medical image segmentation and classification model, and is used to extract lesion segmentation images from the medical image information of the femoral head and predict the classification result of femoral head necrosis; The text encoding unit stores a natural language processing (NLP) model for identifying the structural features of the patient from the femoral head necrosis classification result and the text information; The clinical decision support unit stores a clinical decision support model, and predicts a diagnosis and treatment plan and decision evidence based on the structural features and the lesion segmentation image; the decision evidence includes evidence text and evidence image; The result generating unit is used to send out diagnosis and treatment plans and decision-making evidence.

[0030] Through the above system, after inputting the medical records of ONFH patients, the corresponding diagnosis and treatment plans can be obtained automatically, quickly and reliably, thereby providing standardized and homogeneous clinical decision support for orthopedic surgeons of different years and different hospitals.

[0031] Furthermore, if Figure 2 As shown, the medical image segmentation and classification model is a dual-task deep neural network model, which can simultaneously handle the two tasks of lesion segmentation and femoral head necrosis classification. The specific model structure includes two groups of convolutional neural networks; each group of convolutional neural networks includes an input layer, a hidden layer and an output layer arranged in sequence; the hidden layer includes n convolutional layers, a pooling layer and a fully connected layer; The input layer of the first convolutional neural network receives the MRI image; the input layer of the second convolutional neural network receives the CT image; The first convolutional layer of the positive number of the first convolutional neural network is connected to the n convolutional layers of the second convolutional neural network; the second convolutional layer of the positive number of the first convolutional neural network is connected to the n-1 convolutional layers of the reciprocal of the second convolutional neural network; the third convolutional layer of the positive number of the first convolutional neural network is connected to the n-2 convolutional layers of the reciprocal of the second convolutional neural network; and so on, until the nth convolutional layer of the positive number of the first convolutional neural network is connected to the first convolutional layer of the reciprocal of the second convolutional neural network; in this way, the intermediate features of the MRI image and the CT image can be fused, so that the shallow convolution and the deep convolution can be fully and orderly fused, and the detailed information of the MRI image and the CT image can be effectively retained, thereby bringing together the imaging advantages of the MRI image and the CT image, and effectively compressing the feature dimension, thereby avoiding the problem of excessive dimension caused by splicing and fusion in the prior art.

[0032] Furthermore, the training steps of the medical image segmentation and typing model include: Step a1: Based on the medical imaging information of the femoral head of the existing case, the three-column range and the range of the necrotic focus at each level of the MRI image are delineated, such as Figure 3 As shown; Step a2: Calculate the first necrosis ratio of each layer. The specific calculation formula can be: The first necrosis ratio = (necrotic area / femoral head area) * 100%; Step a3, select the plane with the largest first necrosis ratio in each plane and the mid-plane of the femoral head, divide the necrotic lesions by the dividing line of the three cylinders, and calculate the second necrosis ratio in each cylinder in these two planes. The specific calculation formula can be: The second necrosis ratio = (necrotic area in each column / femoral head area in each column) * 100%; Step a4, taking the necrotic focus range in the first layer with the largest necrosis ratio as a reference, extracting lesion segmentation images from the MRI image and the CT image respectively; Step a5, according to the CJFH classification standard, based on the second necrosis ratio, the MRI image is classified and marked by a threshold determination method; The CJFH classification standard refers to the classification standard of femoral head necrosis of the China-Japan Friendship Hospital, such as Figure 4 As shown in the figure, this classification standard is based on the JIC classification standard. For patients with ARCO stage I to III femoral head necrosis, T1WI (TR / TE=550 / 18) coronal femoral head mid-plane MRI images are used to divide femoral head necrosis into five types according to the location of the necrosis involving the three columns: M type (medial type): the necrotic area involves the medial column, and the central column and lateral column are retained; C type (central type): the necrotic area involves the central column and the medial column, and the lateral column is retained; L1 type (sublateral type): the necrotic area involves the three columns, and the lateral column is partially retained; L2 type (extreme lateral type): the necrotic area involves the lateral column and part of the central column, and part of the central column and medial column are retained; L3 type (whole femoral head type): the necrotic area penetrates the entire cortex and bone marrow of the lateral, central and medial columns of the femoral head; The threshold determination method means: when the second necrosis ratio in each column is greater than or equal to the first preset threshold, it is determined that the necrotic area penetrates the column; when the second necrosis ratio in each column is less than or equal to the second preset threshold, it is determined that the column remains; when the second necrosis ratio in each column is between the first preset threshold and the second preset threshold, it is determined that part of the column remains; Preferably, the first preset threshold is greater than the second preset threshold; Preferably, the first preset threshold is 95%, and the second preset threshold is 1%; Step a6: randomly divide the femoral head medical imaging information into a training set and a test set according to a preset ratio; use the lesion segmentation image as the image label and the CJFH classification result as the text label to train and test the medical imaging segmentation and classification model.

[0033] Furthermore, the natural language processing model is a BERT model.

[0034] Furthermore, the training method of the BERT model includes the following steps: Step b1, based on the text information, diagnosis and treatment guidelines and field literature related to femoral head necrosis in the medical record data of existing cases, mark the medical terms therein; Step b2: randomly divide text information, diagnosis and treatment guidelines, and field literature into training sets and test sets according to a preset division ratio; train and test the BERT model; Step b3: normalize and map the medical term entities identified by the BERT model into standard medical terms; the standard medical terms include gender, age group, symptoms, test items, diseases, drugs, and surgeries; Step b4: extract relations, clean graphs and sort entities of standard medical terms to obtain a medical knowledge graph for identifying structured features related to medicine in text information.

[0035] Furthermore, the clinical decision support model includes a factor extraction model and a decision tree model; the factor extraction model is used to extract the patient's structural features and diagnostic and treatment factors in the lesion segmentation image; the decision tree model makes a diagnosis and treatment plan decision based on the diagnostic and treatment factors.

[0036] Furthermore, if Figure 5 As shown, the feature extraction model includes a residual network model, a Transformer, a CRF model, a DETR model and a coreference elimination model; The residual network model is used to extract image representations related to the medical knowledge graph from the lesion segmentation image; The Transformer is used to exchange information between structured features and image representations to obtain a fusion representation of text modality and image modality; the Transformer belongs to the prior art and is a model based on an encoder-decoder structure. Through a self-attention mechanism, it can be trained in parallel and has global information; The CRF model is used to perform sequence labeling on the text representation in the fusion representation and extract a set of text elements; The DETR model is used to perform target detection on the image representation in the fusion representation to obtain an image entity set; The co-reference elimination model is based on a set of text elements and a set of image entities. It uses a cosine distance function to calculate the similarity of elements in the text and entities in the image one by one to obtain a matching value. When the matching value is greater than a preset matching threshold, the corresponding text elements and image entities are input into the diagnosis and treatment element extraction set.

[0037] Furthermore, the extraction method of the factor extraction model specifically includes: Step c1, inputting the lesion segmentation image into the residual network model, and outputting an image representation; for example, the image representation may include "bone of middle-aged and elderly people", "late stage of central column and medial column necrosis", etc.; Step c2: Input the patient's structural features and image representation into the first linear layer respectively, project them into the same vector space for concatenation, and then input them into the Transformer for information interaction, and output the fusion representation of text modality and image modality; Step c3: Input the text representation in the fusion representation into the second linear layer and convert it into a spectrum graph to obtain a representation vector A. The dimension of the representation vector A can be the category n of the disease. The specific expression can be: A=(a 1 ,a 2 ,...,a n ); Among them, a represents the probability matrix of each disease category label corresponding to each frame of the spectrum graph; Through the CRF model, the representation vector A is sequence labeled to extract the text element set; for example, in the text "Excessive drinking leads to a rapid expansion of the scope of C-type middle-aged femoral head necrosis", the element entities include "wine", "femoral head", "middle-aged" and "scope of necrosis", the disease category belongs to "C-type middle-aged alcoholic femoral head necrosis", and the element trend is "expansion"; Step c4, input the image representation in the fusion representation into the DETR model for target detection, and output the image entity set; the DETR model adds a feedforward neural network and a Hungarian loss function after each decoder layer of the Transformer; the feedforward neural network is used to predict the category and position of each query object in the decoder output result, including a shared layer; the shared layer is used to standardize the input value of each feedforward neural network through the shared layer norm, thereby achieving standardization processing; the Hungarian loss function is used to calculate the minimum loss between the predicted value and the paired label and return the gradient during model training; Step c5: Input the text element set and the image entity set into the coreference elimination model, calculate the similarity of the elements in the text and the entities in the image one by one through the cosine distance function, and output the matching value Ma. When the matching value Ma is greater than the preset matching threshold, the corresponding text element and image entity are input into the diagnosis and treatment element extraction set. The specific formula can be: Ma=(t·i) / (||t||*||i||); Among them, t represents the element vector in the text; i represents the entity vector in the image; ||t|| represents the norm of t; ||i|| represents the norm of i; for example, after coreference elimination, we get "Type C middle-aged alcoholic skeletal head necrosis expanded to the late stage" Furthermore, the specific method of performing sequence labeling in step c3 includes: Step c31: Add a start frame B and an end frame E at the beginning and the end of the characterization vector A, respectively, so as to define a standard data frame. The specific expression can be: A=(B,a 1 ,a 2 ,...,a n ,E); Step c32: define the disease category label sequence G, the specific expression can be: G=(B,g 1 ,g 2 ,...,g n ,E); Among them, g represents each disease category label; Step c33, define the loss function , so as to train the CRF model by gradient descent, the specific formula can be: ; in, Represents the path score of a sequence, specifically the sum of the emission score and the transfer score; Indicates the score conversion value of the sequence path; The score conversion value of the correct sequence path, that is, the score conversion value of the sequence path with the highest score among all sequence paths, corresponds to the prediction result of the CRF model; Represents the total number of paths; as the gradient continues to decrease, the proportion of the correct sequence path score to the sum of all sequence path scores will gradually increase, making the prediction effect of the CRF model more and more accurate; Step c34: After concatenating the vector representation of the disease category with the text representation, sequence labeling is performed through conditional random fields to obtain a set of disease element entities and corresponding element trends as a text element set.

[0038] Furthermore, if Figure 6 As shown, the decision tree model includes a knowledge graph of femoral head necrosis and a decision logic diagram; the knowledge graph of femoral head necrosis can be obtained by experienced experts in the field of orthopedics according to the diagnosis and treatment guidelines, defining the fields related to clinical decision-making of femoral head necrosis in the medical knowledge graph; the decision logic diagram can define each decision node, treatment branch and diagnosis and treatment plan in a mind map manner; the diagnosis and treatment plan includes core medullary decompression, impact bone grafting, tantalum rod implantation, pedicled / non-pedicled fibula implantation, rotational osteotomy at the base of the femoral neck and fine needle decompression; in this way, a decision tree model with clear evidence source, easy to understand and clear diagnosis and treatment plan can be obtained.

[0039] Furthermore, if Figure 7 As shown, the decision tree model can also embed machine learning models at preset decision nodes in several treatment branches to predict diagnosis and treatment plans, so as to solve complex problems such as new treatment branches not covered by the diagnosis and treatment guidelines, multiple treatment branches appearing at a decision node, and branch decisions of decision nodes relying on multiple features; the machine learning model can be a supervised model such as logistic regression, support vector machine or neural network; this can avoid the problems of slow computing speed and high computing bandwidth pressure caused by the global use of large models such as neural networks.

[0040] Furthermore, the training process of the machine learning model includes: Step d1: Based on the text information of the medical record data of the existing case, mark the structural features of the current decision node and the corresponding diagnosis and treatment plan; Step d2: randomly divide the text information into a training set and a test set according to a preset division ratio; and train and test the machine learning model.

[0041] Furthermore, the system uses Spark big data technology to achieve distributed processing, so as to flexibly realize big data computing and interaction between different hospitals, different equipment and different business systems. The Spark big data technology belongs to the prior art and is a big data processing framework built around speed, ease of use and complex analysis. Compared with Hadoop, Storm and MapReduce technology, it has the characteristics of diverse data types, fast running speed and good compatibility.

[0042] Embodiment 2: This embodiment provides a clinical decision support device for femoral head necrosis, including a processor, a memory and a bus. The memory stores instructions and data that can be read by the processor. The processor is used to call the instructions and data in the memory to implement the system as described above. The bus connects the functional components for transmitting information.

[0043] In another embodiment, the present solution can be implemented by a device, which may include a corresponding module for performing each or several steps in the above-mentioned embodiments. The module may be one or more hardware modules specially configured to perform the corresponding steps, or implemented by a processor configured to perform the corresponding steps, or stored in a computer-readable medium for implementation by a processor, or implemented by some combination.

[0044] The processor performs the various methods and processes described above. For example, the method implementation in the present solution can be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via a memory and / or a communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps in the method described above can be executed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods by any other appropriate means (for example, by means of firmware).

[0045] The device can be implemented using a bus architecture. The bus architecture can include any number of interconnecting buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus connects various circuits including one or more processors, memories, and / or hardware modules together. The bus can also connect various other circuits such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0046] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0047] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A clinical decision support system for femoral head necrosis, characterized in that: It includes a data receiving module, a data processing module and a result generating module; The data receiving module is used to receive the patient's medical record data; the medical record data includes text information and femoral head medical imaging information; the text information includes demographic information, medical history, outpatient diagnosis, test results, drug prescriptions and surgical records; the femoral head medical imaging information includes MRI images and CT images; The data processing module includes a medical image segmentation and classification unit, a text encoding unit and a clinical decision support unit; The medical image segmentation and classification unit stores a medical image segmentation and classification model, and is used to extract lesion segmentation images from the medical image information of the femoral head and predict the classification result of femoral head necrosis; The text encoding unit stores a natural language processing model for identifying the structural characteristics of the patient from the femoral head necrosis classification result and the text information; The clinical decision support unit stores a clinical decision support model, and predicts a diagnosis and treatment plan and decision evidence based on the structural features and the lesion segmentation image; the decision evidence includes evidence text and evidence image; The result generating unit is used to send out diagnosis and treatment plans and decision-making evidence.

2. The system according to claim 1, characterized in that The medical image segmentation and classification model is a dual-task deep neural network model, and the specific model structure includes two groups of convolutional neural networks; each group of convolutional neural networks includes an input layer, a hidden layer and an output layer arranged in sequence; the hidden layer includes n convolutional layers, a pooling layer and a fully connected layer; The input layer of the first convolutional neural network receives the MRI image; the input layer of the second convolutional neural network receives the CT image; The first convolutional layer of the positive number of the first convolutional neural network is connected to the n convolutional layers of the second convolutional neural network; the second convolutional layer of the positive number of the first convolutional neural network is connected to the n-1 convolutional layers of the reciprocal second convolutional neural network; the third convolutional layer of the positive number of the first convolutional neural network is connected to the n-2 convolutional layers of the reciprocal second convolutional neural network; and so on, until the nth convolutional layer of the positive number of the first convolutional neural network is connected to the first convolutional layer of the reciprocal second convolutional neural network.

3. The system according to claim 2, characterized in that The training steps of the medical image segmentation classification model include: Step a1: based on the medical imaging information of the femoral head of the existing case, the three-column range and the range of the necrotic focus at each level of the MRI image are delineated; Step a2: Calculate the first necrosis ratio of each layer. The specific calculation formula is: The first necrosis ratio = (necrotic area / femoral head area) * 100%; Step a3, select the plane with the largest first necrosis ratio in each plane and the mid-plane of the femoral head, divide the necrotic lesions by the dividing line of the three cylinders, and calculate the second necrosis ratio in each cylinder in these two planes. The specific calculation formula is: The second necrosis ratio = (necrotic area in each column / femoral head area in each column) * 100%; Step a4, taking the necrotic focus range in the first layer with the largest necrosis ratio as a reference, extracting lesion segmentation images from the MRI image and the CT image respectively; Step a5, according to the CJFH classification standard, based on the second necrosis ratio, the MRI image is classified and marked by a threshold determination method; the threshold determination method refers to: when the second necrosis ratio in each column is greater than or equal to the first preset threshold, it is determined that the necrotic area penetrates the column; when the second necrosis ratio in each column is less than or equal to the second preset threshold, it is determined that the column remains; when the second necrosis ratio in each column is between the first preset threshold and the second preset threshold, it is determined that the column is partially retained; the CJFH classification standard refers to the classification standard of femoral head necrosis of the China-Japan Friendship Hospital; Step a6: randomly divide the femoral head medical imaging information into a training set and a test set according to a preset ratio; use the lesion segmentation image as the image label and the CJFH classification result as the text label to train and test the medical imaging segmentation and classification model.

4. The system according to claim 3, characterized in that The first preset threshold is 95%, and the second preset threshold is 1%.

5. The system according to claim 1, characterized in that The natural language processing model is a BERT model; and the training method of the BERT model includes: Step b1, based on the text information, diagnosis and treatment guidelines and field literature related to femoral head necrosis in the medical record data of existing cases, mark the medical terms therein; Step b2, randomly dividing the text information, the diagnosis and treatment guidelines, and the field literature into a training set and a test set according to a preset division ratio; training and testing the BERT model; Step b3: normalize and map the medical term entities identified by the BERT model into standard medical terms; the standard medical terms include gender, age group, symptoms, test items, diseases, drugs and surgeries; Step b4: extract relations, clean graphs and sort entities of standard medical terms to obtain a medical knowledge graph for identifying structured features related to medicine in text information.

6. The system according to claim 1, characterized in that The clinical decision support model includes a factor extraction model and a decision tree model; the factor extraction model is used to extract the patient's structural features and the diagnosis and treatment factors in the lesion segmentation image; the decision tree model makes a diagnosis and treatment plan decision based on the diagnosis and treatment factors.

7. The system according to claim 6, characterized in that The feature extraction model includes a residual network model, a Transformer, a CRF model, a DETR model and a coreference elimination model; The residual network model is used to extract image representations related to the medical knowledge graph from the lesion segmentation image; The Transformer is used to perform information interaction between the structured features and the image representation to obtain a fusion representation of the text modality and the image modality; The CRF model is used to perform sequence labeling on the text representation in the fusion representation and extract a set of text elements; The DETR model is used to perform target detection on the image representation in the fusion representation to obtain an image entity set; The co-reference elimination model is based on a set of text elements and a set of image entities. It uses a cosine distance function to calculate the similarity of elements in the text and entities in the image one by one to obtain a matching value. When the matching value is greater than a preset matching threshold, the corresponding text elements and image entities are input into the diagnosis and treatment element extraction set.

8. The system according to claim 6, characterized in that The decision tree model includes a femoral head necrosis knowledge graph and a decision logic graph; the femoral head necrosis knowledge graph is obtained by defining the fields related to femoral head necrosis clinical decision in the medical knowledge graph according to the diagnosis and treatment guidelines by experienced experts in the field of orthopedics; The decision logic diagram defines each decision node, treatment branch and diagnosis and treatment plan through a mind map.

9. The system according to claim 8, characterized in that The decision tree model embeds a machine learning model at preset decision nodes in several treatment branches to predict diagnosis and treatment plans; The training process of the machine learning model includes: Step d1: Based on the text information of the medical record data of the existing case, mark the structural features of the current decision node and the corresponding diagnosis and treatment plan; Step d2: randomly divide the text information into a training set and a test set according to a preset division ratio; and train and test the machine learning model.

10. A clinical decision support device for femoral head necrosis, characterized in that: It includes a processor, a memory and a bus, wherein the memory stores instructions and data read by the processor, the processor is used to call the instructions and data in the memory to implement the system as described in any one of claims 1-9, and the bus connects the functional components for transmitting information.

Citation Information

Patent Citations

  • MR image femoral head ischemic necrosis identification method and device

    CN111553882A

  • Total hip replacement revision preoperative planning method and equipment based on deep learning

    CN112971981A

  • Breast cancer image determination method and system

    CN113393445A

  • Deep learning-based CT medical image processing model training method and diagnosis and treatment system

    CN113627564A

  • Hip protection surgical treatment system and device based on CJFH typing

    CN116350349A

Cited By

  • Orthognathic surgery treatment suggestion generation system, method, equipment and medium

    CN119207715A

  • Pancreatic lesion resection type auxiliary decision making system, method and device

    CN120452753A

  • Pancreatic lesion resection procedure assistance system, method, and apparatus

    CN120452753B

  • Femoral head image analysis system

    CN121074057A

  • A femoral head image analysis system

    CN121074057B