Clinical decision-making system for craniocerebral trauma based on text and image recognition

Abstract

The present invention relates to the field of medical devices, and in particular to a clinical decision-making system for craniocerebral trauma based on text and image recognition. The system comprises: an image recognition module for generating a text report after recognizing CT images; establishing a mapping relationship between CT signs and clinical conditions and treatment measures; a key named entity recognition module for constructing a BERT-IDCNN-BiLSTM-CRF model based on a BERT pre-trained language model to extract key named entities from the global or local text report; a knowledge-based semantic representation module for performing feature analysis on key named entities, establishing a Word2vec model, vectorizing related medical texts, and establishing a medical text weight scoring mechanism and a medical text similarity measurement method based on knowledge to obtain medical texts; and a deep learning decision-making module for understanding the differences between medical terms after extracting medical text features, intelligently identifying and interpreting medical texts for TBI patients, and making diagnosis and treatment decisions. The system can improve the efficiency of TBI treatment.

Description

Technical Field

[0001] The present invention relates to the field of medical devices, and in particular to a clinical decision-making system for craniocerebral trauma based on text and image recognition. Background Art

[0002] Traumatic brain injury (TBI), one of the most common critical neurosurgery emergencies, is characterized by high morbidity and mortality, posing a particularly serious threat to the health and well-being of young adults. Furthermore, TBI not only severely impacts patients' physical and mental health and quality of life, but also imposes significant economic losses and a heavy medical burden on families and society. Therefore, research on improving the efficiency of TBI treatment and curbing its harmful effects has significant social and economic benefits.

[0003] The invention patent with application number 202311626792.9 discloses a multi-task hybrid neural network driven CT image processing method and brain-like diagnostic system, which relates to artificial intelligence medical imaging. The method includes generating a hybrid neural network, inputting a CT image, performing hybrid pulse-convolution calculations at each layer; generating a common encoder and a multi-task decoder, and outputting a multi-task result image. The system includes a processor, a memory, and a display. The memory stores at least one instruction, which is loaded and executed by the processor to implement the following steps: acquiring a CT image; generating a common encoder based on a hybrid neural network, calling the encoder to encode the CT image, and generating joint features; generating a multi-task decoder based on a hybrid neural network, calling the decoder to perform classification and segmentation operations on the joint features, and obtaining a multi-task diagnostic result. The present invention can improve the efficiency of multi-task diagnosis of CT images, while ensuring the accuracy of multi-tasks and reducing the computing power consumption of the diagnostic system.

[0004] The above-mentioned technologies accurately recognize CT images, identify image features based on image segmentation, and produce multi-task diagnostic results for CT images. However, these technologies still require doctors to make the final decision and diagnosis. The existing TBI diagnosis and treatment model has many problems that negatively impact treatment efficiency. These problems can be summarized as three major "pain points": "Pain point 1": A severe shortage of neurosurgeons is unable to cope with the growing number of TBI patients. "Pain point 2": Long and excessive workloads lead to doctor fatigue and reduce doctor treatment efficiency. "Pain point 3": Multiple factors may affect doctor judgment, thereby affecting the treatment of TBI patients. The simplest and most effective solution to these three "pain points" is to quickly increase the number of energetic, tireless, calm, and clinically experienced neurosurgeons, but this is obviously not feasible. A technology is needed to improve the efficiency of TBI treatment. Summary of the Invention

[0005] The present invention provides a clinical decision-making system for craniocerebral trauma based on text and image recognition, which can improve the efficiency of TBI treatment.

[0006] In order to solve the above technical problems, this application provides the following technical solutions:

[0007] Clinical decision-making system for craniocerebral trauma based on text and image recognition, including:

[0008] Image recognition module: used to generate text reports after identifying CT images; establish mapping relationships between CT signs and clinical conditions and diagnosis and treatment measures;

[0009] Key named entity recognition module: Based on the BERT pre-trained language model, a BERT-IDCNN-BiLSTM-CRF model is built to extract key named entities from the global or local content of text reports;

[0010] Knowledge-based semantic representation module: This module performs feature analysis on key named entities, establishes a Word2vec model, vectorizes relevant medical text, and establishes a medical text weight scoring mechanism and a medical text similarity measurement method based on knowledge to obtain medical text.

[0011] Deep learning decision module: After extracting medical text features, it understands the differences between medical terms, intelligently identifies and interprets medical texts for TBI patients, and makes diagnosis and treatment decisions.

[0012] The basic principles and beneficial effects of this program: Based on the previous basic, clinical and applied research, we deeply study and simulate the diagnostic and treatment ideas of neurosurgeons for TBI, so as to build a new CDSS for craniocerebral trauma, namely "T-SYSTEM". T-SYSTEM will directly obtain key medical images and text information including medical history, physical examination, and important auxiliary examinations from the patient's electronic medical record (EMR), avoiding data loss caused by inconsistent information source channels. After obtaining key information such as medical history, physical signs, and CT manifestations, the system draws a diagnosis and treatment conclusion through clinical analysis. At the same time, the system refines the diagnosis and treatment plan of TBI and prioritizes the treatment measures according to the severity of the disease, including both the diagnosis and treatment measures required for exclusion diagnosis and rescue measures. T-SYSTEM includes four major modules: (1) image recognition module; (2) key named entity recognition system; (3) knowledge-based semantic representation system; (4) deep learning decision system. In the image recognition module, CT findings are carefully identified and a text report is generated. This serves as a crucial data source for T-SYSTEM's disease assessment and decision-making. Based on this data, a mapping relationship is established between key CT signs and clinical conditions and treatment measures. Furthermore, based on the BERT pre-trained language model, we constructed a BERT-IDCNN-BiLSTM-CRF model. This model effectively extracts key named entities from global and local text, demonstrating particularly strong recognition results in medical text recognition. The medical knowledge-based word embedding space in the deep learning decision system is more suitable for medical scenarios. After extracting medical text features, the model can better understand the differences between medical terms. Deep learning-based models also provide sufficient representational power for text, enabling rapid and high-quality problem solving. Information in the data can be effectively "encoded" within the model.

[0013] By recognizing CT images, the system generates textual information displaying the features within the CT images. It then performs semantic recognition on the features within the textual information, deriving the diagnostic and treatment measures for the conditions identified by those features, and then making appropriate decisions. This decision is made based on the appropriate treatment measures for the condition. By intelligently learning from doctors' treatment methods through data models, machine decision-making is achieved, freeing up manpower, reducing the recognition errors that can easily occur with long-term human effort, and improving processing efficiency, enabling simultaneous diagnosis of numerous patients.

[0014] Furthermore, the image recognition module is also used to preprocess the skull CT images: the skull CT images of all normal people and patients are saved in DICOM format, the improved Kalman filter algorithm is used to denoise the CT images, and the skull CT images are annotated using the LableImg tool; the skull CT images of TBI patients are then divided into skull injuries and brain injuries. The skull CT images of TBI patients are segmented using the Co-UNet model at the bone window and brain window levels of the skull CT images, respectively. A deep convolutional network model with single-connected domain boundary interpolation is designed to extract image feature information of normal skull CT images. Abnormal information is identified based on the DenseNet network. The improved threshold segmentation algorithm is used for secondary screening of the recognition results. Different types of skull injuries and brain injuries in the two-dimensional skull CT images of TBI patients are identified by lateralization, location, qualitative and quantitative analysis. Beneficial effects: The skull CT images of all normal people and patients are saved in DICOM format, the improved Kalman filter algorithm is used to denoise the CT images, and the LableImg tool is used to annotate the skull CT images, providing data support for recognition.

[0015] Furthermore, the deep learning decision-making module is also used to: use TBI patients' emergency electronic medical records (EMRs) as the source of medical text information, collecting content mainly including medical history, physical examination, and auxiliary examination results; among them, the cranial CT examination results are the text results output by the image recognition module; and guided by the latest domestic and international craniocerebral trauma diagnosis and treatment guidelines, combined with clinical practice experience and the research team's previous basic, clinical, and applied research results, assign different weighted scores to important auxiliary examinations such as the medical history, physical examination, and CT of craniocerebral trauma patients; at the same time, refine various craniocerebral trauma diagnosis and treatment measures, and after repeated clinical data verification, study and construct a mapping relationship between multidimensional, multi-source, heterogeneous data and corresponding diagnosis and treatment decisions; and through intelligent recognition of TBI patients' cranial CT images and EMRs, obtain diagnosis and treatment results output in Chinese text form, including emergency diagnostic conclusions and emergency diagnosis and treatment decisions.

[0016] Furthermore, when identifying skull CT images, the normal skull CT images are identified: the bone window and brain window images of normal skull CT images are stored and analyzed separately; at the bone window, the skull is segmented using the Co-UNet model, including the facial skull and brain skull, and the features of these bony landmark structures are extracted; at the brain window, the brain parenchyma area is segmented using the Co-UNet model, including the supratentorial and infratentorial brain parenchyma areas, and the features of these landmark structures within the brain parenchyma are extracted, and the average area of ​​the bilateral lateral ventricles at different skull CT levels of normal people is calculated;

[0017] Identification of TBI patients' head CT images: The injuries of TBI patients are divided into skull injuries and brain injuries. The bone window and brain window positions of TBI patients' head CT images are stored and analyzed separately, and brain injuries and skull injuries are identified at different window positions.

[0018] Furthermore, the identification of brain window positions in cranial CT images of TBI patients mainly includes: identifying brain injuries, including lateralization, location, qualitative and quantitative identification; summarizing the lateralization, location, qualitative and quantitative identification of brain injuries;

[0019] Identification of skull injuries at the bone window level of TBI patients' head CT images mainly includes: identification of the location and type of skull injury, including lateralization, positioning, qualitative and quantitative identification; and summary of lateralization, positioning, qualitative and quantitative identification of skull injuries.

[0020] Furthermore, the text information output by the image processing module includes conclusive results and descriptive results; the conclusive results: output the results of brain injury and / or skull injury according to the diagnostic thinking of neurosurgery and neuroimaging, including lateralization, positioning and qualitative results; the descriptive results: provide a more detailed description of the head CT image, including lateralization, positioning, qualitative and quantitative results.

[0021] Furthermore, the key named entity recognition module includes:

[0022] BERT module:

[0023] BERT uses a masked language model similar to cloze test to complete pre-training. The purpose of the masked language model is to randomly mask some words in a sentence and replace the original words with a token [MASK]. The further goal of the masked language model is to predict the masked words based on the context to the left and right of the token. The calculation method of the BERT model is shown in formula (1):

[0024]

[0025] Among them, Q, K, V are input vector matrices, QK T Indicates the calculation of the similarity matrix of the input vector, d k is the dimension of the input vector matrix K.

[0026] In BERT's downstream tasks, such as entailment and sentiment analysis, the first token of each sequence is processed. This token is called a special classification token [CLS]. In the BERT model, the final hidden state corresponding to the [CLS] token aggregates the feature representation of a sentence or a pair of sentences.

[0027] IDCNN module:

[0028] IDCNN uses dilated convolution to reduce data loss, expand the perception range, and achieve longer capture of text by adjusting the dilation width of the convolution kernel. The receptive field calculation formula of dilated convolution is:

[0029] F i+1 =(2 i+2 -1) 2

[0030] The IDCNN module has three convolutional layers with dilation widths of 1, 1, and 2, respectively. The module undergoes four iterations, and parameter sharing during the iterations effectively prevents overfitting. Each parameter increases linearly with the number of layers, while the receptive field value increases exponentially, allowing the receptive field to quickly cover the entire input sequence.

[0031] BiLSTM module:

[0032] BiLSTM uses a gating mechanism in a recurrent neural network, namely an input gate, an output gate, and a forget gate. BiLSTM consists of a bidirectional LSTM, extracting global context features from two directions.

[0033] CRF module:

[0034] For an input word sequence X, there is a general label sequence Y representing the word sequence X. The probability model of the conditional random field defines a distribution P(Y|X) over all possible label sequences Y given the input word sequence X. Its form is as follows:

[0035]

[0036] Beneficial effects: In view of the characteristics of unstructured data noise, multiple fine-grained texts, difficult information extraction, and unstable recognition accuracy in the construction process of medical-related knowledge representation systems and their semantic analysis systems, we plan to build a BERT-IDCNN-BiLSTM-CRF model specifically for the characteristics of medical text information to solve the key problem of accurately identifying key medical-related named entities.

[0037] Furthermore, the knowledge-based semantic representation module is used to calculate a persistent distributed representation of medical text; a text vector is generated as an output from the input of a medical text corpus through a neural network structure with a hidden layer; a vocabulary is first generated from the input text in the Word2vec model, and then word vectors are learned through backpropagation and stochastic gradient methods;

[0038] Also used for:

[0039] A similarity measurement method is established to analyze and match corresponding treatment measures based on the weight score of medical performance; the medical text word vectors generated by Word2vec are used to generate sentence vector representations of key texts, and the sentence vector representation is obtained by the following formula.

[0040]

[0041] Where V S is the sentence vector representation, V i is the word vector representation of word i, w(i) is the weight, and m is the number of words;

[0042] This method uses cosine similarity to match key medical texts with a weighted scoring table developed by medical experts, and analyzes corresponding treatment measures based on medical manifestations. Cosine similarity ranges from 0 to 1. When the cosine similarity is close to 1, the two vectors are considered similar, and when the cosine similarity is close to 0, the two vectors are considered dissimilar. For different key medical entities, we develop corresponding similarity thresholds through experiments.

[0043]

[0044] Among them, v1 and v2 are different sentence vector representations.

[0045] Furthermore, the CT image analyzed by the image recognition module comes from a CT detector; it includes a frame, and the CT detector includes an X-ray transmitter arranged in a horizontal direction and an X-ray receiver corresponding to the X-ray transmitter; the X-ray transmitter is fixed on the frame, and a support platform is fixed on the frame between the X-ray transmitter and the X-ray receiver, and a headrest is fixed on the support platform, and deflection blocks are hinged on both sides of the headrest, and a sliding rod is hinged on the bottom of the deflection block, and the support platform is provided with a first slide and a second slide from top to bottom, and the first slide and the second slide are distributed in an eight-shaped shape, and an air duct is also provided in the support platform, and the air duct is respectively connected to the bottom of the first slide and the second slide, and the sliding rods on the left and right sides of the headrest are slidably connected to the first slide and the second slide, and the bottom of the deflection block and the support A first spring is fixed to the top of the table, and the first spring is sleeved on the outside of the sliding rod; a third slide in the vertical direction is provided on the support table close to the X-ray emitter side, the bottom of the third slide is connected to the air duct, and a first lead block is slidably connected in the third slide, and a second spring is fixedly connected between the bottom of the first lead block and the inner wall of the third sliding bottom. The third slide at the bottom of the first lead block, the first slide and the second slide at the bottom of the two sliding rods, and the air duct form a closed space; when the sliding rod is not pressed down, the first lead block is not within the irradiation range of the X-ray emitter; when the deflection block above the sliding rod is pressurized, the first lead block is pushed out and appears within the irradiation range of the X-ray; the data processing module collects the pictures drawn by the X-ray detector, and discards the CT pictures with the lead block images and transmits them to the image recognition module.

[0046] Furthermore, position sensors are provided in the first slide and the second slide, and the position sensors send a reminder signal when the slide rod moves downward in the first slide / second slide; an electric telescopic rod is fixed on the frame above the headrest, and the moving direction of the free end of the electric telescopic rod is downward, and a second lead block is fixed at the bottom of the electric telescopic rod; the data processing module receives the reminder signal, controls the electric telescopic rod to start, and extends the second lead block into the X-ray irradiation area, and after the reminder signal disappears, the electric telescopic rod is reset.

[0047] Beneficial Effects: During head CT scans on children, due to their poor self-control, they may move their heads during the test, causing unnecessary movements that can affect the final results. For example, any head movement can blur the image, affecting image quality. Artifact Generation: Inappropriate movement can produce artifacts in the image, which can interfere with image interpretation and potentially lead to misdiagnosis or missed diagnosis. This solution uses a decision-making system to issue a diagnosis and treatment opinion based on the patient's condition. Deviations in the CT image can significantly affect accuracy. Therefore, this solution employs a telescopic lead block. When the patient moves, it triggers the deflection block to deflect, compressing the airway. The first lead block, under pressure, is pushed out and appears within the X-ray irradiation area. This creates a noticeable shadow on the resulting image, indicating that the image is unusable. The data processing module identifies and discards images with shadows, ensuring that the CT images transmitted to the image processing module are completely accurate, improving the accuracy of the data sample and reducing the probability of errors in the decision-making system. Furthermore, this solution performs CT scans continuously without stopping the system, preventing radiation hazards caused by multiple exposures if image defects occur.

[0048] In order to prevent measurement errors, the telescopic rod is set to send a signal only when the slide bar moves down (that is, the deflection block is under pressure) to control the extension of the telescopic rod. That is, two shadows may appear on the image taken when the patient is deflected, thus reducing the error in the shadow generation. For example, when there is a metal foreign body in the patient's head, it may also cause a shadow, which is easy to be confused with the movement of the head. By setting the telescopic rod, the patient can be reminded while being extended, and a secondary mark can be made on the image to reduce sample errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flowchart of T-SYSTEM's diagnosis and treatment of TBI patients;

[0050] Figure 2 This is a schematic diagram of the T-SYSTEM workflow;

[0051] Figure 3 This is a schematic diagram of the construction process of T-SYSTEM;

[0052] Figure 4 A knowledge representation system for TBI diagnosis and treatment;

[0053] Figure 5 The process of image recognition system identifying different craniocerebral injuries;

[0054] Figure 6 A comparison chart of multiple models;

[0055] Figure 7 It is a structural diagram of a CT detector;

[0056] Figure 8 for Figure 7 Another state diagram.

[0057] Figure markings in the specification: support platform 1, first slide 11, air duct 12, second slide 13, slide rod 14, first spring 15, deflection block 16, headrest 17, second spring 2, first lead block 3, support frame 4, X-ray transmitter, X-ray receiver 6, frame 7, electric telescopic rod 8. DETAILED DESCRIPTION

[0058] The following is further described in detail through specific implementation methods:

[0059] Since the content that needs to be automatically segmented and recognized in craniocerebral trauma CT images includes multiple tasks such as hematoma volume, degree of lateral ventricular compression, midline structure displacement, fracture location and depth, and involves both two-dimensional and three-dimensional image information, we combined 3D-UNet and 2D-UNet to construct a comprehensive image recognition model based on 3D-UNet and 2D-UNet neural networks, and named it Co-UNet model, to automatically segment and recognize cranial CT images of patients with craniocerebral trauma.

[0060] Conventional 2D-UNet models fail to fully utilize data between layers and lack spatial information. While 3D-UNet models can utilize more spatial information, they have a large number of parameters and consume significant computational resources. The 3D-UNet architecture employed in this project primarily consists of an encoder, a decoder, dense connections, and deep supervision. The encoder performs convolution operations, ReLU activations, and batch normalization to extract image features. The decoder uses transposed convolutional layers for upsampling, integrating features and recovering the segmentation results. Skip connections connect the encoder and decoder, providing the network with high-level semantic image information, including sufficient spatial and semantic features. Deep supervision accelerates network convergence and improves the learning of both deep and shallow information. The 2D-UNet architecture primarily consists of an encoder and a decoder. The encoder extracts image features, while the decoder maps these features back to the original image size and generates segmentation results. The encoder and decoder each consist of four competitive dense blocks (CDBs). The introduction of the maxout activation function increases the network's flexibility and expressiveness, improving performance. Dense connections within convolutional blocks are achieved by concatenating features, effectively doubling the number of learnable parameters within each encoder and decoder block, significantly increasing memory requirements. Each block consists of a sequence of three layers: parameter-rectified linear units (PReLU), convolution (Conv), and batch normalization (BN).

[0061] Example 1 is as shown in the attached Figure 1-5 As shown, the clinical decision-making system for craniocerebral trauma based on text and image recognition includes: an image recognition module: used to generate a text report after recognizing CT images; establish a mapping relationship between CT signs and clinical conditions and diagnosis and treatment measures;

[0062] Key named entity recognition module: Based on the BERT pre-trained language model, a BERT-IDCNN-BiLSTM-CRF model is built to extract key named entities from the global or local content of text reports;

[0063] Knowledge-based semantic representation module: This module performs feature analysis on key named entities, establishes a Word2vec model, vectorizes relevant medical text, and establishes a medical text weight scoring mechanism and a medical text similarity measurement method based on knowledge to obtain medical text.

[0064] Deep learning decision module: After extracting medical text features, it understands the differences between medical terms, intelligently identifies and interprets medical texts for TBI patients, and makes diagnosis and treatment decisions.

[0065] The image processing module collects head CT images and completes image preprocessing. A head CT image database of TBI patients and normal subjects is established, and all head CT images are saved in DICOM format. Figure 7-8 As shown, the CT image analyzed by the image recognition module comes from a CT detector; the CT detector includes a frame 7, and the CT detector includes an X-ray transmitter arranged in a horizontal direction and an XX light receiver 6 corresponding to the X-ray transmitter; the X-ray transmitter is fixed on the frame 7, and a support platform 1 is fixed on the frame 7 between the X-ray transmitter and the XX light receiver 6, and a headrest 17 is fixed on the support platform 1, and a deflection block 16 is hinged on both sides of the headrest 17, and a slide bar 14 is hinged on the bottom of the deflection block 16, and a first slide 11 and a second slide 13 are provided on the support platform 1 from top to bottom, and the first slide 11 and the second slide 13 are distributed in an eight-shaped shape, and an air duct 12 is also provided in the support platform 1, and the air duct 12 is respectively connected to the bottom of the first slide 11 and the second slide 13, and the slide bars 14 on the left and right sides of the headrest 17 are respectively slidably connected to the first slide 11 and the second slide 13. A first spring 15 is fixed to the bottom of the deflection block 16 and the top of the support platform 1, and the first spring 15 is sleeved on the outside of the slide rod 14; a third slide in the vertical direction is opened on the support platform 1 near the side of the X-ray emitter, and the bottom of the third slide is connected to the air duct 12, and the first lead block 3 is slidably connected in the third slide, and a second spring 2 is fixedly connected between the bottom of the first lead block 3 and the inner wall of the third sliding bottom. The third slide at the bottom of the first lead block 3, the first slide 11 and the second slide 13 at the bottom of the two slide rods 14, and the air duct 12 form a closed space; when the slide rod 14 is not pressed down, the first lead block 3 is not within the irradiation range of the X-ray emitter; when the deflection block 16 above the slide rod 14 is pressurized, the first lead block 3 is pushed out and appears within the irradiation range of the X-ray; the data processing module collects the pictures drawn by the X-ray detector, and discards the CT pictures with lead block images and transmits them to the image recognition module.

[0066] Position sensors are provided in the first slide 11 and the second slide 13, and the position sensors send out a reminder signal when the slide rod 14 moves downward in the first slide 11 / the second slide 13; an electric telescopic rod 8 is fixed on the frame 7 above the headrest 17, and the moving direction of the free end of the electric telescopic rod 8 is downward, and a second lead block is fixed at the bottom of the electric telescopic rod 8; the data processing module receives the reminder signal, controls the electric telescopic rod 8 to start, and extends the second lead block into the X-ray irradiation area. After the reminder signal disappears, the electric telescopic rod 8 is reset.

[0067] When performing head CT scans on children, due to their poor self-control, they may shake their heads during the test, causing unnecessary movements that affect the final test results; for example, any movement of the head may cause the image to appear blurred, affecting the image quality. Artifact generation: Inappropriate movement may produce artifacts on the image, which will interfere with the interpretation of the image and may lead to misdiagnosis or missed diagnosis. This solution uses a decision-making system to issue diagnosis and treatment opinions on the patient's condition. If the CT image has deviations, it will greatly affect the accuracy. Therefore, this solution sets the extension and contraction of the lead block, which will trigger the deflection block 16 to deflect when the patient moves, thereby squeezing the airway 12. The first lead block 3 is pushed out when under pressure and appears in the X-ray irradiation area. At this time, a clear shadow will appear on the image, indicating that this image is unusable. The data processing module identifies and discards images with shadows, ensuring that the CT images transmitted to the image processing module are completely correct, improving the accuracy of the data samples, and thereby reducing the probability of errors in the decision-making system. In addition, this solution continuously detects CT without stopping the machine, which can prevent radiation hazards caused by multiple exposures if defects appear in the image.

[0068] In order to prevent measurement errors, the telescopic rod is set up so that a signal is sent only when the slide bar 14 moves down (that is, the deflection block 16 is under pressure) to control the extension of the telescopic rod. That is, two shadows may appear on the image taken when the patient is deflected, thereby reducing the error in the shadow generation. For example, when there is a metal foreign body in the patient's head, it may also cause a shadow, which is easy to be confused with the movement of the head. By setting the telescopic rod, the patient can be reminded while being extended, and a secondary mark can be made on the image to reduce sample errors.

[0069] We do not directly exclude X-ray images of the current time period when the reminder signal is generated, mainly to prevent the discarding of too many samples, and signal transmission is prone to delays and other problems, which can easily cause more errors, so we use image recognition to identify them.

[0070] Recognition of head CT images:

[0071] ①Identify normal head CT images.

[0072] Bone window and brain window images of normal human skull CT images were stored and analyzed separately. At the bone window, the Co-UNet model was used to segment the skull, including the facial and cranial bones, and extract features of these bony landmarks. At the brain window, the Co-UNet model was used to segment the brain parenchyma, including the supratentorial and infratentorial regions, extract features of these intraparenchymal landmarks, and calculate the average bilateral lateral ventricle area at different CT levels in normal subjects.

[0073] ② Recognition of head CT images of TBI patients.

[0074] The injuries of TBI patients were divided into skull injuries and brain injuries. The bone window and brain window levels of TBI patients' head CT images were stored and analyzed separately, and brain injuries and skull injuries were identified at different window levels.

[0075] Ⅰ. Identify brain injury at the brain window position.

[0076] i. Identify brain injuries, including lateralization, positioning, qualitative and quantitative identification. Lateralization: that is, determine the laterality of the injury, and clarify whether the injury is on the left or right side. Positioning: that is, determine the specific location of the injury, and clarify which lobe or part of the brain the injury is located, such as the frontal lobe, temporal lobe, brainstem, etc. Qualitative: that is, determine the nature of the brain injury. The nature of brain injuries is classified, including: epidural hematoma, subdural hematoma (including acute subdural hematoma, subacute subdural hematoma and chronic subdural hematoma), intracerebral hematoma, subarachnoid hemorrhage, cerebral contusion, diffuse axonal injury, traumatic cerebral infarction, etc. Quantitative: that is, measure the actual values ​​such as the size of the brain injury focus and the distance of brain tissue displacement. Quantitatively identify the severity of different types of brain injuries, including the volume of the injury focus (hematoma, cerebral contusion, etc.), the displacement of the midline structure, the compression of the lateral ventricle, etc. The Co-UNet model is used to extract brain injury lesions in the brain parenchyma area in the brain window position, and complete the lateralization, positioning, qualitative and quantitative analysis of the brain injury lesions.

[0077] i) Lesion volume calculation: The Co-UNet model was used to identify the lesion contours in each image layer, and the lesion area in each image layer was calculated. The lesion region was three-dimensionally modeled using an orthogonal hexahedral grid using the HP graphic language (PLT) grid data structure. The obtained three-dimensional lesion model volume represents the volume of the lesion region.

[0078] ii) Midline shift: The Co-UNet model was used to identify the anterior and posterior attachment points of the falx cerebri on the inner skull plate. An imaginary line connecting these two points was drawn as the ideal midline (iML). The Co-UNet model was also used to identify the septum pellucidum. The vertical distance between the septum pellucidum and the ideal midline was calculated at different levels. The maximum vertical distance represented the midline shift (MLS).

[0079] iii) Lateral ventricle compression: The Co-UNet model was used to extract bilateral lateral ventricle images at different CT slices. The area of ​​the lateral ventricles on the lesion side and the normal side, as well as their ratio, were calculated for each slice: the lateral ventricle compression index (LVCI) = (normal side lateral ventricle area - lesion side lateral ventricle area) / normal side lateral ventricle area × 100%. If bilateral lesions were present, the bilateral lateral ventricles were compared with the average lateral ventricle area from CT images of similar slices in healthy subjects: the LVCI = (normal ipsilateral average lateral ventricle area - lesion side lateral ventricle area) / normal ipsilateral average lateral ventricle area × 100%. The maximum value was taken as the final LVCI.

[0080] ⅱ. Summarize the lateralization, location, qualitative and quantitative identification of brain injury.

[0081] i) Epidural, subdural, and intracerebral hematomas: Lateral, localized, qualitative, and quantitative identification is required, including the following attributes: hematoma location (left or right, specific brain site or lobe), hematoma nature (epidural, subdural, intracerebral), hematoma volume (in milliliters), midline structure shift (MLS, in millimeters), and lateral ventricular compression index (LVCI, expressed as a percentage).

[0082] ii) Cerebral contusion: Lateral determination, localization, qualitative and quantitative identification are required, including the following attributes: location of cerebral contusion (left or right, specific cerebral lobe), volume of cerebral contusion (in milliliters), midline structure shift (MLS, in millimeters), and lateral ventricular compression index (LVCI, expressed as a percentage).

[0083] iii) Subarachnoid hemorrhage: It requires lateralization, localization, and qualitative identification, mainly including the location and nature of the lesion (left or right side, specific cerebral lobe, sulcus, or cisternal hemorrhage, etc.).

[0084] iv) Diffuse axonal injury: requires lateralization, localization, and qualitative identification, mainly including the location and nature of the injury (left or right side, specific cerebral lobe, small patchy hemorrhages near the midline structure or brain contusion, etc.).

[0085] v) Traumatic cerebral infarction: Lateral determination, localization, qualitative and quantitative identification are required, including the following attributes: infarct location (left or right, specific cerebral lobe), midline structure shift (MLS, in millimeters), and lateral ventricular compression index (LVCI, expressed as a percentage).

[0086] II. Identify skull injuries at the bone window.

[0087] i. Identify the location and type of skull injury, including lateralization, positioning, qualitative and quantitative identification. Skull injury types are divided into: common linear fractures (no displacement or sinking of the skull), skull base fractures (including anterior, middle and posterior skull base fractures), and depressed fractures. The Co-UNet model is used to extract depressed fractures in the bone window position. By comparing the shape of the skull on the normal side, the original shape of the skull is automatically restored. The distance from the actual position of the sunken fracture fragment to the edge of the skull with the original shape is measured, and this is used as the depth of the fracture fragment's sinking.

[0088] ⅱ. Summarize the lateralization, location, qualitative and quantitative identification of skull injuries.

[0089] i) Depressed fracture: The following attributes should be included: fracture site (left or right, specific location, whether located in the functional area and near the venous sinus), depression depth (in millimeters), midline structure displacement (MLS, in millimeters), and lateral ventricular compression index (LVCI, expressed as a percentage).

[0090] ii) Skull base fracture: The following attributes must be included: fracture location (left or right, anterior, middle, or posterior skull base).

[0091] iii) Common linear fracture: The following attributes must be included: fracture site (left or right, specific location).

[0092] 3) Output the interpretation results of the head CT image.

[0093] The output results are text information for use in the next step of medical text recognition, including conclusive results and descriptive results.

[0094] ① Conclusive findings: Based on neurosurgery and neuroimaging diagnostic thinking, the results of brain and / or skull injuries are output, including lateralization, localization, and qualitative results. For example, a cranial CT scan of a patient with TBI and concurrent brain and skull injuries would yield the following conclusive findings: "1. Left temporal bone fracture; 2. Left temporal epidural hematoma; 3. Right frontal depressed fracture."

[0095] ② Descriptive results: A more detailed description of the head CT image, including lateralization, positioning, qualitative, and quantitative results. If the patient's head CT conclusive results are: "1. Left temporal bone fracture; 2. Left temporal epidural hematoma; 3. Right frontal depressed fracture," the descriptive results will include: "1. Left temporal bone fracture: linear fracture of the left temporal bone; 2. Left temporal epidural hematoma, with a blood volume of approximately 20 ml, midline structures shifted to the right by approximately 2 mm, and left ventricular compression of approximately 25%; 3. Right frontal depressed fracture, with fracture fragments embedded approximately 3 mm deep and not near the functional area or venous sinus."

[0096] (2) Collection and interpretation of medical text information.

[0097] 1) Collection of medical text information.

[0098] ① Source of medical text information: The medical text information source is the emergency medical records (EMR) of TBI patients upon admission. Medical text information is collected from the emergency electronic medical records (EMRs) of TBI patients upon admission. To protect patient privacy, each patient's key personal information will be hidden and deleted.

[0099] ② Content of medical text information: Medical text information will include medical history, physical examination, and auxiliary examinations. Since the source of medical text information is emergency medical records, the medical history will mainly include: chief complaint, current medical history, and important past and personal history. The physical examination will mainly include the results of the neurological examination and other important physical examination results. Ancillary examinations will mainly include cranial CT examinations and other important emergency auxiliary examinations. Among them, the cranial CT examination results are the text results output by the system after automatically interpreting the cranial CT images.

[0100] Construct a TBI diagnosis and treatment knowledge representation system - TWS.

[0101] ① We collected domestic and international guidelines for the diagnosis and treatment of craniocerebral trauma and, in combination with clinical experience, refined the various protocols for the diagnosis and treatment of craniocerebral trauma. For example, we categorized treatment measures into diagnostic and therapeutic measures. Diagnostic measures include vascular examinations such as CTA and MRA, MRI, and coagulation tests. Therapeutic measures are further divided into rescue measures, emergency surgery, and drug therapy. Rescue measures are further subdivided into cardiopulmonary resuscitation, airway management, and maintenance of vital signs.

[0102] ② Based on the previous research results of this research team, key information that affects the diagnosis and treatment plan decision-making and prognosis during the diagnosis and treatment of TBI was extracted, such as the time of injury, injury mechanism, presence of elevated intracranial pressure, rate of disease progression, vital signs, state of consciousness, Glasgow Coma Scale (GCS) score, pupil size and light reflex, limb movement, intracranial hemorrhage volume, midline structure shift, and lateral ventricular compression.

[0103] ③ Based on previous research results and diagnosis and treatment guidelines on craniocerebral trauma, different weight scores were assigned to each type of key information. Through repeated training, internal and external validation, the mapping relationship between different clinical and CT manifestations and various clinical diagnosis and treatment plans was studied to form a scoring system with high accuracy, good specificity and sensitivity - TWS.

[0104] 3) Key named entity recognition

[0105] We plan to build a BERT-IDCNN-BiLSTM-CRF model based on the BERT pre-trained language model to better extract key named entities from global and local text. The entire system consists of four modules: BERT, IDCNN, BiLSTM, and CRF. The BERT module analyzes complex grammar to obtain word vector representations. Based on this, the word vector representations are optimized through the IDCNN and BiLSTM modules. The final text features are then input into the CRF for correction.

[0106] ①BERT module

[0107] BERT uses a masked language model, similar to a cloze test, for pre-training. The goal of a masked language model is to randomly mask some words in a sentence and replace them with a token [MASK]. A further goal of the masked language model is to predict the masked word based on the context to the left and right of the token. One of the keys to the BERT model is the introduction of a self-attention mechanism to calculate the correlation between words in a text sequence. The calculation method is shown in the following formula:

[0108]

[0109] Among them, Q, K, V are input vector matrices, QK T Indicates the calculation of the similarity matrix of the input vector, d k is the dimension of the input vector matrix K.

[0110] Downstream tasks of BERT, such as entailment and sentiment analysis, are processed using the first token of each sequence, which is called a classification-specific token [CLS]. In the BERT model, the final hidden state corresponding to the [CLS] token aggregates the feature representation of a sentence or a pair of sentences.

[0111] ②IDCNN module

[0112] IDCNN uses dilated convolution to reduce data loss, expand the perception range, and achieve longer capture of text by adjusting the dilation width of the convolution kernel. The receptive field calculation formula of dilated convolution is:

[0113] F i+1 =(2 i+2 -1) 2

[0114] The IDCNN module has three convolutional layers with dilation widths of 1, 1, and 2, respectively. The module undergoes four iterations, and parameter sharing during the iterations effectively prevents model overfitting. Each parameter increases linearly with the number of layers, while the receptive field value grows exponentially, enabling the receptive field to quickly cover the entire input sequence.

[0115] ③BiLSTM module

[0116] BiLSTM incorporates a gating mechanism within a recurrent neural network, namely an input gate, an output gate, and a forget gate. BiLSTM consists of a bidirectional LSTM, extracting global contextual features from both directions. The key to LSTM is the cell state, which is analogous to a conveyor belt. It operates directly across the entire chain, with only a few linear interactions. Information flows easily through it, remaining unchanged. LSTM has the ability to remove or add information to the cell state through carefully designed structures called "gates." Gates are a method for selectively passing information through. They consist of a sigmoid neural network layer and a pointwise multiplication operation.

[0117] ④CRF module

[0118] For key entity recognition, the correlation between labels is crucial, so instead of decoding each label independently, we use conditional random fields (CRF) to jointly model the label sequence. For an input word sequence X, there is a general label sequence Y that represents the word sequence X. The probability model of the conditional random field defines a distribution P(Y|X) over all possible label sequences Y given the input word sequence X, which is as follows:

[0119]

[0120] 4) Knowledge-based semantic analysis

[0121] In order to further perform feature analysis on key named entities, we plan to establish a Word2vec model to vectorize the relevant medical texts, and establish a medical text weight scoring mechanism and a medical text similarity measurement method based on knowledge.

[0122] ①Word2vec model

[0123] This project aims to develop a Word2vec model for computing persistent distributed representations of medical text. This model, a simple neural network with one hidden layer, generates text vectors from a corpus of medical text input. The Word2vec model first generates a vocabulary from the input text, then learns word vectors using backpropagation and stochastic gradient descent.

[0124] ②Medical text similarity measurement

[0125] This project establishes a similarity metric to analyze and match treatments based on weighted scores of medical presentations. Using Word2vec-generated medical text word vectors, we generate sentence vector representations for key texts. These sentence vector representations are derived using the following formula.

[0126]

[0127] Where V S is the sentence vector representation, V i is the word vector representation of word i, w(i) is the weight, and m is the number of words.

[0128] This method uses cosine similarity to match key medical texts with a weighted scoring table developed by medical experts, analyzing the corresponding treatment measures for medical manifestations. The cosine similarity between two vectors is calculated using formula (15). Cosine similarity ranges from 0 to 1. When the cosine similarity is close to 1, the two vectors are considered similar, and when the cosine similarity is close to 0, the two vectors are considered dissimilar. For different key medical entities, we experimentally develop corresponding similarity thresholds.

[0129]

[0130] Among them, v1 and v2 are different sentence vector representations.

[0131] 5) Knowledge representation system based on deep learning

[0132] The knowledge representation system analyzes key named entities extracted from TWS using regularized logical reasoning combined with deep learning semantic similarity matching. This deep learning-based knowledge system utilizes an end-to-end pipeline. First, BERT captures and extracts key named entities of interest from the original medical records, identifying clinical and CT factors relevant to treatment decisions. Second, these extracted key named entities are input into the knowledge representation system, which uses regularized natural language processing techniques and deep learning semantic similarity matching to perform a logical diagnosis of the entities. Regularization primarily applies to entity statements concerning numerical values, extracting the numbers and comparing them with threshold ranges within TWS to obtain different weighted scores. Semantic similarity matching primarily applies to text entities concerning CT findings. Features are extracted from the text descriptions and converted into vectorized representations using Word2vec. The vector representations of the entities are then compared to the vector representations of pre-established standard templates using cosine distance similarity.

[0133] 6) The output results after medical text information interpretation.

[0134] ① Output 1: Emergency diagnostic conclusion, including the lateralization, location, and qualitative information of the injury. For example, "left parietal depressed fracture," "right temporal acute subdural hematoma," "left frontotemporal parietal chronic subdural hematoma," etc.

[0135] ② Output result 2: It is an emergency diagnosis and treatment decision, including emergency diagnostic measures and emergency treatment measures. Among them, diagnostic measures include whether to conduct further examinations, such as cranial MRI, cerebral blood vessels, coagulation function, and related examinations of other organs. Therapeutic measures include surgical treatment and non-surgical treatment measures. Surgical treatment measures mainly include emergency craniotomy and hematoma removal surgery, recommended bone craniotomy surgery, drilling and drainage, ICP probe insertion, open craniocerebral injury debridement and suture, etc.; non-surgical treatment measures include: whether life support is needed (cardiopulmonary resuscitation, intravenous access establishment, endotracheal intubation and airway management, mechanical ventilation, etc.), and the use of emergency drugs (including hemostatic drugs, anti-epileptic drugs, mannitol, etc.).

[0136] (3) Analysis of the accuracy and efficiency of automatic diagnosis and treatment results.

[0137] 1) T-SYSTEM Accuracy Analysis:

[0138] ① Analysis of the accuracy of automatic recognition of head CT images based on the Co-UNet model. By comparing the conclusive and descriptive results obtained by neurosurgeons, the accuracy, specificity, and sensitivity of the imaging diagnostic results output by this system were determined.

[0139] ② Accuracy analysis of diagnosis and treatment decisions based on knowledge graph reasoning. By comparing the system’s output of the initial clinical diagnosis and treatment plan with those made by neurosurgeons, the accuracy, specificity, and sensitivity of the system’s output is determined.

[0140] 2) Efficiency analysis of T-SYSTEM.

[0141] ① T-SYSTEM Accuracy Analysis: By comparing with neurosurgeons, we tested the accuracy, specificity, and sensitivity of the Co-UNet model-based automatic recognition of cranial CT images and diagnosis and treatment decisions based on knowledge representation and deep learning.

[0142] ② T-SYSTEM efficiency analysis: By comparing the number and accuracy of head CT images and EMRs recognized by neurosurgeons and T-SYSTEM in the same unit time, the actual efficiency was analyzed.

[0143] As attached Figure 6As shown, the BERT-IDCNN-BiLSTM-CRF model also demonstrated excellent performance in recognizing TBI patient EMR text data. In a randomized, double-blind comparison of over 1,000 cases, compared with neurosurgeons, its sensitivity, specificity, accuracy, and AUC were 96.00% (93.52-98.48)%, 96.27% (91.79-100)%, 95.00% (91.10-98.90)%, and 0.956% (0.926-0.986), respectively. While there is room for further improvement in the module's comprehensive recognition capabilities, optimization and iteration are currently underway on a larger dataset. Universal validation has been completed on EMR text data and imaging data from multiple medical institutions.

[0144] The above are only embodiments of the present invention. The invention is not limited to the fields involved in this implementation case. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Ordinary technicians in the relevant field are aware of all common technical knowledge in the technical field to which the invention belongs before the application date or priority date, can obtain all existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the inspiration given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several variations and improvements can be made, which should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A clinical decision-making system for craniocerebral trauma based on text and image recognition, characterized by: include: Image recognition module: used to generate text reports after identifying CT images; Establish a mapping relationship between CT signs and clinical conditions and diagnostic and treatment measures; The image recognition module is also used to preprocess the skull CT images: save all skull CT images of normal people and patients in DICOM format, improve the Kalman filter algorithm to denoise the CT images, and use the LableImg tool to annotate the skull CT images; then divide the skull CT images of TBI patients into skull injuries and brain injuries, and construct a comprehensive image recognition model based on 3D-UNet and 2D-UNet neural networks at the bone window and brain window of the skull CT images, and name it Co-UNet model. The Co-UNet model is used to segment the skull CT images of TBI patients, and a deep convolutional network model with single connected domain boundary interpolation is designed to extract positive and negative images. The image feature information of routine head CT is used to identify abnormal information based on the DenseNet network, and the threshold segmentation algorithm is improved to perform secondary screening on the identification results, and the different types of skull injuries and brain injuries in the two-dimensional head CT images of TBI patients are identified in terms of side, location, quality and quantity. The CT image analyzed by the image recognition module comes from a CT detector; it includes a frame, and the CT detector includes an X-ray transmitter arranged in a horizontal direction and an X-ray receiver corresponding to the X-ray transmitter; the X-ray transmitter is fixed on the frame, and a support table is fixed on the frame between the X-ray transmitter and the X-ray receiver, and a headrest is fixed on the support table, and deflection blocks are hinged on both sides of the headrest, and the bottom of the deflection block is hinged The sliding rod, the support platform is provided with a first slide and a second slide from top to bottom, the first slide and the second slide are distributed in an eight-shaped shape, and an air duct is further provided in the support platform, and the air duct is respectively connected to the bottom of the first slide and the second slide, and the sliding rods on the left and right sides of the headrest are slidably connected to the first slide and the second slide respectively, and the bottom of the deflection block and the top of the support platform are fixed with a first spring, and the first spring is sleeved on the outside of the sliding rod; a third slide in a vertical direction is provided on the support platform near the X-ray emitter side, the bottom of the third slide is connected to the air duct, and a first lead block is slidably connected in the third slide, and a second spring is fixedly connected between the bottom of the first lead block and the inner wall of the third sliding bottom, and the third slide at the bottom of the first lead block, the two The first slide and the second slide at the bottom of the slide bar and the air duct form a closed space; when the slide bar is not pressed down, the first lead block is not within the irradiation range of the X-ray emitter; when the deflection block above the slide bar is pressurized, the first lead block is pushed out and appears within the irradiation range of the X-ray; the data processing module collects the images drawn by the X-ray detector, and discards the CT images with the lead block images and transmits them to the image recognition module; position sensors are provided in the first slide and the second slide, and the position sensors send a reminder signal when the slide bar moves downward in the first slide / the second slide; an electric telescopic rod is fixed to the frame above the headrest, the free end of the electric telescopic rod moves downward, and a second lead block is fixed to the bottom of the electric telescopic rod;The data processing module receives the reminder signal and controls the electric telescopic rod to start and extend the second lead block into the X-ray irradiation area. After the reminder signal disappears, the electric telescopic rod is reset. Key named entity recognition module: Based on the BERT pre-trained language model, a BERT-IDCNN-BiLSTM-CRF model is built to extract key named entities from the global or local content of text reports; Knowledge-based semantic representation module: This module performs feature analysis on key named entities, establishes a Word2vec model, vectorizes relevant medical text, and establishes a medical text weight scoring mechanism and a medical text similarity measurement method based on knowledge to obtain medical text. Deep learning decision module: After extracting medical text features, it understands the differences between medical terms, intelligently identifies and interprets medical texts for TBI patients, and makes diagnosis and treatment decisions.

2. The craniocerebral trauma clinical decision-making system based on text and image recognition according to claim 1, characterized in that: The deep learning decision-making module is also used to: use TBI patients' emergency electronic medical records as the source of medical text information, and collect content mainly including medical history, physical examination and auxiliary examination results; among them, the head CT examination results are the text results output by the image recognition module; guided by the latest domestic and international craniocerebral trauma diagnosis and treatment guidelines, combined with clinical practice experience and the research team's previous basic, clinical and applied research results, the medical history, physical examination, CT and other important auxiliary examinations of craniocerebral trauma patients are given different weight scores; at the same time, various diagnostic and treatment measures for craniocerebral trauma are refined, and after repeated clinical data verification, the mapping relationship between multidimensional, multi-source, heterogeneous data and corresponding diagnostic and treatment decisions is studied and constructed; and through intelligent recognition of TBI patients' head CT images and emergency electronic medical records, the diagnostic and treatment results are output in the form of Chinese text, including emergency diagnostic conclusions and emergency diagnostic and treatment decisions.

3. The craniocerebral trauma clinical decision-making system based on text and image recognition according to claim 1, characterized in that: When recognizing skull CT images, normal skull CT images are recognized: the bone window and brain window images of normal skull CT images are stored and analyzed separately; at the bone window, the skull is segmented using the Co-UNet model, including the facial skull and brain skull, and the features of these bony landmark structures are extracted; at the brain window, the brain parenchyma area is segmented using the Co-UNet model, including the supratentorial and infratentorial brain parenchyma areas, and the features of these landmark structures within the brain parenchyma are extracted, and the average area of ​​the bilateral lateral ventricles at different skull CT levels of normal people is calculated; Identification of TBI patients' head CT images: The injuries of TBI patients are divided into skull injuries and brain injuries. The bone window and brain window positions of TBI patients' head CT images are stored and analyzed separately, and brain injuries and skull injuries are identified at different window positions.

4. The craniocerebral trauma clinical decision-making system based on text and image recognition according to claim 3, characterized in that: Identifying brain window positions on cranial CT images of TBI patients mainly includes: identifying brain injuries, including lateralization, location, qualitative and quantitative identification; summarizing the lateralization, location, qualitative and quantitative identification of brain injuries; Identification of skull injuries at the bone window level of TBI patients' head CT images mainly includes: identification of the location and type of skull injury, including lateralization, positioning, qualitative and quantitative identification; and summary of lateralization, positioning, qualitative and quantitative identification of skull injuries.

5. The craniocerebral trauma clinical decision-making system based on text and image recognition according to claim 4, characterized in that: The text information output by the image recognition module includes conclusive results and descriptive results; the conclusive results: output the results of brain injury and / or skull injury according to the diagnostic thinking of neurosurgery and neuroimaging, including lateralization, positioning and qualitative results; the descriptive results: provide a more detailed description of the head CT image, including lateralization, positioning, qualitative and quantitative results.

6. The craniocerebral trauma clinical decision-making system based on text and image recognition according to claim 5, characterized in that: The key named entity recognition module includes: BERT module: BERT uses a masked language model similar to cloze test questions to complete pre-training. The purpose of the masked language model is to randomly mask some words in a sentence and replace the original words with a token [MASK]. The further goal of the masked language model is to predict the masked words based on the context to the left and right of the token. The BERT model calculation method is shown in the following formula: in, is the input vector matrix, Indicates the calculation of the similarity matrix of the input vector, is the input vector matrix Dimensions; In BERT’s downstream tasks, entailment and sentiment analysis are processed using the first token of each sequence, which is called a classification-specific token [CLS]. In the BERT model, the final hidden state corresponding to the [CLS] token aggregates the feature representation of a sentence or a pair of sentences. IDCNN module: IDCNN uses dilated convolution to reduce data loss, expand the perception range, and achieve longer capture of text by adjusting the dilation width of the convolution kernel. The receptive field calculation formula of dilated convolution is: The IDCNN module has three convolutional layers with dilation widths of 1, 1, and 2, respectively. The module undergoes four iterations, and parameter sharing during the iterations effectively prevents overfitting. Each parameter increases linearly with the number of layers, while the receptive field value increases exponentially, allowing the receptive field to quickly cover the entire input sequence. BiLSTM module: BiLSTM uses a gating mechanism in a recurrent neural network, namely an input gate, an output gate, and a forget gate. BiLSTM consists of a bidirectional LSTM, extracting global context features from two directions. CRF module: For an input word sequence , there exists a sequence of words General tag sequence , the probability model of the conditional random field is given an input word sequence All possible tag sequences A distribution is defined on , which has the following form: 。 7. The craniocerebral trauma clinical decision-making system based on text and image recognition according to claim 6, characterized in that: The knowledge-based semantic representation module is used to calculate a persistent distributed representation of medical text; a text vector is generated as output from the input of a medical text corpus through a neural network structure with a hidden layer; in the Word2vec model, a vocabulary is first generated from the input text, and then word vectors are learned through backpropagation and stochastic gradient methods; Also used for: A similarity measurement method is established to analyze and match corresponding treatment measures based on the weighted scores of medical manifestations. The sentence vector representation of key texts is generated using the medical text word vectors generated by Word2vec. The sentence vector representation is obtained by the following formula: in is the sentence vector representation, for The word vector representation of the word, is the weight, is the number of words; The similarity measurement method uses cosine similarity to match key medical texts with a weighted scoring table developed by medical experts, and analyzes corresponding treatment measures based on medical manifestations. Cosine similarity ranges from 0 to 1. When the cosine similarity is close to 1, the two vectors are considered similar, and when the cosine similarity is close to 0, the two vectors are considered dissimilar. For different key medical entities, corresponding similarity thresholds are developed through experiments. in Different sentence vector representations.

Images