Craniocerebral trauma clinical decision-making system based on text and image recognition

Abstract

The invention relates to the field of medical instruments, in particular to a craniocerebral trauma clinical decision-making system based on text and image recognition. Comprising an image recognition module used for recognizing a CT image and then generating a text report; establishing a mapping relation between the CT signs and clinical conditions and diagnosis and treatment measures; the key named entity recognition module is used for constructing a BERT-IDCNN-BiLSTM-CRF model on the basis of the BERT pre-training language model, and extracting key named entities from the global or local part of the text report; the knowledge-based semantic representation module is used for performing feature analysis on the key named entities, establishing a Word2vec model, performing vectorization representation on related medical texts, and establishing a medical text weight scoring mechanism and a medical text similarity measurement method based on knowledge to obtain medical texts; and the deep learning decision module is used for understanding the difference between the medical words after extracting the medical text features, performing intelligent identification and interpretation on the medical text of the TBI patient, and making a diagnosis and treatment decision. And the TBI treatment efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of medical devices, and in particular to a clinical decision-making system for craniocerebral trauma based on text and image recognition. Background Art

[0002] Traumatic brain injury (TBI) is one of the most common critical diseases in neurosurgery, with high morbidity and mortality rates, and poses a serious threat to the life and health of young and middle-aged people. At the same time, TBI not only seriously affects the physical and mental health and quality of life of patients, but also brings huge economic losses and heavy medical burdens to families and society. Therefore, studying how to improve the efficiency of TBI treatment and curb its harm has significant social benefits and economic value.

[0003] The invention patent with application number 202311626792.9 discloses a multi-task hybrid neural network driven CT image processing method and brain-like diagnostic system, which involves artificial intelligence medical imaging. The method includes generating a hybrid neural network, inputting a CT image, performing a hybrid pulse-convolution calculation at each layer; generating a common encoder and a multi-task decoder, and outputting a multi-task result image. The system includes a processor, a memory, and a display. The memory stores at least one instruction, which is loaded and executed by the processor to implement the following steps: acquiring a CT image; generating a common encoder based on a hybrid neural network, calling the encoder to encode the CT image, and generating joint features; generating a multi-task decoder based on a hybrid neural network, calling the decoder to perform classification operations and segmentation operations on the joint features, and obtaining a multi-task diagnosis result. The present invention can improve the efficiency of multi-task diagnosis of CT images, while ensuring the accuracy of multi-tasks and reducing the computing power consumption of the diagnostic system.

[0004] The above technology has completed the accurate recognition of CT images, identified image features according to image segmentation, and obtained multi-task diagnosis results of CT images. However, the above technology still requires doctors to make the final decision and diagnosis. There are many problems in the existing TBI diagnosis and treatment model, which have an adverse impact on its treatment efficiency. These problems can be summarized into three major "pain points": "Pain point" 1: There is a serious shortage of neurosurgeons who cannot cope with the growing number of TBI patients. "Pain point" 2: Long-term overload work leads to fatigue of doctors and reduces the efficiency of doctors' diagnosis and treatment. "Pain point" 3: Various factors may affect doctors' judgment and thus affect the treatment of TBI patients. For the above three "pain points", the simplest and most effective solution is to increase a large number of energetic, non-tired, calm and clinically experienced neurosurgeons in a short period of time, but this is obviously impossible. Now a technology is needed to improve the efficiency of TBI treatment. Summary of the invention

[0005] The present invention provides a clinical decision-making system for craniocerebral trauma based on text and image recognition, which can improve the efficiency of TBI treatment.

[0006] In order to solve the above technical problems, this application provides the following technical solutions:

[0007] Clinical decision-making system for craniocerebral trauma based on text and image recognition, including:

[0008] Image recognition module: used to generate text reports after identifying CT images; establish mapping relationships between CT signs and clinical conditions and diagnosis and treatment measures;

[0009] Key named entity recognition module: Based on the BERT pre-trained language model, a BERT-IDCNN-BiLSTM-CRF model is built to extract key named entities from the global or local content of the text report;

[0010] Knowledge-based semantic representation module: perform feature analysis on key named entities, establish a Word2vec model, vectorize the relevant medical text, and establish a medical text weight scoring mechanism and a medical text similarity measurement method based on knowledge to obtain medical text;

[0011] Deep learning decision module: It is used to understand the differences between medical terms after extracting medical text features, intelligently identify and interpret medical texts of TBI patients, and make diagnosis and treatment decisions.

[0012] The basic principle and beneficial effects of this scheme: Based on the previous basic, clinical and applied research, the diagnostic and treatment ideas of neurosurgeons for TBI are deeply studied and simulated, so as to build a new CDSS for craniocerebral trauma, namely "T-SYSTEM". T-SYSTEM will directly obtain key medical images and text information including medical history, physical examination, and important auxiliary examinations from the patient's electronic medical record (EMR) to avoid data loss caused by inconsistent information sources. After obtaining key information such as medical history, physical signs, and CT manifestations, the system draws a diagnosis and treatment conclusion through clinical analysis. At the same time, the system refines the diagnosis and treatment plan of TBI and prioritizes the treatment measures according to the severity of the disease, including both the diagnosis and treatment measures required for exclusion diagnosis and rescue measures. T-SYSTEM includes four major modules: (1) image recognition module; (2) key named entity recognition system; (3) knowledge-based semantic representation system; (4) deep learning decision system. In the image recognition module, CT manifestations will be carefully identified and a text report will be generated as an important data source for T-SYSTEM to interpret the condition and make decisions. On this basis, a mapping relationship between important CT signs and clinical conditions and treatment measures is established. At the same time, based on the BERT pre-trained language model, we built a BERT-IDCNN-BiLSTM-CRF model, which can well extract key named entities from global and local texts, especially in medical text recognition. The word vector space based on medical knowledge in the deep learning decision system is more suitable for medical scenarios. After extracting medical text features, the model can better understand the differences between medical words. Models based on deep learning can also provide sufficient representation capabilities for text to solve problems quickly and with high quality, and the information in the data can be effectively "encoded" in the model.

[0013] By recognizing CT images, text information is generated to show the features in the CT images, and then semantic recognition is performed on the features in the text information to obtain the diagnosis and treatment measures for the disease under the features, and then a corresponding decision is made. The decision is made based on the treatment measures corresponding to the disease. Through the intelligent learning of the doctor's treatment method through the data model, machine decision-making is achieved, manpower is liberated, and the recognition error problem that is prone to occur in long-term labor is reduced. It also improves processing efficiency and can diagnose many patients at the same time.

[0014] Furthermore, the image recognition module is also used to pre-process the head CT images: save the head CT images of all normal people and patients in DICOM format, improve the Kalman filter algorithm to denoise the CT images, and use the LableImg tool to annotate the head CT images; then divide the head CT images of TBI patients into skull injuries and brain injuries, respectively, in the bone window and brain window of the skull CT images, use the Co-UNet model to segment the head CT images of TBI patients, design a deep convolutional network model of single connected domain boundary interpolation to extract the image feature information of normal skull CT, realize the recognition of abnormal information based on the DenseNet network, improve the threshold segmentation algorithm to perform secondary screening on the recognition results, and determine the side, position, qualitative and quantitative recognition of different types of skull injuries and brain injuries in the two-dimensional images of TBI patients' skull CT. Beneficial effects: save the head CT images of all normal people and patients in DICOM format, improve the Kalman filter algorithm to denoise the CT images, use the LableImg tool to annotate the head CT images, and provide data guarantee for recognition.

[0015] Furthermore, the deep learning decision-making module is also used to: use the emergency electronic medical record (EMR) of TBI patients as the source of medical text information, and the collected content mainly includes medical history, physical examination and auxiliary examination results; among them, the head CT examination results are the text results output by the image recognition module; and guided by the latest domestic and international craniocerebral trauma diagnosis and treatment guidelines, combined with clinical practice experience and the previous basic, clinical and applied research results of this research team, the medical history, physical examination, CT and other important auxiliary examinations of craniocerebral trauma patients are given different weight scores; at the same time, various diagnostic and treatment measures for craniocerebral trauma are refined, and after repeated clinical data verification, the mapping relationship between multidimensional, multi-source, heterogeneous data and corresponding diagnostic and treatment decisions is studied and constructed; and through intelligent recognition of TBI patients' head CT images and EMR, the diagnostic and treatment results output in the form of Chinese text are obtained, including emergency diagnostic conclusions and emergency diagnostic and treatment decisions.

[0016] Furthermore, when identifying the head CT, the normal head CT image is identified: the bone window and brain window images of the normal head CT image are stored and analyzed separately; at the bone window, the skull is segmented by the Co-UNet model, including the facial skull and the brain skull, and the features of these bone landmark structures are extracted; at the brain window, the Co-UNet model is used to segment the brain parenchyma area, including the supratentorial and infratentorial brain parenchyma areas, and the features of these landmark structures in the brain parenchyma are extracted, and the average area of ​​the bilateral lateral ventricles at different head CT levels of normal people is calculated;

[0017] Identification of head CT images of TBI patients: The injuries of TBI patients are divided into skull injuries and brain injuries, and the bone window and brain window positions of TBI patients' head CT images are stored and analyzed separately, and brain injuries and skull injuries are identified at different window positions.

[0018] Furthermore, the identification of brain window positions in the cranial CT images of TBI patients mainly includes: identification of brain injury, including lateralization, location, qualitative and quantitative identification; summarizing the lateralization, location, qualitative and quantitative identification of brain injury;

[0019] The identification of skull injuries at the bone window level of the head CT images of TBI patients mainly includes: identification of the location and type of skull injury, including lateralization, positioning, qualitative and quantitative identification; and summary of lateralization, positioning, qualitative and quantitative identification of skull injuries.

[0020] Furthermore, the text information output by the image processing module includes conclusive results and descriptive results; wherein the conclusive results: output the results of brain injury and / or skull injury according to the diagnostic thinking of neurosurgery and neuroimaging, including lateral determination, positioning and qualitative results; the descriptive results: provide a more detailed description of the head CT image, including lateral determination, positioning, qualitative and quantitative results.

[0021] Furthermore, the key named entity recognition module includes:

[0022] BERT module:

[0023] BERT uses a masked language model similar to the cloze test to complete pre-training. The purpose of the masked language model is to randomly mask some words in a sentence and replace the original words with a tag [MASK]. The further goal of the masked language model is to predict the masked words based on the context on the left and right of the tag. The BERT model calculation method is shown in formula (1):

[0024]

[0025] Among them, Q, K, V are input vector matrices, QK T Indicates the calculation of the similarity matrix of the input vector, d k is the dimension of the input vector matrix K.

[0026] Downstream tasks of BERT, such as entailment and sentiment analysis, are processed using the first token of each sequence, which is called a classification-specific token [CLS]. In the BERT model, the final hidden state corresponding to the [CLS] token aggregates the feature representation of a sentence or a pair of sentences.

[0027] IDCNN module:

[0028] IDCNN uses dilated convolution. By adjusting the dilation width of the convolution kernel, it reduces data loss, expands the perception range, and achieves longer capture of text. The calculation formula for the receptive field of dilated convolution is:

[0029] F i+1 =(2 i+2 -1) 2

[0030] There are three convolutional layers in the IDCNN module, with expansion widths of 1, 1, and 2 respectively. The module undergoes four iterations in total, and parameter sharing during the iteration process effectively prevents the model from overfitting. Each parameter increases linearly with the number of layers, while the receptive field value increases exponentially, allowing the receptive field to quickly cover the entire input sequence.

[0031] BiLSTM module:

[0032] BiLSTM uses the gating mechanism in the recurrent neural network, namely the input gate, output gate and forget gate; BiLSTM consists of a bidirectional LSTM to extract global context features from two directions respectively;

[0033] CRF module:

[0034] For an input word sequence X, there is a general label sequence Y representing the word sequence X. The probability model of the conditional random field defines a distribution P(Y|X) over all possible label sequences Y given the input word sequence X, which is in the following form:

[0035]

[0036] Beneficial effects: In view of the characteristics of unstructured data noise, multiple fine-grained texts, difficult information extraction, and unstable recognition accuracy in the construction process of medical-related knowledge representation systems and their semantic analysis systems, we intend to build a BERT-IDCNN-BiLSTM-CRF model specifically for the characteristics of medical text information to solve the key problem of accurately identifying key medical-related named entities.

[0037] Further, the knowledge-based semantic representation module is used to calculate the continuous distributed representation of medical text; a text vector is generated as an output from the input of the medical text corpus through a neural network structure with a hidden layer; a vocabulary is first generated from the input text in the Word2vec model, and then the word vector is learned through back propagation and stochastic gradient method;

[0038] Also used for:

[0039] A similarity measurement method is established to analyze and match corresponding treatment measures according to the weight score of medical manifestations; the medical text word vectors generated by Word2vec are used to generate sentence vector representations of key texts, and the sentence vector representation is obtained by the following formula.

[0040]

[0041] Where V S is the sentence vector representation, V i is the word vector representation of word i, w(i) is the weight, and m is the number of words;

[0042] This method uses cosine similarity to match key medical texts with weighted scoring tables developed by medical experts, and analyzes corresponding treatment measures for medical manifestations; cosine similarity is between 0 and 1. When the cosine similarity is close to 1, the two vectors are said to be similar, and when the cosine similarity is close to 0, the two vectors are said to be dissimilar; for different key medical entities, we develop corresponding similarity thresholds through experiments;

[0043]

[0044] Among them, v1 and v2 are different sentence vector representations.

[0045] Furthermore, the CT image analyzed by the image recognition module originates from a CT detector; it includes a frame, the CT detector includes an X-ray transmitter arranged in a horizontal direction and an X-ray receiver corresponding to the X-ray transmitter; the X-ray transmitter is fixed on the frame, a support table is fixed on the frame between the X-ray transmitter and the X-ray receiver, a headrest is fixed on the support table, deflection blocks are hinged on both sides of the headrest, a sliding rod is hinged on the bottom of the deflection block, the support table is provided with a first slide and a second slide from top to bottom, the first slide and the second slide are distributed in an eight-shaped shape, an air duct is also provided in the support table, the air duct is connected to the bottom of the first slide and the second slide respectively, the sliding rods on the left and right sides of the headrest are slidably connected to the first slide and the second slide respectively, the bottom of the deflection block and the support A first spring is fixed on the top of the table, and the first spring is sleeved on the outside of the sliding rod; a third slide in the vertical direction is opened on the support table close to the X-ray transmitter side, the bottom of the third slide is connected to the air duct, and a first lead block is slidably connected in the third slide, and a second spring is fixedly connected between the bottom of the first lead block and the inner wall of the third sliding bottom. The third slide at the bottom of the first lead block, the first slide and the second slide at the bottom of the two sliding rods, and the air duct form a closed space; when the sliding rod is not pressed down, the first lead block is not within the irradiation range of the X-ray transmitter; when the deflection block above the sliding rod is pressurized, the first lead block is pushed out and appears within the irradiation range of the X-ray; the data processing module collects the pictures drawn by the X-ray detector, and discards the CT pictures with the lead block images and transmits them to the image recognition module.

[0046] Furthermore, position sensors are arranged in the first slide and the second slide, and the position sensors send out a reminder signal when the slide rod moves downward in the first slide / the second slide; an electric telescopic rod is fixed on the frame above the headrest, and the moving direction of the free end of the electric telescopic rod is downward, and a second lead block is fixed at the bottom of the electric telescopic rod; the data processing module receives the reminder signal, controls the electric telescopic rod to start, and extends the second lead block into the X-ray irradiation area, and the electric telescopic rod is reset after the reminder signal disappears.

[0047] Beneficial effects: When performing head CT examination on children, due to their poor self-control, their heads may shake during the examination, resulting in unnecessary movements that affect the final test results; for example, any movement of the head may cause the image to be blurred, affecting the image quality. Artifact generation: Inappropriate movement may produce artifacts on the image, which will interfere with the interpretation of the image and may lead to misdiagnosis or missed diagnosis. This solution uses a decision-making system to issue diagnosis and treatment opinions on the patient's symptoms. If the CT image is deviated, it will greatly affect the accuracy. Therefore, this solution sets the extension and contraction of the lead block, which will trigger the deflection of the deflection block when the patient moves, thereby squeezing the airway. The first lead block is pushed out when it is compressed and appears in the irradiation area of ​​the X-ray. At this time, an obvious shadow will appear on the image, indicating that this image is unavailable. The data processing module identifies the image with shadows and discards it, ensuring that the CT image transmitted to the image processing module is completely correct, improving the accuracy of the data sample, and thereby reducing the probability of errors in the decision-making system. In addition, this solution does not stop and continuously detects CT, which can prevent radiation hazards caused by multiple irradiations when there are defects in the image.

[0048] In order to prevent measurement errors, the telescopic rod is set to send a signal only when the sliding rod moves down (that is, the deflection block is under pressure) to control the telescopic rod to be pushed out. That is, two shadows may appear on the image taken when the patient is deflected, thereby reducing the error in the shadow generation. For example, when there is a metal foreign body in the patient's head, it may also cause a shadow, which can easily be confused with the movement of the head. By setting the telescopic rod, the patient can be reminded while extending it, and a secondary mark can be made on the image to reduce sample errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a schematic diagram of the T-SYSTEM process for diagnosing and treating TBI patients;

[0050] Figure 2 This is a schematic diagram of the T-SYSTEM workflow;

[0051] Figure 3 This is a schematic diagram of the construction process of T-SYSTEM;

[0052] Figure 4 It is a knowledge representation system related to TBI diagnosis and treatment;

[0053] Figure 5 The process of identifying different craniocerebral injuries by the image recognition system;

[0054] Figure 6 This is a comparison chart of multiple models;

[0055] Figure 7 It is a structural schematic diagram of a CT detector;

[0056] Figure 8 for Figure 7 Another state diagram of .

[0057] Figure numbers in the specification: support platform 1, first slide 11, air duct 12, second slide 13, slide rod 14, first spring 15, deflection block 16, headrest 17, second spring 2, first lead block 3, support frame 4, X-ray transmitter, X-ray receiver 6, frame 7, electric telescopic rod 8. DETAILED DESCRIPTION

[0058] The following is further described in detail through specific implementation methods:

[0059] Since the contents that need to be automatically segmented and recognized in craniocerebral trauma CT images include multiple tasks such as hematoma volume, degree of lateral ventricle compression, midline structure displacement, fracture location and depth, and involve two-dimensional and three-dimensional image information, we combine 3D-UNet and 2D-UNet to build a comprehensive image recognition model based on 3D-UNet and 2D-UNet neural networks, and name it Co-UNet model, which can automatically segment and recognize cranial CT images of patients with craniocerebral trauma.

[0060] Conventional 2D-UNet models cannot make full use of data information between different layers, and the model lacks the use of spatial information; while the 3D-UNet model can use more spatial information, but the number of parameters is large and requires a lot of computing resources. The 3D-UNet structure used in this project is mainly composed of encoder, decoder, dense connection and deep supervision. Among them, the encoder performs convolution operations, ReLU activation and batch normalization to extract image features; the decoder uses the transposed convolution layer for upsampling to achieve feature fusion and restore the segmentation results; the encoder and decoder are connected through jump connections to provide high-level semantic image information for the network, thereby including sufficient spatial and semantic information features. The deep supervision structure is used to accelerate the convergence of the network to improve the learning ability of deep information and shallow information. The 2D-UNet result mainly includes an encoder and a decoder, in which the encoder is used to extract image features, and the decoder maps these features back to the size of the original image and generates segmentation results. The encoder and decoder of the network are composed of four competitive dense blocks (CDBs) respectively. The activation function maxout is introduced to increase the flexibility and expression ability of the network and improve network performance. The dense connections within the convolutional blocks are achieved by concatenating features, which effectively doubles the number of learnable parameters within each encoder and decoder block, greatly increasing the memory requirements. Each block consists of a sequence of three steps: parameter rectified linear unit (PReLU), convolution (Conv), and batch normalization (BN).

[0061] Embodiment 1 is as attached Figure 1-5 As shown, the clinical decision-making system for craniocerebral trauma based on text and image recognition includes: an image recognition module: used to generate a text report after recognizing CT images; to establish a mapping relationship between CT signs and clinical conditions and diagnosis and treatment measures;

[0062] Key named entity recognition module: Based on the BERT pre-trained language model, a BERT-IDCNN-BiLSTM-CRF model is built to extract key named entities from the global or local content of the text report;

[0063] Knowledge-based semantic representation module: perform feature analysis on key named entities, establish a Word2vec model, vectorize the relevant medical text, and establish a medical text weight scoring mechanism and a medical text similarity measurement method based on knowledge to obtain medical text;

[0064] Deep learning decision module: It is used to understand the differences between medical terms after extracting medical text features, intelligently identify and interpret medical texts of TBI patients, and make diagnosis and treatment decisions.

[0065] The image processing module collects head CT images and completes image preprocessing. A head CT image database for TBI patients and normal subjects is established, and all head CT images are saved in DICOM format. Figure 7-8 As shown, the CT image analyzed by the image recognition module comes from a CT detector; the CT detector includes a frame 7, and the CT detector includes an X-ray transmitter arranged in a horizontal direction and an XX light receiver 6 corresponding to the X-ray transmitter; the X-ray transmitter is fixed on the frame 7, and a support table 1 is fixed on the frame 7 between the X-ray transmitter and the XX light receiver 6, and a headrest 17 is fixed on the support table 1, and a deflection block 16 is hinged on both sides of the headrest 17, and a slide bar 14 is hinged on the bottom of the deflection block 16, and the support table 1 is provided with a first slide 11 and a second slide 13 from top to bottom, and the first slide 11 and the second slide 13 are distributed in an eight-shaped shape, and an air duct 12 is also provided in the support table 1, and the air duct 12 is respectively connected to the bottom of the first slide 11 and the second slide 13, and the slide bars 14 on the left and right sides of the headrest 17 are respectively slidably connected to the first slide 11 and the second slide 13, so A first spring 15 is fixed at the bottom of the deflection block 16 and the top of the support platform 1, and the first spring 15 is sleeved on the outside of the slide rod 14; a third slide in the vertical direction is opened on the support platform 1 close to the X-ray transmitter side, and the bottom of the third slide is connected to the air duct 12, and a first lead block 3 is slidably connected in the third slide, and a second spring 2 is fixedly connected between the bottom of the first lead block 3 and the inner wall of the third sliding bottom, the third slide at the bottom of the first lead block 3, the first slide 11 and the second slide 13 at the bottom of the two slide rods 14, and the air duct 12 form a closed space; when the slide rod 14 is not pressed down, the first lead block 3 is not within the irradiation range of the X-ray transmitter; when the deflection block 16 above the slide rod 14 is pressed, the first lead block 3 is pushed out and appears within the irradiation range of the X-ray; the data processing module collects the pictures drawn by the X-ray detector, and discards the CT pictures with lead block images and transmits them to the image recognition module.

[0066] Position sensors are arranged in the first slide 11 and the second slide 13, and the position sensors send out a reminder signal when the slide bar 14 moves downward in the first slide 11 / the second slide 13; an electric telescopic rod 8 is fixed on the frame 7 above the headrest 17, and the moving direction of the free end of the electric telescopic rod 8 is downward, and a second lead block is fixed at the bottom of the electric telescopic rod 8; the data processing module receives the reminder signal, controls the electric telescopic rod 8 to start, and extends the second lead block into the X-ray irradiation area, and the electric telescopic rod 8 is reset after the reminder signal disappears.

[0067] When performing head CT scans on children, due to their poor self-control, their heads may shake during the test, resulting in unnecessary movements that affect the final test results; for example, any movement of the head may cause the image to appear blurred, affecting the image quality. Artifact generation: Inappropriate movement may produce artifacts on the image, which will interfere with the interpretation of the image and may lead to misdiagnosis or missed diagnosis. This solution uses a decision-making system to issue diagnosis and treatment opinions for the patient's condition. If the CT image is deviated, it will greatly affect the accuracy. Therefore, this solution sets the extension and contraction of the lead block, which will trigger the deflection of the deflection block 16 when the patient moves, thereby squeezing the airway 12. The first lead block 3 is pushed out when under pressure and appears in the X-ray irradiation area. At this time, an obvious shadow will appear on the image, indicating that this image is unavailable. The data processing module identifies and discards images with shadows, ensuring that the CT images transmitted to the image processing module are completely correct, improving the accuracy of data samples, and thereby reducing the probability of errors in the decision-making system. In addition, this solution does not stop the machine and continuously detects CT, which can prevent radiation hazards caused by multiple exposures if defects appear in the image.

[0068] In order to prevent measurement errors, the telescopic rod is set up so that a signal is sent only when the sliding rod 14 moves down (that is, the deflection block 16 is under pressure) to control the telescopic rod to be pushed out. That is, two shadows may appear on the image taken when the patient is deflected, thereby reducing the error in the shadow. For example, when there is a metal foreign body in the patient's head, a shadow may be caused, which may be easily confused with the movement of the head. The telescopic rod can be set up to remind the patient while extending it, and a secondary mark can be made on the image to reduce sample errors.

[0069] We do not directly exclude X-ray images of the current time period when the reminder signal is generated, mainly to prevent discarding too many samples, and signal transmission is prone to delays and other problems, which can easily cause more errors, so we use image recognition to identify them.

[0070] Recognition of head CT images:

[0071] ①Identify normal head CT images.

[0072] The bone window and brain window images of normal head CT images were stored and analyzed separately. In the bone window, the skull was segmented using the Co-UNet model, including the facial skull and the brain skull, and the features of these bone landmark structures were extracted. In the brain window, the Co-UNet model was used to segment the brain parenchyma area, including the supratentorial and infratentorial brain parenchyma areas, to extract the features of these landmark structures in the brain parenchyma, and calculate the average area of ​​the bilateral lateral ventricles at different skull CT levels of normal people.

[0073] ②Recognition of head CT images of TBI patients.

[0074] The injuries of TBI patients were divided into skull injuries and brain injuries. The bone window and brain window levels of TBI patients' head CT images were stored and analyzed separately, and brain injuries and skull injuries were identified at different window levels.

[0075] Ⅰ. Identify brain injury at the brain window position.

[0076] ⅰ. Identify brain injuries, including side determination, location, qualitative and quantitative identification. Side determination: that is, determine the side of the injury focus, and clarify whether the injury focus is on the left or right side. Location: that is, determine the specific location of the injury focus, and clarify which brain lobe or part the injury focus is located, such as the frontal lobe, temporal lobe, brain stem, etc. Qualitative: that is, determine which category the nature of the brain injury belongs to. The nature of brain injuries is classified, including: epidural hematoma, subdural hematoma (including acute subdural hematoma, subacute subdural hematoma and chronic subdural hematoma), intracerebral hematoma, subarachnoid hemorrhage, cerebral contusion, diffuse axonal injury, traumatic cerebral infarction, etc. Quantitative: that is, measure the actual values ​​of the size of the brain injury focus, the distance of brain tissue displacement, etc. Quantitatively identify the degree of injury of different types of brain injuries, including the volume of the injury focus (hematoma, brain contusion, etc.), the displacement of the midline structure, the compression of the lateral ventricle, etc. The Co-UNet model is used to extract brain injury lesions in the brain parenchyma area in the brain window position, and complete the lateralization, positioning, qualitative and quantitative analysis of the brain injury lesions.

[0077] i) Calculation of lesion volume: The Co-UNet model is used to identify the lesion contours in each layer of the image, and the lesion area in each layer of the image is calculated. The lesion area is three-dimensionally modeled with an orthogonal hexahedral grid using the HP graphic language (PLT) grid data structure, so that the obtained three-dimensional model volume represents the volume of the lesion area.

[0078] ⅱ) Midline structure shift: The Co-UNet model was used to identify the attachment points of the falx cerebri in front and behind the inner plate of the skull, and an imaginary line was drawn between the two points as the ideal midline (iML). The Co-UNet model was used to identify the septum pellucidum, and the vertical distance between the septum pellucidum and the ideal midline was calculated at different levels. The maximum vertical distance represented the midline structure shift (MLS).

[0079] ⅲ) Compression of the lateral ventricles: The Co-UNet model was used to extract bilateral lateral ventricle images at different CT levels, and the area values ​​and ratios of the lateral ventricles on the lesion side and the normal side were calculated at each level, namely, the lateral ventricle compression index (LVCI) = (normal side lateral ventricle area - lesion side lateral ventricle area) / normal side lateral ventricle area × 100%. If there were lesions on both sides of the brain parenchyma, the average area of ​​the lateral ventricles on both sides was compared with that on the CT images of similar levels of normal people, namely, LVCI = (average area of ​​the ipsilateral lateral ventricles of normal people - area of ​​the lateral ventricles on the lesion side) / average area of ​​the ipsilateral lateral ventricles of normal people × 100%. The maximum value was taken as the final LVCI.

[0080] ⅱ. Summarize the lateralization, location, qualitative and quantitative identification of brain injury.

[0081] i) Epidural, subdural and intracerebral hematomas: Lateral, localized, qualitative and quantitative identification is required, including the following attributes: hematoma location (left or right, specific brain location or lobe location), hematoma nature (epidural? subdural? intracerebral?), hematoma volume (in milliliters), midline structure shift (MLS, in millimeters), and lateral ventricular compression index (LVCI, expressed as a percentage).

[0082] ii) Cerebral contusion: It needs to be identified laterally, locally, qualitatively, and quantitatively, including the following attributes: location of the contusion (left or right, specific lobe of the brain), volume of the contusion (in milliliters), midline structure shift (MLS, in millimeters), and lateral ventricular compression index (LVCI, expressed as a percentage).

[0083] ⅲ) Subarachnoid hemorrhage: It needs to be determined on the side, positioned and identified qualitatively, mainly including the location and nature of the lesion (left or right side, bleeding in a specific cerebral lobe, sulcus or cisterna, etc.).

[0084] iv) Diffuse axonal injury: It requires lateralization, positioning, and qualitative identification, mainly including the location and nature of the injury (left or right side, specific cerebral lobe, small patchy hemorrhages near the midline structure or brain contusion, etc.).

[0085] v) Traumatic cerebral infarction: Lateral determination, localization, qualitative and quantitative identification are required, including the following attributes: infarct site (left or right, specific cerebral lobe), midline structure shift (MLS, in millimeters), and lateral ventricular compression index (LVCI, expressed as a percentage).

[0086] Ⅱ. Identify skull injuries at the bone window.

[0087] ⅰ. Identify the location and type of skull injury, including lateralization, positioning, qualitative and quantitative identification. The types of skull injuries are divided into: common linear fractures (no displacement or sinking of the skull), skull base fractures (including anterior, middle and posterior skull base fractures) and depressed fractures. The Co-UNet model is used to extract depressed fractures in the bone window position, and the original shape of the skull is automatically restored by comparing with the shape of the skull on the normal side. The distance from the actual position of the sunken fracture fragment to the edge of the skull with the original shape of the skull is measured, and this is used as the depth of the fracture fragment sinking.

[0088] ⅱ. Summarize the lateralization, location, qualitative and quantitative identification of skull injuries.

[0089] i) Depressed fracture: The following attributes need to be included: fracture site (left or right, specific site, whether located in the functional area and near the venous sinus), depression depth (in millimeters), midline structure displacement (MLS, in millimeters), and lateral ventricular compression index (LVCI, expressed as a percentage).

[0090] ii) Skull base fracture: The following attributes need to be included: fracture site (left or right, anterior, middle, or posterior skull base).

[0091] ⅲ) Ordinary linear fracture: The following attributes need to be included: fracture site (left or right, specific site).

[0092] 3) Output the interpretation results of the head CT image.

[0093] The output result is text information for use in the next step of medical text recognition, including conclusive results and descriptive results.

[0094] ① Conclusive results: According to the diagnostic thinking of neurosurgery and neuroimaging, the results of brain injury and / or skull injury are output, including lateralization, positioning and qualitative results. Taking the head CT image of a TBI patient with both brain injury and skull injury as an example, the conclusive results are: "1. Left temporal bone fracture; 2. Left temporal epidural hematoma; 3. Right frontal bone depressed fracture".

[0095] ②Descriptive results: A more detailed description of the head CT image, including the lateralization, positioning, qualitative and quantitative results. If the patient's head CT conclusive results are: "1. Left temporal bone fracture; 2. Left temporal epidural hematoma; 3. Right frontal depressed fracture", then the descriptive results will include, "1. Left temporal bone fracture: linear fracture of the left temporal bone; 2. Left temporal epidural hematoma, bleeding volume of about 20 ml, midline structure shifted to the right about 2 mm, left ventricle compressed about 25%; 3. Right frontal depressed fracture, fracture fragments sunk about 3 mm deep, not close to the functional area and venous sinus."

[0096] (2) Collection and interpretation of medical text information.

[0097] 1) Collection of medical text information.

[0098] ① Source of medical text information: The source of medical text information is the emergency medical record (EMR) of TBI patients, and the medical text information in the emergency electronic medical record of TBI patients when they are admitted to the hospital is collected. To protect patient privacy, the key personal information of each patient will be hidden and deleted.

[0099] ②Content of medical text information: Medical text information will include medical history, physical examination and auxiliary examination. Since the source of medical text information is the emergency medical record, the medical history will mainly include: chief complaint, current medical history, and important past history and personal history. Physical examination will mainly include the results of neurological examination and other important system physical examination results. Auxiliary examinations mainly include head CT examination and other important emergency auxiliary examinations. Among them, the head CT examination results are the text results output by this system after automatically interpreting the head CT image.

[0100] Construct TBI diagnosis and treatment knowledge representation system - TWS.

[0101] ① Collect domestic and foreign craniocerebral trauma diagnosis and treatment guidelines, combine clinical practice experience, and refine various plans in the diagnosis and treatment of craniocerebral trauma, such as dividing diagnosis and treatment measures into diagnostic measures and therapeutic measures. Diagnostic measures include vascular examinations such as CTA and MRA, MRI examinations, coagulation function tests, etc. Therapeutic measures are further divided into rescue measures, emergency surgical treatment, and drug treatment. Rescue measures are further subdivided into cardiopulmonary resuscitation, airway management, and maintenance of vital signs.

[0102] ② Based on the previous research results of this research team, key information that affects the diagnosis and treatment plan decision-making and prognosis during the diagnosis and treatment of TBI was extracted, such as the time of injury, injury mechanism, whether there are signs of high intracranial pressure, the rate of disease progression, vital signs, state of consciousness, GCS score, pupil size and light reflex, limb activity, intracranial hemorrhage volume, midline structure shift, lateral ventricle compression, etc.

[0103] ③ Based on the previous research results and diagnosis and treatment guidelines on craniocerebral trauma, different weight scores are assigned to each type of key information. Through repeated training, internal and external verification, the mapping relationship between different clinical and CT manifestations and various clinical diagnosis and treatment plans is studied to form a scoring system with high accuracy, good specificity and sensitivity - TWS.

[0104] 3) Key named entity recognition

[0105] We plan to build a BERT-IDCNN-BiLSTM-CRF model based on the BERT pre-trained language model, so as to better extract key named entities from global and local texts. The entire system consists of four modules: BERT, IDCNN, BiLSTM, and CRF. The BERT module obtains the expression of word vectors by analyzing complex grammar; on this basis, the word vector expression will be optimized through the IDCNN and BiLSTM modules; the final text features are input into the CRF for correction.

[0106] ①BERT module

[0107] BERT uses a masked language model similar to cloze test to complete pre-training. The purpose of the masked language model is to randomly mask some words in a sentence and replace the original words with a token [MASK]. A further goal of the masked language model is to predict the masked words based on the context to the left and right of the token. One of the keys to the BERT model is the introduction of a self-attention mechanism to calculate the correlation between words in a text sequence. The calculation method is shown in the following formula:

[0108]

[0109] Among them, Q, K, V are input vector matrices, QK T Indicates the calculation of the similarity matrix of the input vector, d k is the dimension of the input vector matrix K.

[0110] Downstream tasks of BERT, such as entailment and sentiment analysis, are processed using the first token of each sequence, which is called a classification-specific token [CLS]. In the BERT model, the final hidden state corresponding to the [CLS] token aggregates the feature representation of a sentence or a pair of sentences.

[0111] ②IDCNN module

[0112] IDCNN uses dilated convolution to reduce data loss, expand the perception range, and achieve longer capture of text by adjusting the dilation width of the convolution kernel. The calculation formula for the receptive field of dilated convolution is:

[0113] F i+1 =(2 i+2 -1) 2

[0114] There are three convolutional layers in the IDCNN module, with expansion widths of 1, 1, and 2 respectively. The module undergoes 4 iterations in total, and parameter sharing during the iteration process effectively prevents the model from overfitting. Each parameter increases linearly with the number of layers, while the receptive field value increases exponentially, allowing the receptive field to quickly cover the entire input sequence.

[0115] ③BiLSTM module

[0116] BiLSTM uses the gating mechanism in recurrent neural networks, namely input gate, output gate and forget gate. BiLSTM consists of bidirectional LSTM, which extracts global context features from two directions. The key to LSTM is the cell state, which is similar to a conveyor belt. It runs directly on the entire chain with only a small amount of linear interaction. It is easy for information to flow unchanged on it. LSTM has the ability to remove or add information to the cell state through carefully designed structures called "gates". Gates are a way to selectively let information pass through. They contain a sigmoid neural network layer and a pointwise multiplication operation.

[0117] ④CRF module

[0118] For key entity recognition, the correlation between labels is crucial, so instead of decoding each label independently, we use conditional random fields (CRF) to jointly model the label sequence. For an input word sequence X, there is a general label sequence Y representing the word sequence X. The probability model of the conditional random field defines a distribution P(Y|X) over all possible label sequences Y given the input word sequence X, which is as follows:

[0119]

[0120] 4) Knowledge-based semantic analysis

[0121] In order to further perform feature analysis on key named entities, we plan to establish a Word2vec model to vectorize the relevant medical texts, and establish a medical text weight scoring mechanism and a medical text similarity measurement method based on knowledge.

[0122] ①Word2vec model

[0123] This project intends to build a Word2vec model for computing persistent distributed representations of medical text. The model has a simple neural network structure with one hidden layer, which generates text vectors as output from the input of a medical text corpus. In the Word2vec model, a vocabulary is first generated from the input text, and then word vectors are learned through back propagation and stochastic gradient methods.

[0124] ② Medical text similarity measurement

[0125] This project establishes a similarity measurement method to analyze and match the corresponding treatment measures according to the weight score of medical performance. The sentence vector representation of the key text is generated by using the medical text word vector generated by Word2vec. The sentence vector representation is obtained by the following formula.

[0126]

[0127] Where V S is the sentence vector representation, V i is the word vector representation of word i, w(i) is the weight, and m is the number of words.

[0128] This method uses cosine similarity to match key medical texts with the weighted scoring table developed by medical experts, and analyzes the corresponding treatment measures for medical manifestations. The cosine similarity between two vectors is calculated by formula (15). The cosine similarity is between 0 and 1. When the cosine similarity is close to 1, the two vectors are said to be similar, and when the cosine similarity is close to 0, the two vectors are said to be dissimilar. For different key medical entities, we formulate corresponding similarity thresholds through experiments.

[0129]

[0130] Among them, v1 and v2 are different sentence vector representations.

[0131] 5) Knowledge representation system based on deep learning

[0132] The knowledge representation system analyzes the extracted key named entities according to TWS, using regularized logical judgment combined with deep learning semantic similarity matching. The knowledge system based on deep learning adopts an end-to-end pipeline mode. First, BERT captures and extracts the key named entities of interest on TWS from the original medical records, that is, to find out the clinical and CT factors associated with diagnosis and treatment decisions. Secondly, the extracted key named entities are input into the knowledge representation system, and the knowledge representation system combines the semantic similarity matching of deep learning with the regularization technology of natural language processing to perform logical diagnosis on the entities. Regularization technology mainly acts on entity sentences that care about numerical values, extracts the numbers from them and compares them with the threshold range in TWS to obtain different weight scores. Semantic similarity matching mainly acts on text entities that care about CT manifestations, extracts features from text descriptions, uses Word2vec to convert them into vector representations, and then calculates the similarity of cosine distance between the vector representation of the entity and the vector representation of the pre-established standard template.

[0133] 6) The output results after the medical text information is interpreted.

[0134] ① Output 1: Emergency diagnostic conclusion, including the side, location and qualitative information of the injury. For example: "left parietal depressed fracture", "right temporal acute subdural hematoma", "left frontal temporal parietal chronic subdural hematoma", etc.

[0135] ② Output result 2: It is an emergency diagnosis and treatment decision, including emergency diagnostic measures and emergency therapeutic measures. Among them, diagnostic measures include whether to conduct further examinations, such as cranial MRI, cerebral blood vessels, coagulation function, and related examinations of other organs. Therapeutic measures include surgical treatment and non-surgical treatment measures. Surgical treatment measures mainly include emergency craniotomy and hematoma removal surgery, recommended craniectomy surgery, drilling and drainage, ICP probe insertion, open craniocerebral injury debridement and suture, etc.; non-surgical treatment measures include: whether life support is needed (cardiopulmonary resuscitation, intravenous access establishment, endotracheal intubation and airway management, mechanical ventilation, etc.), and the use of emergency drugs (including hemostatic drugs, anti-epileptic drugs, mannitol, etc.).

[0136] (3) Analysis of the accuracy and efficiency of automatic diagnosis and treatment results.

[0137] 1) T-SYSTEM accuracy analysis:

[0138] ① Analysis of the accuracy of automatic recognition of head CT images based on the Co-UNet model. By comparing the conclusive and descriptive results obtained by neurosurgeons, the accuracy, specificity and sensitivity of the imaging diagnosis results output by this system were determined.

[0139] ② Analysis of the accuracy of diagnosis and treatment decisions based on knowledge graph reasoning. By comparing with the emergency clinical diagnosis and treatment plan made by neurosurgeons, the accuracy, specificity and sensitivity of the preliminary clinical diagnosis and treatment plan output by this system are determined.

[0140] 2) Efficiency analysis of T-SYSTEM.

[0141] ① Analysis of the accuracy of T-SYSTEM. By comparing with neurosurgeons, the accuracy, specificity, and sensitivity of automatic recognition of cranial CT images based on the Co-UNet model and diagnosis and treatment decisions based on knowledge representation and deep learning were tested.

[0142] ② Analysis of T-SYSTEM efficiency: By comparing the number and accuracy of head CT images and EMRs recognized by neurosurgeons and T-SYSTEM in the same unit time, the actual efficiency was analyzed.

[0143] As attached Figure 6As shown in the figure, the BERT-IDCNN-BiLSTM-CRF model also demonstrated good performance in the process of identifying EMR text data of TBI patients. In a randomized double-blind comparison of more than 1,000 cases, compared with neurosurgeons, its sensitivity, specificity, accuracy and AUC were 96.00 (93.52-98.48)%, 96.27 (91.79-100)%, 95.00 (91.10-98.90)%, and 0.956 (0.926-0.986), respectively. The comprehensive recognition ability of this module still has room for further improvement. Currently, the data volume is being expanded for optimization and iteration, and the universal verification of EMR text data and imaging data from multiple different medical institutions has been completed.

[0144] The above are only embodiments of the present invention. The invention is not limited to the fields involved in this implementation case. The common sense such as the known specific structures and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or the priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to explain the content of the claims.

Claims

1. A clinical decision-making system for craniocerebral trauma based on text and image recognition, characterized in that: include: Image recognition module: used to generate text reports after identifying CT images; Establish the mapping relationship between CT signs and clinical conditions and diagnostic and treatment measures; Key named entity recognition module: Based on the BERT pre-trained language model, a BERT-IDCNN-BiLSTM-CRF model is built to extract key named entities from the global or local content of the text report; Knowledge-based semantic representation module: perform feature analysis on key named entities, establish a Word2vec model, vectorize the relevant medical text, and establish a medical text weight scoring mechanism and a medical text similarity measurement method based on knowledge to obtain medical text; Deep learning decision module: It is used to understand the differences between medical terms after extracting medical text features, intelligently identify and interpret medical texts of TBI patients, and make diagnosis and treatment decisions.

2. The craniocerebral trauma clinical decision system based on text and image recognition according to claim 1, characterized in that: The image recognition module is also used to pre-process the head CT images: save the head CT images of all normal people and patients in DICOM format, improve the Kalman filter algorithm to denoise the CT images, and use the LabelImg tool to annotate the head CT images; then divide the head CT images of TBI patients into skull injuries and brain injuries, use the Co-UNet model to segment the head CT images of TBI patients at the bone window and brain window of the skull CT images, design a deep convolutional network model of single connected domain boundary interpolation to extract image feature information of normal skull CT, realize the recognition of abnormal information based on the DenseNet network, improve the threshold segmentation algorithm to perform secondary screening on the recognition results, and determine the side, position, qualitatively and quantitatively identify different types of skull injuries and brain injuries in the two-dimensional skull CT images of TBI patients.

3. The craniocerebral trauma clinical decision system based on text and image recognition according to claim 2, characterized in that: The deep learning decision-making module is also used to: use the emergency electronic medical record (EMR) of TBI patients as the source of medical text information, and the collected content mainly includes medical history, physical examination and auxiliary examination results; among them, the head CT examination results are the text results output by the image recognition module; guided by the latest domestic and international craniocerebral trauma diagnosis and treatment guidelines, combined with clinical practice experience and the previous basic, clinical and applied research results of this research team, the medical history, physical examination, CT and other important auxiliary examinations of craniocerebral trauma patients are given different weight scores; at the same time, various diagnostic and treatment measures for craniocerebral trauma are refined, and after repeated clinical data verification, the mapping relationship between multi-dimensional, multi-source, heterogeneous data and corresponding diagnostic and treatment decisions is studied and constructed; and through intelligent recognition of TBI patients' head CT images and EMR, the diagnostic and treatment results in the form of Chinese text are obtained as output, including emergency diagnostic conclusions and emergency diagnostic and treatment decisions.

4. The craniocerebral trauma clinical decision system based on text and image recognition according to claim 1, characterized in that: When recognizing the head CT, recognize the normal head CT image: store and analyze the bone window and brain window images of the normal head CT image separately; at the bone window, segment the skull using the Co-UNet model, including the facial skull and brain skull, and extract the features of these bone landmark structures; at the brain window, segment the brain parenchyma area using the Co-UNet model, including the supratentorial and infratentorial brain parenchyma areas, extract the features of these landmark structures in the brain parenchyma, and calculate the average area of ​​the bilateral lateral ventricles at different head CT levels of normal people; Identification of head CT images of TBI patients: The injuries of TBI patients are divided into skull injuries and brain injuries, and the bone window and brain window positions of TBI patients' head CT images are stored and analyzed separately, and brain injuries and skull injuries are identified at different window positions.

5. The craniocerebral trauma clinical decision system based on text and image recognition according to claim 4, characterized in that: Identification of brain window positions in cranial CT images of TBI patients mainly includes: identification of brain injury, including lateralization, location, qualitative and quantitative identification; summary of lateralization, location, qualitative and quantitative identification of brain injury; The identification of skull injuries at the bone window level of the head CT images of TBI patients mainly includes: identification of the location and type of skull injury, including lateralization, positioning, qualitative and quantitative identification; and summary of lateralization, positioning, qualitative and quantitative identification of skull injuries.

6. The craniocerebral trauma clinical decision system based on text and image recognition according to claim 5, characterized in that: The text information output by the image processing module includes conclusive results and descriptive results; the conclusive results: output the results of brain injury and / or skull injury according to the diagnostic thinking of neurosurgery and neuroimaging, including lateral determination, positioning and qualitative results; the descriptive results: a more detailed description of the head CT image, including lateral determination, positioning, qualitative and quantitative results.

7. The craniocerebral trauma clinical decision system based on text and image recognition according to claim 6, characterized in that: The key named entity recognition module includes: BERT module: BERT uses a masked language model similar to the cloze test to complete pre-training. The purpose of the masked language model is to randomly mask some words in a sentence and replace the original words with a tag [MASK]. The further goal of the masked language model is to predict the masked words based on the context on the left and right of the tag. The BERT model calculation method is shown in formula (1): Among them, Q, K, V are input vector matrices, QK T Indicates the similarity matrix of the input vector calculated, d k is the dimension of the input vector matrix K. Downstream tasks of BERT, such as entailment and sentiment analysis, are processed using the first token of each sequence, which is called a classification-specific token [CLS]. In the BERT model, the final hidden state corresponding to the [CLS] token aggregates the feature representation of a sentence or a pair of sentences. IDCNN module: IDCNN uses dilated convolution. By adjusting the dilation width of the convolution kernel, it reduces data loss, expands the perception range, and achieves longer capture of text. The calculation formula for the receptive field of dilated convolution is: F i+1 ＝2 i+2 -1 2 There are three convolutional layers in the IDCNN module, with expansion widths of 1, 1, and 2 respectively. The module undergoes four iterations in total, and parameter sharing during the iteration process effectively prevents the model from overfitting. Each parameter increases linearly with the number of layers, while the receptive field value increases exponentially, allowing the receptive field to quickly cover the entire input sequence. BiLSTM module: BiLSTM uses the gating mechanism in the recurrent neural network, namely the input gate, output gate and forget gate; BiLSTM consists of a bidirectional LSTM to extract global context features from two directions respectively; CRF module: For an input word sequence X, there is a general label sequence Y representing the word sequence X. The probability model of the conditional random field defines a distribution P(Y|X) over all possible label sequences Y given the input word sequence X, which is in the following form:

8. The craniocerebral trauma clinical decision system based on text and image recognition according to claim 7, characterized in that: The knowledge-based semantic representation module is used to calculate the continuous distributed representation of medical text; a text vector is generated as output from the input of the medical text corpus through a neural network structure with a hidden layer; a vocabulary is first generated from the input text in the Word2vec model, and then the word vector is learned through back propagation and stochastic gradient methods; Also used for: A similarity measurement method is established to analyze and match corresponding treatment measures according to the weight score of medical manifestations; the medical text word vectors generated by Word2vec are used to generate sentence vector representations of key texts, and the sentence vector representation is obtained by the following formula. Where V S is the sentence vector representation, V i is the word vector representation of word i, w(i) is the weight, and m is the number of words; This method uses cosine similarity to match key medical texts with weighted scoring tables developed by medical experts, and analyzes corresponding treatment measures for medical manifestations; cosine similarity is between 0 and 1. When the cosine similarity is close to 1, the two vectors are said to be similar, and when the cosine similarity is close to 0, the two vectors are said to be dissimilar; for different key medical entities, we develop corresponding similarity thresholds through experiments; Among them, v1 and v2 are different sentence vector representations.

9. The craniocerebral trauma clinical decision system based on text and image recognition according to claim 1, characterized in that: The CT image analyzed by the image recognition module comes from a CT detector; it includes a frame, and the CT detector includes an X-ray transmitter arranged in a horizontal direction and an X-ray receiver corresponding to the X-ray transmitter; the X-ray transmitter and the X-ray receiver are fixed on the frame, and a support table is fixed on the frame between the X-ray transmitter and the X-ray receiver, and a headrest is fixed on the support table, and deflection blocks are hinged on both sides of the headrest, and a sliding rod is hinged on the bottom of the deflection block, and the support table is provided with a first slide and a second slide from top to bottom, and the first slide and the second slide are distributed in an eight-shaped shape, and an air duct is also provided in the support table, and the air duct is connected to the bottom of the first slide and the second slide respectively, and the sliding rods on the left and right sides of the headrest are slidably connected to the first slide and the second slide respectively, and the bottom of the deflection block and the top of the support table are connected to each other. A first spring is fixed on the bottom, and the first spring is sleeved on the outside of the sliding rod; a third slide in the vertical direction is opened on the support platform close to the X-ray emitter side, the bottom of the third slide is connected to the air duct, and a first lead block is slidably connected in the third slide, and a second spring is fixedly connected between the bottom of the first lead block and the inner wall of the third sliding bottom. The third slide at the bottom of the first lead block, the first slide and the second slide at the bottom of the two sliding rods, and the air duct form a closed space; when the sliding rod is not pressed down, the first lead block is not within the irradiation range of the X-ray emitter; when the deflection block above the sliding rod is pressurized, the first lead block is pushed out and appears within the irradiation range of the X-ray; the data processing module collects the pictures drawn by the X-ray detector, and discards the CT pictures with the lead block images and transmits them to the image recognition module.

10. The craniocerebral trauma clinical decision system based on text and image recognition according to claim 9, characterized in that: Position sensors are arranged in the first slide and the second slide, and the position sensors send out a reminder signal when the slide rod moves downward in the first slide / the second slide; an electric telescopic rod is fixed on the frame above the headrest, and the moving direction of the free end of the electric telescopic rod is downward, and a second lead block is fixed at the bottom of the electric telescopic rod; the data processing module receives the reminder signal, controls the electric telescopic rod to start, and extends the second lead block into the X-ray irradiation area, and the electric telescopic rod is reset after the reminder signal disappears.

Images