HICH intelligent rehabilitation large health system based on image and text recognition multi-round complementation

Abstract

The invention relates to the field of HICH auxiliary nursing systems, in particular to an HICH intelligent rehabilitation large health system based on image and text recognition multi-round complementation. According to the scheme, the system comprises an acquisition module used for acquiring electrophysiological data, iconography data and clinical text data of past treatment events of a patient, integrating the electrophysiological data, the iconography data and the clinical text data into training data and integrating the training data into first data; the graph construction module is used for recognizing the iconography data and the electrophysiological data as text data, performing character recognition and semantic extraction on the text data, determining an entity relationship through multi-party mapping of the extracted content, and establishing an HICH rehabilitation knowledge graph; the data processing module is also used for performing character recognition and semantic extraction on the clinical text data, and checking and complementing the HICH rehabilitation knowledge map by using the extracted content; and the decision module comprises a treatment weight unit and a Gaussian model unit. According to the scheme, the problem that the reliability of the HICH rehabilitation diagnosis and treatment result calculated through AI recognition is low is solved.

Description

Technical Field

[0001] The present invention relates to the field of HICH auxiliary nursing systems, and in particular to a HICH intelligent rehabilitation health system based on multiple rounds of complementary image and text recognition. Background Art

[0002] Hypertensive cerebral hemorrhage (HICH) is a common neurological disease with extremely high mortality and disability rates. Studies have shown that about 80% of HICH patients (referred to as "patients" in this article) will be left with various disabilities and complications, which requires most patients to undergo long-term rehabilitation treatment, which not only consumes a lot of manpower, but also consumes a staggering amount of economic resources.

[0003] However, most patients and their families cannot afford such huge expenses and can only choose to recuperate and rehabilitate at home. Since patients cannot get professional guidance and support at home, their condition often recurs or deteriorates, which seriously affects the quality of life of patients and may even directly lead to their death. With the current limited medical resources, there is a shortage of professionals required for rehabilitation treatment, and the training of professionals for rehabilitation treatment takes a long time, requires a large investment, and has a slow effect, resulting in high economic costs for rehabilitation treatment. At present, with the further development of AI technology, more and more information can be quickly processed through AI technology.

[0004] At present, some diseases can be assisted in diagnosis and treatment by AI technology. Usually, AI technology is trained to mark or identify some suspected lesions in the imaging diagnosis and treatment stage to improve the diagnosis and treatment efficiency of doctors. However, the use of deep learning (AI) technology to conduct more accurate and personalized analysis of patients' conditions, complete auxiliary diagnosis and treatment decisions, output relevant suggestions for medical rehabilitation programs, and ultimately provide HICH patients with more convenient and accurate intelligent rehabilitation treatment programs still faces certain problems.

[0005] The main difficulty is that HICH needs to collect imaging and electrophysiological data of patients during rehabilitation treatment, including CT, MRI and PET image data, as well as electroencephalogram (EEG), electromyography, evoked potential and brain oxygen saturation. These data are too large and complex in structure. Using independent AI models for training, the amount of training data is limited in the initial stage due to too many input parameters (variables). In addition, there are great individual differences between patients, resulting in low reliability of HICH rehabilitation diagnosis and treatment results given by AI recognition calculations. Summary of the invention

[0006] The purpose of this solution is to provide a HICH intelligent rehabilitation health system based on multiple rounds of complementary image and text recognition to solve the problem of low reliability of HICH rehabilitation diagnosis and treatment results given by AI recognition calculations.

[0007] In order to achieve the above objectives, this solution provides a HICH intelligent rehabilitation health system based on multiple rounds of complementary image and text recognition, including: An acquisition module, used to collect electrophysiological data, imaging data and clinical text data of patients' past treatment events, integrate them into training data and integrate them into first data; The graph construction module is used to identify imaging data and electrophysiological data as text data, to perform text recognition and semantic extraction on text data, to determine entities and entity relationships through multiple verifications of the extracted content, and to establish the HICH rehabilitation knowledge graph; it is also used to perform text recognition and semantic extraction on clinical text data, and to use the extracted content to check and supplement the HICH rehabilitation knowledge graph; The decision module includes a treatment weight unit and a Gaussian model unit. The treatment weight unit is a computational model constructed based on TransG-DNN. The treatment weight unit combines the HICH rehabilitation guidelines and consensus with the actual situation during rehabilitation treatment. The Gaussian model unit is a Gaussian mixture model established based on the HICH rehabilitation knowledge graph. The decision module is used to put the first data into the Gaussian mixture model to calculate the condition information, and then put the condition information into the treatment weight unit to calculate the treatment plan and treatment expectations.

[0008] The principle and effect of this scheme are: this scheme extracts entities from the patient's electrophysiological data and imaging data and verifies the entity relationship, and then combines the results of extraction and verification with each other, and performs multiple rounds of complementarity with each other. The treatment weight unit of this scheme is jointly constructed by specialists and AI researchers from neurosurgery, neurology, rehabilitation, radiology, nuclear medicine, electrophysiology center, mental health center, etc., based on the latest domestic and international HICH rehabilitation guidelines and consensus, combined with the actual situation in the rehabilitation treatment process, so that the treatment weight unit can output reliable rehabilitation diagnosis and treatment results through the condition. At the same time, the knowledge graph has the advantage of connecting structured and unstructured data. This scheme constructs a personalized HICH rehabilitation knowledge graph based on the actual situation of the patient, which solves the problem of complex structure of imaging data and electrophysiological data in this scheme, and also provides sophisticated and powerful basic technical support for the analysis of complex imaging data and electrophysiological data. This solution is based on the Gaussian mixture model established based on the HICH rehabilitation knowledge graph. Based on the constructed HICH rehabilitation knowledge graph, the Gaussian mixture embedding model (TransG) is used to vectorize each triple in the HICH rehabilitation knowledge graph to complete the preliminary classification. On this basis, the DNN training model is used. After obtaining the optimal training model, the optimal training model is applied to the training data to achieve the classification effect and stability of the HICH rehabilitation decision-making model, and finally complete the HICH rehabilitation automatic decision with high prediction accuracy, solving the problem of complex data and too many input parameters (variables).

[0009] In summary, this solution solves the problem of low reliability of HICH rehabilitation diagnosis and treatment results given by AI recognition and calculation.

[0010] Furthermore, the acquisition module also includes a recognition unit, which is a computing model based on U-net++ deep learning and neural network. When the acquisition unit acquires multimodal MRI and PET images in the imaging data, the recognition unit mainly automatically recognizes the cranial MRI and PET images of the patient during the rehabilitation process. The recognition unit is used to evaluate the integrity of the patient's brain parenchyma and white matter fiber bundles, cerebral blood perfusion, and brain function retention, and finally outputs the final result of automatic interpretation of MRI and PET images in the form of Chinese text; the recognition unit uses the method of CT image recognition to determine the side, position and quantitatively recognize the MRI and PET image data; the recognition unit uses U-net++ to add a dense block and a convolutional layer between the encoder and the decoder, and on the basis of the original U-net++, adds a redesigned jump path to combine the output of the previous convolutional layer of the same dense block with the output of the lower dense layer. The sampling outputs corresponding to the block are fused to make the semantic level of the encoded features closer to the semantic level of the feature map to be in the decoder; at the same time, dense skip connections are used to realize the jump path between the encoder and the decoder to ensure that all prior feature maps are accumulated and reach the current node through the dense convolution blocks on each jump path, generating feature maps with full resolution at multiple semantic levels, improving segmentation accuracy and improving gradient flow; in addition, the recognition unit is also used to increase deep supervision, adjust the complexity of the model by pruning the model, and adjust the balance between computational inference speed and performance; after the recognition unit performs localization, positioning and quantitative recognition on the MRI and PET image data, the recognition result obtained from one of the MRI and PET image data is used as a reference result, and the recognition obtained from other MRI and PET image data collected by the same examination is obtained as a complementary result, and multiple recognition results are verified and compared with the reference results in turn, and the reference results are complemented for multiple rounds, and the results of multiple rounds of complementation with the reference results are output as the final recognition result.

[0011] Furthermore, when the acquisition unit acquires electrophysiological data, the recognition unit smoothes the spike signal by half-wave processing, uses the principal component analysis algorithm to perform wavelet decomposition on the electrophysiological data to obtain signal components, then applies the principal component analysis algorithm to the signal components, and uses the independent component analysis algorithm to find the mixed signal matrix of artifact signals and EEG signals, thereby separating and processing various artifacts from the mixed composite signals; the recognition unit determines the threshold of the abnormal wave through the threshold initial screening, and if the amplitude, frequency, or area under the effect curve exceeds the determined abnormal wave threshold, it is determined to be an abnormal wave; the recognition unit uses expert feature analysis to perform initial screening based on the characteristics of various types of typical interference waveforms in the past, and removes eye movement artifacts, blink artifacts, and electrode artifacts. Artifacts, and at the same time, a bandpass filter is used to complete the filtering processing of the above-mentioned EEG signals to obtain EEG signals in useful frequency bands, and various interference signals outside the frequency band are filtered out, and a simple scaling method is used to normalize the electrophysiological data, perform segmented storage processing and label processing; then, the preprocessed electrophysiological sample data is input into a sparse autoencoder, and the features of the electrophysiological data are extracted based on the sparse autoencoder. The results output by the sparse autoencoder are input into a long short-term memory recurrent neural network, the extracted data features are analyzed, and the analysis results based on the long short-term memory recurrent neural network are input into a softmax classifier, thereby completing the classification of the electrophysiological data, and finally outputting the final result of automatic interpretation of the electrophysiological data in the form of Chinese text.

[0012] Furthermore, when the acquisition unit acquires the CT image, the recognition unit segments the hemorrhage foci, edematous brain tissue, normal brain tissue, each ventricle and important brain cisterns in the brain window position of the head CT through U-net++ and extracts their corresponding features, thereby completing the automatic recognition of the head CT image of the HICH patient, including lateral determination, positioning and quantitative recognition, and finally outputting the final result of the automatic interpretation of the CT image in the form of Chinese text.

[0013] The automatic discrimination of imaging data and electrophysiological data by the recognition unit can not only reduce the workload of medical staff, but also quickly provide a huge computing basis for the spectrum construction module and the decision module, thereby improving the output speed and output efficiency of the treatment plan of this scheme. At the same time, the recognition unit uses different discrimination methods for different examination data, which can more accurately output the final result of automatic judgment, thereby further improving the reliability of the HICH rehabilitation diagnosis and treatment results given by this scheme. The recognition unit outputs the final result of automatic judgment in the form of text, which is convenient for medical staff to check and change the final result, and can also unify the storage and reading methods of electrophysiological data, imaging data and clinical text data, which is more convenient for the spectrum construction module and the decision module to read and use; due to the complexity of electrophysiological data, imaging data and clinical text data, the spectrum construction module and the decision module have many parameters (variables) input during the construction or calculation process, and this method can also reduce the problem of errors in the operation of the spectrum construction module and the decision module.

[0014] Furthermore, it also includes an analysis unit, which is used to compare the final result output by the recognition unit with the result obtained by the doctor to determine the accuracy, specificity and sensitivity of the recognition unit; the analysis unit is also used to compare the number of final results output by the recognition unit with that of the doctor within the same unit time, and simultaneously calculate and compare the accuracy of the two.

[0015] By comparing the doctor's judgment with the final result output by the recognition unit through the analysis unit, more accurate training data can be provided to the recognition unit, thereby improving the accuracy, specificity and sensitivity of the recognition unit, and further improving the reliability of the HICH rehabilitation diagnosis and treatment results given by this scheme.

[0016] Furthermore, it also includes a rehabilitation pillow, a medical terminal and a patient terminal. The rehabilitation pillow is communicatively connected to the patient terminal, the medical terminal is communicatively connected to the acquisition module and the decision-making module, and the patient terminal is communicatively connected to the acquisition module and the decision-making module; the rehabilitation pillow is used to collect the patient's brain oxygen saturation data, and transmit the brain oxygen saturation data as rehabilitation acquisition data to the patient terminal, the patient terminal is used to receive and display the rehabilitation acquisition data, and then transmit the rehabilitation acquisition data as electrophysiological data to the acquisition module, and the patient terminal is also used to receive and display the data transmitted from the decision-making module; the medical terminal is used to read the acquisition data from the acquisition terminal, receive data and input clinical text data.

[0017] The rehabilitation pillow, medical terminal and patient terminal are added to allow the rehabilitation pillow to provide reliable monitoring for patients. Patients can understand their physical condition through the rehabilitation pillow, and can also receive reliable treatment plans output by the decision-making module through the patient terminal, which is convenient for collecting electrophysiological data of patients and can also provide patients with timely treatment plans that are more suitable for their current symptoms. The medical terminal can also timely understand the patient's symptoms and recovery status through the collection module, which is convenient for medical staff to input clinical text data that is more suitable for the patient's current symptoms through the medical terminal, further improving the reliability of the HICH rehabilitation diagnosis and treatment results given by this solution.

[0018] Furthermore, the rehabilitation pillow includes a shell and a plurality of near-infrared light probes, the plurality of near-infrared light probes are embedded in the surface of the shell, the surface of the shell is also embedded with a plurality of pressure sensors, a temperature regulating device is embedded in the shell, a power supply device and a processing device are also fixedly connected in the shell, the processing device is communicatively connected with the near-infrared light probe, the pressure sensor and the temperature regulating device, and the power supply device is electrically connected with the near-infrared light probe, the pressure sensor, the temperature regulating device and the processing device; the near-infrared light probe is used to collect the patient's brain oxygen saturation data and transmit it to the processing device; the pressure sensor is used to collect the pressure values ​​of various points on the rehabilitation pillow and transmit it to the processing device; the temperature regulating device is used to obtain the shell temperature and transmit it to the processing device; the processing device is used to receive the brain oxygen saturation data, the pressure value and the shell temperature, and analyze the carrier on the rehabilitation pillow according to the pressure value and the pressure point providing the pressure value, and then eliminate the interference data in the brain oxygen saturation data through the carrier, so as to obtain the brain oxygen saturation collection data, and finally send the brain oxygen saturation collection data as electrophysiological data to the patient terminal.

[0019] There are many factors in the patient's living environment that interfere with the collection of brain oxygen saturation data. Collecting brain oxygen saturation data while the patient is sleeping does not affect the patient's daily life and can collect more accurate brain oxygen saturation data. The pressure sensor can assist in determining the collection object of the rehabilitation pillow and exclude the data that is interfered with during the collection of brain oxygen saturation data.

[0020] Furthermore, multiple pressure sensors are evenly distributed on the shell, and the processing device stores the received brain oxygen saturation data, pressure value and shell temperature in the cache. When the processing device is used to eliminate interference data in the brain oxygen saturation data, a three-dimensional model of the carrier is established as a temporary model through the pressure value and the pressure point that provides the pressure value. A reference model is preset in the processing module, and the reference model is a three-dimensional model of the patient's head and neck that is entered in advance. The processing module is used to compare the temporary model and the reference model to obtain similarity. When the similarity is less than 90%, the brain oxygen saturation data collected at the same time as the pressure value is deleted from the cache as interference data, and the pressure value and the shell temperature collected at the same time as the pressure value are deleted at the same time. When the similarity is greater than or equal to 90%, the median of the brain oxygen saturation data collected at the same time as the pressure value is selected as the brain oxygen saturation collection data. When the similarity is between 90% and 97%, the brain oxygen saturation collection data is corrected and supplemented by the temporary model, and then the brain oxygen saturation collection data is sent to the patient terminal as electrophysiological data, and then other data outside the temporary model in the cache are cleared.

[0021] By building and comparing the three-dimensional model of the carrier and the patient's head and neck, the patient and non-patient objects can be accurately distinguished, thereby judging and eliminating the interference of the environment on the brain oxygen saturation data during collection, and achieving the effect of accurate collection. At the same time, the temporary model can also be used to obtain the patient's posture when collecting brain oxygen saturation data, so as to judge the interference caused by the patient himself to the collection process, and correct the brain oxygen saturation collection data according to the patient's current posture (own condition), further achieving the effect of accurate collection.

[0022] Furthermore, the temperature regulating device is also used to adjust the temperature of the housing. The processing device is preset with the highest brain oxygen saturation, the lowest brain oxygen saturation, the single waiting time and the maximum waiting time. After sending the brain oxygen saturation acquisition data as electrophysiological data to the patient terminal and before clearing other data in the cache, the processing device performs the following steps: S10: Calculate the difference between the cerebral oxygen saturation collection data and the highest cerebral oxygen saturation; S20a: If the difference value increases continuously after three consecutive waiting times, a first instruction is sent to the patient terminal; S20b: If the patient waits for multiple times in a row and the accumulated waiting time exceeds the maximum waiting time, a second instruction is sent to the patient terminal; S20c: If the brain oxygen saturation collected data is lower than the minimum brain oxygen saturation, a third instruction is sent to the patient terminal; S20d: If the difference is not less than 0 and the last difference is less than 0, the temperature regulating device is controlled to be turned on, and the housing temperature is controlled to be maintained at 34 degrees Celsius. After counting down the single waiting time, the process returns to S10; S20e: If the difference is less than 0 and the last difference is not less than 0, turn off the temperature adjustment device and return to S10; S20f: If the difference between the two most recent calculations is not less than 0, and the difference is greater than the previous difference, the housing temperature is controlled to be maintained at 32 degrees Celsius, and after the countdown of the single waiting time, the system returns to S10; S20g: If the difference between the two most recent calculations is not less than 0, and the difference is not greater than the previous difference, count down the single waiting time and return to S10; The patient terminal is preset with a first prompt, a second prompt, a third prompt and a closing time. After receiving the first instruction, the second instruction or the third prompt, the patient terminal immediately displays the first prompt, the second prompt or the third prompt, and broadcasts the first prompt, the second prompt or the third prompt in a loop. If the time of the loop broadcast exceeds the closing time, an alarm signal is sent to the acquisition terminal; if the patient terminal turns off the loop broadcast within the closing time, and selects a delay time after turning off the loop broadcast, the first instruction, the second instruction and the third instruction will not be received during the delay time; if the patient terminal turns off the loop broadcast within the closing time, and chooses to send an alarm signal after turning off the loop broadcast, an alarm signal is sent to the acquisition terminal.

[0023] Based on the accurate collection of brain oxygen saturation data, this solution makes multiple judgments on the patient's brain oxygen saturation data when abnormal data is detected (when the brain oxygen saturation is too high), and verifies the judgment results by lowering the contact temperature during the multiple judgments. This can not only reduce misjudgments, but also reduce the contact temperature of the patient's brain and neck. This method will not affect the collection of brain oxygen saturation data, but also reduce the patient's brain oxygen saturation to a certain extent, reduce the occurrence of sudden accidents in patients, or buy more rescue time for patients.

[0024] It also interacts with the patient through the display and broadcast of the patient terminal to further determine the patient's safety status. If the patient does not respond, the patient is likely to be isolated and unable to respond (such as: coma, speechless, unable to move, etc.). At this time, sending an alarm signal to the acquisition terminal can provide timely help to the patient and buy more rescue time for the patient. If the patient responds and selects the delay time, the patient is in a safe state, and the patient can clearly judge that the rehabilitation pillow has made a mistake. At this time, the rehabilitation pillow is in a damaged state. The patient terminal does not receive the first, second and third instructions, which can allow the patient to maintain a quiet sleep. If the patient responds and chooses to send an alarm signal, sending an alarm signal to the acquisition terminal can provide timely help to the patient.

[0025] Furthermore, the temperature adjustment device is also used to send the collected shell temperature and adjustment records to the patient terminal after being turned on. After the patient terminal receives the shell temperature and adjustment records, it matches the shell temperature and adjustment records with the brain oxygen saturation collection data according to the collection time, and converts them into clinical text data and sends them to the collection module; after the collection module receives the alarm signal, it uses the alarm signal sending time as the accident time node, and sends the electrophysiological data, clinical text data and alarm signal one hour before and after the accident time node to the medical terminal and the decision-making module; after receiving the alarm signal, the medical terminal emits an emergency prompt sound and displays the received electrophysiological data and clinical text data. After receiving the alarm signal, the decision module marks the received electrophysiological data and clinical text data as the highest priority, and uses the electrophysiological data and clinical text data to calculate an emergency treatment plan, and sends the emergency treatment plan to the patient terminal. After receiving the emergency treatment plan, the patient terminal plays the emergency treatment plan.

[0026] When the acquisition module receives an alarm signal, the patient is in need of help. This solution can provide medical staff with the most relevant data on the patient's condition, allowing medical staff and decision-making units to provide the most professional help to the patient, further buying time and opportunities for rescue. At the same time, this solution plays emergency treatment plans, and can also provide professional first aid guidance to the patient and the caregivers who may be around the patient, helping to speed up the patient's escape from danger. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is the logic block diagram of Example 1 of this scheme.

[0028] Figure 2 This is the logic block diagram of Example 2 of this scheme.

[0029] Figure 3 This is a schematic diagram of the structure of the rehabilitation pillow according to Example 2 of this scheme.

[0030] The following is a further detailed description through specific implementation methods: The figure marks in the drawings of the specification include: 1. light hole; 2. pressure sensor; 3. housing; 4. power supply device; 5. processing device; 6. near-infrared light probe; 7. temperature adjustment device. DETAILED DESCRIPTION

[0031] The following will clearly and completely describe the concept and technical effects of the present invention in combination with the embodiments, so as to fully understand the purpose, features and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention: Embodiment 1: HICH intelligent rehabilitation health system based on multiple rounds of complementary image and text recognition (such as Figure 1 shown), including: A collection module, used to collect electrophysiological data, imaging data and clinical text data of the patient's past treatment events and integrate them into training data, and also used to collect electrophysiological data, imaging data and clinical text data of the patient's current rehabilitation treatment and integrate them into first data; The graph construction module is used to identify imaging data and electrophysiological data as text data, to perform text recognition and semantic extraction on text data and clinical text data, to determine entities and entity relationships through multiple verifications of the extracted content, and to establish a HICH rehabilitation knowledge graph; The decision module includes a treatment weight unit and a Gaussian model unit. The treatment weight unit is a computational model constructed based on TransG-DNN. The treatment weight unit combines the HICH rehabilitation guidelines and consensus with the actual situation during rehabilitation treatment. The Gaussian model unit is a Gaussian mixture model established based on the HICH rehabilitation knowledge graph. The decision module is used to put the first data into the Gaussian mixture model to calculate the condition information, and then put the condition information into the treatment weight unit to calculate the treatment plan and treatment expectations.

[0032] The acquisition module also includes a recognition unit, which is a computing model based on U-net++ deep learning and neural networks. When the acquisition unit acquires multimodal MRI and PET images in the imaging data, the recognition unit mainly automatically recognizes the cranial MRI and PET images of the patient during the rehabilitation process. The recognition unit is used to evaluate the integrity of the patient's brain parenchyma and white matter fiber bundles, cerebral blood perfusion, and brain function retention, and finally outputs the final results of the automatic interpretation of MRI and PET images in the form of Chinese text; the recognition unit uses the methods used in CT image recognition to determine the side, position, and quantitatively recognize MRI and PET image data; the recognition unit uses U-net++ to add dense blocks and convolutional layers between the encoder and decoder, and on the basis of the original U-net++, adds a redesigned jump path to combine the output of the previous convolutional layer of the same dense block with the output of the lower dense layer. The sampling outputs corresponding to the block are fused to make the semantic level of the encoded features closer to the semantic level of the feature map to be in the decoder; at the same time, dense skip connections are used to realize the jump path between the encoder and the decoder to ensure that all prior feature maps are accumulated and reach the current node through the dense convolution blocks on each jump path, generating feature maps with full resolution at multiple semantic levels, improving segmentation accuracy and improving gradient flow; in addition, the recognition unit is also used to increase deep supervision, adjust the complexity of the model by pruning the model, and adjust the balance between computational inference speed and performance; after the recognition unit performs localization, positioning and quantitative recognition on the MRI and PET image data, the recognition result obtained from one of the MRI and PET image data is used as a reference result, and the recognition obtained from other MRI and PET image data collected by the same examination is obtained as a complementary result, and multiple recognition results are verified and compared with the reference results in turn, and the reference results are complemented for multiple rounds, and the results of multiple rounds of complementation with the reference results are output as the final recognition result.

[0033] When the acquisition unit collects electrophysiological data, the recognition unit smoothes the spike signal through half-wave processing, uses the principal component analysis algorithm to perform wavelet decomposition on the electrophysiological data to obtain signal components, then applies the principal component analysis algorithm to the signal components, and uses the independent component analysis algorithm to find the mixed signal matrix of artifact signals and EEG signals, so as to separate and process various artifacts from the mixed composite signals; the recognition unit determines the threshold of abnormal waves through threshold screening, and if the amplitude, frequency, or area under the effect curve exceeds the determined abnormal wave threshold, it is determined to be an abnormal wave; the recognition unit uses expert feature analysis to perform preliminary screening based on the characteristics of various typical interference waveforms in the past, remove eye movement artifacts, blink artifacts, electrode artifacts, and at the same time A bandpass filter is used to complete the filtering processing of the above-mentioned EEG signals to obtain EEG signals in a useful frequency band, and various interference signals outside the frequency band are filtered out. A simple scaling method is used to normalize the electrophysiological data, perform segmented storage processing and label processing; then, the preprocessed electrophysiological sample data is input into a sparse autoencoder, and the features of the electrophysiological data are extracted based on the sparse autoencoder. The results output by the sparse autoencoder are input into a long short-term memory recurrent neural network, the extracted data features are analyzed, and the analysis results based on the long short-term memory recurrent neural network are input into a softmax classifier, thereby completing the classification of the electrophysiological data, and finally outputting the final result of automatic interpretation of the electrophysiological data in the form of Chinese text.

[0034] When the acquisition unit acquires the CT image, the recognition unit uses U-net++ to segment the hemorrhage foci, edematous brain tissue, normal brain tissue, each ventricle and important brain cisterns in the brain window of the head CT and extract their corresponding features, completing the automatic recognition of the head CT images of HICH patients, including side determination, positioning and quantitative recognition, and finally outputs the final result of the automatic interpretation of the CT image in the form of Chinese text.

[0035] Among them, when the recognition unit recognizes and interprets the electrophysiological data and CT images, one of the acquired data (electrophysiological data and CT images) is used as the reference data, and the other data acquired at the same time is used as the supplementary data. The supplementary data is used to sequentially compare and verify the interpretation results of the reference data, and the interpretation results after multiple rounds of complementarity of the reference data are output as the final interpretation results.

[0036] It also includes an analysis unit, which is used to compare the final result output by the recognition unit with the result obtained by the doctor to determine the accuracy, specificity and sensitivity of the recognition unit; the analysis unit is also used to compare the number of final results output by the recognition unit with that of the doctor within the same unit time, and to calculate and compare the accuracy of the two.

[0037] Among them, the analysis unit takes the first judgment result output by the recognition unit as the reference result, and the judgment results output thereafter as the comparison results, and uses the comparison results to perform multiple rounds of complementation with the reference results, and then uses the clinical text data (data recorded by medical staff) as the comparison result to perform multiple rounds of complementation with the reference results again. The judgment results after multiple rounds of complementation with the reference results are used for map construction.

[0038] When it comes to specific implementation, The acquisition unit collects electrophysiological data, imaging data and clinical text data of the patient's past treatment events and integrates them into training data, and collects electrophysiological data, imaging data and clinical text data of the patient's current rehabilitation treatment and integrates them into first data.

[0039] After the acquisition unit acquires the CT, MRI, PET and other image data (i.e., imaging image data) of HICH patients, and stores them in the system of this solution in DICOM format, the recognition unit completes the recognition and annotation of the image data, and then performs denoising on the imaging image data and annotates the relevant images. Then, a deep neural network model is constructed to perform image segmentation and extract image feature information to realize automatic recognition of multimodal imaging data of HICH patients. Finally, the final result of automatic interpretation is stored in the training data and the first data in the form of Chinese text.

[0040] After the acquisition unit collects the electrophysiological data and brain oxygen metabolism data (including electroencephalogram (EEG), electromyogram, evoked potential, and brain oxygen saturation, etc.) of HICH patients during rehabilitation treatment, the recognition unit extracts the power spectral density of the electrophysiological data (including EEG, electromyogram, evoked potential, etc.), completes the classification of the brain oxygen saturation data set, converts the one-dimensional signal in the data into a two-dimensional time-frequency domain signal, removes some noise and artifacts, and then extracts and classifies the electrophysiological data and brain oxygen saturation data, completes feature recognition, and finally stores the final result of automatic interpretation in the form of Chinese text in the training data and the first data.

[0041] After the collection unit collects the clinical text data of HICH patients during the treatment and rehabilitation process (including the electronic medical records (EMR) during the acute phase of hospitalization after the onset of the disease and the rehabilitation-related text data recorded during the rehabilitation treatment), the clinical text data is classified according to the text information rules preset in the system.

[0042] The graph construction module first pre-processes the training data and the first data through natural language processing technologies such as word segmentation, syntactic analysis, information extraction, word sense disambiguation, and automatic summarization, solves the text classification and keyword extraction of the training data and the first data, as well as semantic disambiguation, and obtains the pre-processed Chinese medical information string about HICH rehabilitation. Based on various pre-established HICH rehabilitation-related dictionaries, the obtained Chinese medical information string is divided into strings with independent semantics and parsed in different dimensions. Next, the strings parsed in different dimensions are matched in their respective parsing dimensions through mechanical word segmentation.

[0043] The HICH rehabilitation knowledge graph is then defined as a directed label graph consisting of HICH rehabilitation related entities and their relationships. By constructing a triple structure of "entity-relationship-entity" related to HICH rehabilitation, various entities are connected through relationships in the form of subject, predicate, and object, and a network knowledge graph is formed. Among them, we use nodes to represent entities (i.e. "HICH rehabilitation related entities"), and edges to represent relationships (i.e. medical event relationships between TBI diagnosis and treatment related entities), and define them as follows.

[0044] Ⅰ. Set "HICH rehabilitation related entity" to "H", which is defined as various medical entities related to HICH rehabilitation that can be uniquely identified in HICH rehabilitation related text information records.

[0045] Ⅱ. Set “HICH rehabilitation medical event relationship” to “R”, which is defined as the medical event relationship generated or existing between different HICH rehabilitation related entities.

[0046] III. Let "HICH rehabilitation knowledge graph" be "K", which is defined as a directed label graph. Let T be the vertex set of the knowledge graph, representing the set of all HICH rehabilitation related entities; let R be the edge set of the knowledge graph, representing the medical event connections generated or existing between all HICH rehabilitation related entities.

[0047] The relevant entities in the HICH rehabilitation process are defined and classified, and the entities to be identified are divided into types such as "patients, basic information, symptoms, signs, auxiliary examination results, examination plans, prognosis, and rehabilitation plans". Then, the bidirectional long short term memory network model conditional random fields (Bidirectional long short term memory-Conditionalrandom fields, BiLSTM-CRF) is used to build a corpus, generate a model, and apply the model to complete entity extraction (or named entity recognition).

[0048] The HICH rehabilitation-related medical event relationships in the EMR of HICH patients are sorted out, and the entity relationships that need to be identified are divided into "belong_to, instance_of, attribute_of, diagnosis, check, and treatment". At the same time, since attribute extraction can essentially be regarded as relationship extraction, the extraction of "HICH rehabilitation-related medical event relationships" in this project will include entity relationship extraction and attribute extraction. By combining the corresponding kernel functions of the feature-based relationship extraction method and the tree kernel function-based method, a method combining features and tree kernel functions is formed to complete the extraction of HICH rehabilitation-related medical event relationships.

[0049] Then, through the steps of entity resolution, reference resolution, knowledge merging, etc., the TBI-related knowledge integration management is completed.

[0050] Finally, the knowledge processing and management related to TBI diagnosis and treatment is completed through ontology construction and extraction, knowledge reasoning and quality assessment.

[0051] The decision-making module is first jointly worked by specialists from neurosurgery, neurology, rehabilitation, radiology, nuclear medicine, electrophysiology center, mental health center, etc. and AI researchers. According to the latest domestic and international HICH rehabilitation guidelines and consensus, combined with the actual situation in the rehabilitation treatment process and AI algorithm requirements, a treatment weight unit for emergency diagnosis and emergency treatment decision-making of HICH patients is initially constructed. At the same time, a Gaussian model unit based on TransG-DNN is constructed to achieve automatic decision-making with high prediction accuracy and including HICH rehabilitation. Based on the constructed HICH rehabilitation knowledge graph, the Gaussian mixture embedding model (TransG) in the Gaussian model unit is used to vectorize each triple in the HICH rehabilitation knowledge graph to complete the preliminary classification. On this basis, the DNN training model (using training data) is used. After obtaining the optimal training model, the optimal training model is applied to the first data to achieve the classification effect and stability of the HICH rehabilitation decision-making model, and finally complete the automatic decision-making of HICH rehabilitation with high prediction accuracy.

[0052] Treatment expectations for HICH patients include predictive assessments (such as the ICH Modified and Grading Scale, the New ICH Scale, the ICH Functional Outcome Rating Scale, and the Functional Outcome Scale), as well as actual rehabilitation assessments (such as mRS, GOS, consciousness and mental state assessment, intelligence assessment, motor function assessment, balance ability assessment, walking function assessment, and daily life activities assessment). Treatment plans include setting rehabilitation goals (short-term and long-term rehabilitation goals), drug therapy, physical therapy, daily living activities training, exercise therapy, psychotherapy, neuromodulation therapy, brain-computer interface-based rehabilitation training, and traditional Chinese medicine treatment.

[0053] When the analysis unit determines the accuracy, specificity and sensitivity of the recognition unit: Used to complete the accuracy analysis of multimodal imaging image automatic recognition based on U-Net++. Compare the multimodal imaging automatic interpretation results output by this solution with the results obtained by doctors to determine the accuracy, specificity and sensitivity of the imaging results output in this solution.

[0054] It is used to complete the accuracy analysis of the automatic recognition of electrophysiological and brain oxygen metabolism data based on neural networks and deep learning. The automatic interpretation results of electrophysiological and brain oxygen metabolism data output by this program are compared with the results obtained by doctors to determine the accuracy, specificity and sensitivity of the electrophysiological and brain oxygen metabolism results output by this program.

[0055] Used to complete the accuracy analysis of rehabilitation decisions based on knowledge graph reasoning. Compare the treatment plan output by this program with the results obtained by the doctor to determine the accuracy, specificity and sensitivity of the rehabilitation decision output by this program.

[0056] It is used to complete the efficiency analysis of automatic recognition of multimodal imaging images based on U-Net++. In the same unit time, the number of multimodal imaging images processed by this solution is compared with that of doctors, and the accuracy of the two is calculated and compared.

[0057] It is used to complete the efficiency analysis of automatic recognition of electrophysiological and brain oxygen metabolism data based on neural networks and deep learning. In the same unit time, the number of electrophysiological and brain oxygen metabolism data processed by this solution is compared with that of doctors, and the accuracy of the two is calculated and compared.

[0058] It is used to analyze the efficiency of completing the treatment plan based on knowledge graph reasoning. In the same unit time, the number of completed cases processed by this plan is compared with that of doctors, and the accuracy of the two is calculated and compared.

[0059] Example 2 like Figure 2As shown, the HICH intelligent rehabilitation health system based on multiple rounds of complementary image and text recognition also includes a rehabilitation pillow, a medical terminal and a patient terminal. The rehabilitation pillow is communicatively connected to the patient terminal, the medical terminal is communicatively connected to the acquisition module and the decision-making module, and the patient terminal is communicatively connected to the acquisition module and the decision-making module; the rehabilitation pillow is used to collect the patient's brain oxygen saturation data, and transmit the brain oxygen saturation data as rehabilitation acquisition data to the patient terminal, the patient terminal is used to receive and display the rehabilitation acquisition data, and then transmit the rehabilitation acquisition data as electrophysiological data to the acquisition module, and the patient terminal is also used to receive and display the data transmitted from the decision-making module; the medical terminal is used to read the acquisition data from the acquisition terminal, receive the data and input clinical text data.

[0060] like Figure 3 As shown, the rehabilitation pillow includes a shell 3 and a plurality of near-infrared light probes 6. The surface of the shell 3 in contact with the patient is provided with a plurality of light holes 1 arranged in a matrix. The near-infrared light probes 6 are fixedly adhered to the bottom of the light holes 1, and the near-infrared light probes 6 are embedded in the surface of the shell 3. A plurality of pressure sensors 2 are also embedded in the surface of the shell 3. A temperature regulating device 7 is embedded in the shell 3. A power supply device 4 and a processing device 5 are also fixedly adhered in the shell 3. A flexible and fluffy material is also filled between the shell 3 and the power supply device 4 to improve the comfort of use, such as cotton, polyester fiber, foam, etc. The processing device 5 is communicatively connected with the near-infrared light probe 6, the pressure sensor 2 and the temperature regulating device 7, and the power supply device 4 is electrically connected with the near-infrared light probe 6, the pressure sensor 2, the temperature regulating device 7 and the processing device 5; the near-infrared light probe 6 is used to collect the patient's brain oxygen saturation data and transmit it to the processing device 5; the pressure sensor 2 is used to collect the pressure values ​​of various points on the rehabilitation pillow and transmit it to the processing device 5; the temperature regulating device 7 is used to obtain the temperature of the shell 3 and transmit it to the processing device 5; the processing device 5 is used to receive the brain oxygen saturation data, the pressure value and the temperature of the shell 3, and analyze the carrier on the rehabilitation pillow according to the pressure value and the pressure point that provides the pressure value, and then eliminate the interference data in the brain oxygen saturation data through the carrier, so as to obtain the brain oxygen saturation collection data, and finally send the brain oxygen saturation collection data as electrophysiological data to the patient terminal.

[0061] like Figure 3As shown, a plurality of pressure sensors 2 are evenly distributed on the shell 3, and the processing device 5 stores the received brain oxygen saturation data, pressure value and shell 3 temperature in the cache. When the processing device 5 is used to eliminate interference data in the brain oxygen saturation data, a three-dimensional model of the carrier is established as a temporary model through the pressure value and the pressure point providing the pressure value. A reference model is preset in the processing module, and the reference model is a three-dimensional model of the patient's head and neck entered in advance. The processing module is used to compare the temporary model with the reference model to obtain similarity. When the similarity is less than 90%, the brain oxygen saturation data collected at the same time as the pressure value is deleted from the cache as interference data, and the pressure value and the shell 3 temperature collected at the same time as the pressure value are deleted at the same time. When the similarity is greater than or equal to 90%, the median of the brain oxygen saturation data collected at the same time as the pressure value is selected as the brain oxygen saturation collection data. When the similarity is between 90% and 97%, the brain oxygen saturation collection data is corrected and supplemented by the temporary model, and then the brain oxygen saturation collection data is sent to the patient terminal as electrophysiological data, and then other data outside the temporary model in the cache are cleared.

[0062] Among them, when the similarity between the temporary model and the reference model is between 90% and 97%, the patient's body shape has changed slightly. At this time, using the temporary model to supplement the corrected reference model can make the reference model more in line with the patient's current real-time situation. In this way, the real data that best matches the patient's real-time situation can be used to supplement the historical data, so that the historical data can be updated. In the case of multiple updates (that is, using real-time data to complement historical data for multiple rounds), the recognition accuracy of the rehabilitation pillow for the patient (the person himself) is maintained or even improved, reducing the occurrence of misjudgment and improving the patient's user experience.

[0063] The temperature regulating device 7 is also used to adjust the temperature of the housing 3. The processing device 5 is preset with the maximum brain oxygen saturation, the minimum brain oxygen saturation, the single waiting time and the maximum waiting time. After sending the brain oxygen saturation acquisition data as electrophysiological data to the patient terminal and before clearing other data in the cache, the processing device 5 performs the following steps: S10: Calculate the difference between the cerebral oxygen saturation collection data and the highest cerebral oxygen saturation; S20a: If the difference value increases continuously after three consecutive waiting times, a first instruction is sent to the patient terminal; S20b: If the patient waits for multiple times in a row and the accumulated waiting time exceeds the maximum waiting time, a second instruction is sent to the patient terminal; S20c: If the brain oxygen saturation collected data is lower than the minimum brain oxygen saturation, a third instruction is sent to the patient terminal; S20d: If the difference is not less than 0 and the last difference is less than 0, the temperature regulating device 7 is controlled to be turned on, and the temperature of the housing 3 is controlled to be maintained at 34 degrees Celsius. After counting down the single waiting time, the process returns to S10; S20e: If the difference is less than 0 and the last difference is not less than 0, turn off the temperature adjustment device 7 and return to S10; S20f: If the difference between the two most recent calculations is not less than 0, and the difference is greater than the previous difference, the temperature of the housing 3 is controlled to be maintained at 32 degrees Celsius, and after counting down the single waiting time, the process returns to S10; S20g: If the difference between the two most recent calculations is not less than 0, and the difference is not greater than the previous difference, count down the single waiting time and return to S10; The patient terminal is preset with a first prompt, a second prompt, a third prompt and a closing time. After receiving the first instruction, the second instruction or the third prompt, the patient terminal immediately displays the first prompt, the second prompt or the third prompt, and broadcasts the first prompt, the second prompt or the third prompt in a loop. If the time of the loop broadcast exceeds the closing time, an alarm signal is sent to the acquisition terminal; if the patient terminal turns off the loop broadcast within the closing time, and selects a delay time after turning off the loop broadcast, the first instruction, the second instruction and the third instruction will not be received during the delay time; if the patient terminal turns off the loop broadcast within the closing time, and chooses to send an alarm signal after turning off the loop broadcast, an alarm signal is sent to the acquisition terminal.

[0064] When the health pillow collects abnormal data, the abnormal situation does not belong to the actual situation of the patient himself. At this time, the patient or the patient's family members need to verify and supplement the judgment results (made by the health pillow). When the patient or the patient's family members fail to provide verification and supplement operations in time, the judgment results (made by the health pillow) can be verified by the patient's other physiological data, that is, the judgment results (made by the health pillow) can be supplemented for multiple rounds using the patient's other physiological data, so as to increase the accuracy of the health pillow's judgment.

[0065] The temperature regulating device 7 is also used to send the collected temperature and adjustment records of the shell 3 to the patient terminal after being turned on. After the patient terminal receives the temperature and adjustment records of the shell 3, it corresponds the temperature and adjustment records of the shell 3 to the brain oxygen saturation collection data according to the collection time, and converts them into clinical text data and sends them to the collection module; after the collection module receives the alarm signal, it uses the alarm signal sending time as the accident time node, and sends the electrophysiological data, clinical text data and alarm signal one hour before and after the accident time node to the medical terminal and the decision-making module; after receiving the alarm signal, the medical terminal emits an emergency prompt sound and displays the received electrophysiological data and clinical text data. After receiving the alarm signal, the decision module marks the received electrophysiological data and clinical text data as the highest priority, and uses the electrophysiological data and clinical text data to calculate an emergency treatment plan, and sends the emergency treatment plan to the patient terminal. After receiving the emergency treatment plan, the patient terminal plays the emergency treatment plan.

[0066] During the specific implementation, patient A is regarded as the user of the rehabilitation pillow, caregiver A is regarded as the caregiver of patient A, the patient's mobile phone is regarded as the patient terminal, and the doctor's computer is regarded as the medical terminal.

[0067] Before using the rehabilitation pillow, patient A has entered the three-dimensional model of his own head and neck into the processing module as reference model A.

[0068] Patient A did not use the rehabilitation pillow to sleep during the day one day, and placed his mobile phone on the rehabilitation pillow. The pressure sensor 2 on the rehabilitation pillow that detected the pressure transmitted the detected pressure value to the processing device 5, and at the same time started the near-infrared light probe 6 and the temperature adjustment device 7 to collect brain oxygen saturation data and the temperature of the shell 3. The processing device 5 records the point of each pressure sensor 2. After receiving the pressure value, the processing device 5 finds the point corresponding to the pressure sensor 2 that transmits the pressure value as the pressure point, and corresponds the pressure point to the pressure value one by one, and then establishes a three-dimensional model as a temporary model (the three-dimensional model corresponding to the mobile phone) through all the pressure values ​​and the pressure points. The processing device 5 compares the reference model A with the temporary model to obtain the similarity 1, which is 2%, less than 90%. The rehabilitation pillow deletes the brain oxygen saturation data collected at the same time as the pressure value from the cache as interference data, and deletes the pressure value and the temperature of the shell 3 collected at the same time as the pressure value at the same time.

[0069] After a period of time, caregiver A takes a nap using the rehabilitation pillow. The processing device 5 of the rehabilitation pillow compares the reference model A with the temporary model established this time to obtain similarity 2. The similarity 2 is 80%, which is less than 90%. The rehabilitation pillow deletes the brain oxygen saturation data collected at the same time as the pressure value from the cache as interference data, and also deletes the pressure value and the shell 3 temperature collected at the same time as the pressure value.

[0070] That night, patient A used the rehabilitation pillow to fall asleep. The processing device 5 of the rehabilitation pillow compared the reference model A with the temporary model established this time to obtain similarity 3. The similarity 3 was 98%, which was greater than 90%. The rehabilitation pillow selected the median of the brain oxygen saturation data collected at the same time as the pressure value as the brain oxygen saturation collection data (8.09), and then sent the brain oxygen saturation collection data as electrophysiological data to the patient terminal. At the same time, the shell 3 temperature (36.2 degrees Celsius) collected at the same time as the pressure value was transmitted to the patient terminal, and the patient terminal sent the electrophysiological data and the shell 3 temperature to the collection module.

[0071] The processing device 5 presets the maximum brain oxygen saturation to 10.09, the minimum brain oxygen saturation to 7.88, the single waiting time to 5 minutes and the maximum waiting time to 30 minutes. The rehabilitation pillow calculates that the difference between the brain oxygen saturation collection data and the highest brain oxygen saturation is -2, which is less than 0. The rehabilitation pillow continues to collect the patient's brain oxygen saturation data.

[0072] After a period of time, the rehabilitation pillow calculates that the difference between the brain oxygen saturation collection data and the highest brain oxygen saturation is 0.5, which is greater than 0, and controls the temperature adjustment device 7 to turn on (turn it on again to ensure that the temperature adjustment device 7 is in the on state), and controls the temperature of the shell 3 to be maintained at 34 degrees Celsius. After the countdown single waiting time (5 minutes), the difference between the brain oxygen saturation collection data and the highest brain oxygen saturation is calculated again to be 0.2. The differences between the two most recent calculations are not less than 0, and the current difference (0.2) is not greater than the previous difference (0.5). After the countdown single waiting time (5 minutes), the difference between the brain oxygen saturation collection data and the highest brain oxygen saturation is calculated again to be -0.2. The current difference is less than 0 (-0.2) and the previous difference is not less than 0 (0.2). The control device turns off the temperature adjustment device 7, and the rehabilitation pillow continues to collect the patient's brain oxygen saturation data.

[0073] After a period of time, the rehabilitation pillow calculates that the difference between the brain oxygen saturation collection data and the highest brain oxygen saturation is 0.4. After 5 minutes, the difference is measured again to be 0.9. The control device controls the shell 3 to maintain the temperature at 32 degrees Celsius. After 5 minutes, the difference is measured again to be 1.3. The processing device 5 determines that it has been waiting for three consecutive times and the difference is increasing. The processing device 5 sends a first instruction to the patient terminal (mobile phone). After receiving the instruction, the mobile phone displays a first prompt and broadcasts the first prompt in a loop. After hearing the first prompt broadcast by the mobile phone, the attendant A immediately rushes to check the status of the patient A and finds that the patient A is in an abnormal state. After closing the prompt, the attendant A chooses to send an alarm signal. Then the mobile phone sends the alarm signal to the collection module. After receiving the alarm signal, the collection module uses the alarm signal sending time as the accident time node, and sends the electrophysiological data, clinical text data and alarm signal one hour before and after the accident time node to the medical terminal and the decision module; after receiving the alarm signal, the medical terminal (doctor's computer) emits an emergency prompt sound and displays the received electrophysiological data and clinical text data. After hearing the emergency prompt sound, the doctor checks the data displayed on the computer and inputs the clinical text data into the collection module. After receiving the alarm signal, the decision-making module marks the received electrophysiological data and clinical text data as the highest priority, and uses the electrophysiological data and clinical text data to calculate an emergency treatment plan, which is then sent to the patient's mobile phone. After receiving the emergency treatment plan, the patient's mobile phone plays the emergency treatment plan, and caregiver A performs first aid for the patient according to the emergency treatment plan broadcast by the mobile phone.

[0074] The above is only an embodiment of the present invention, and the known specific structure and characteristic common sense in the scheme are not described in detail here. It should be pointed out that for those skilled in the art, several deformations and improvements can be made without departing from the structure of the present invention, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation method records in the specification can be used to interpret the content of the claims.

Claims

1. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition is characterized by: include: An acquisition module, used to collect electrophysiological data, imaging data and clinical text data of patients' past treatment events, integrate them into training data and integrate them into first data; The graph construction module is used to identify imaging data and electrophysiological data as text data, to perform text recognition and semantic extraction on text data, to determine entities and entity relationships through multiple verifications of the extracted content, and to establish the HICH rehabilitation knowledge graph; it is also used to perform text recognition and semantic extraction on clinical text data, and to use the extracted content to check and supplement the HICH rehabilitation knowledge graph; The decision module includes a treatment weight unit and a Gaussian model unit. The treatment weight unit is a computational model constructed based on TransG-DNN. The treatment weight unit combines the HICH rehabilitation guidelines and consensus with the actual situation during rehabilitation treatment. The Gaussian model unit is a Gaussian mixture model established based on the HICH rehabilitation knowledge graph. The decision module is used to put the first data into the Gaussian mixture model to calculate the condition information, and then put the condition information into the treatment weight unit to calculate the treatment plan and treatment expectations.

2. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition according to claim 1 is characterized by: The acquisition module also includes a recognition unit, which is a computing model based on U-net++ deep learning and neural network. When the acquisition unit acquires multimodal MRI and PET images in the imaging data, the recognition unit mainly automatically recognizes the cranial MRI and PET images of the patient during the rehabilitation process. The recognition unit is used to evaluate the integrity of the patient's brain parenchyma and white matter fiber bundles, cerebral blood perfusion, and brain function retention, and finally outputs the final result of the automatic interpretation of the MRI and PET images in the form of Chinese text; the recognition unit uses the method of CT image recognition to determine the side, position and quantitatively recognize the MRI and PET image data; the recognition unit uses U-net++ to add a dense block and a convolutional layer between the encoder and the decoder, and adds a redesigned jump path on the basis of the original U-net++ to combine the output of the previous convolutional layer of the same dense block with the output of the lower dense layer. The sampled outputs corresponding to the blocks are fused to make the semantic level of the encoded features closer to the semantic level of the feature map to be in the decoder; at the same time, dense skip connections are used to implement the jump path between the encoder and decoder to ensure that all prior feature maps are accumulated and reach the current node through the dense convolution blocks on each jump path, generating feature maps with full resolution at multiple semantic levels, improving segmentation accuracy and improving gradient flow; in addition, the recognition unit is also used to increase deep supervision, adjust the complexity of the model by pruning the model, and adjust the balance between computational inference speed and performance; After the recognition unit performs lateral, positioning and quantitative recognition on the MRI and PET image data, the recognition result obtained from one of the MRI and PET image data is used as a reference result, and the recognition obtained from other MRI and PET image data collected by the same examination is obtained as a complementary result. The multiple recognition results are verified and compared with the reference results in sequence, and the reference results are complemented for multiple rounds, and the results of the multiple rounds of complementation with the reference results are output as the final recognition result.

3. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition according to claim 2 is characterized by: When the acquisition unit acquires electrophysiological data, the recognition unit smoothes the spike signal in a half-wave processing manner, uses the principal component analysis algorithm to perform wavelet decomposition on the electrophysiological data to obtain signal components, then applies the principal component analysis algorithm to the signal components, and uses the independent component analysis algorithm to find a mixed signal matrix of artifact signals and EEG signals, thereby separating and processing various artifacts from the mixed composite signals; the recognition unit determines the threshold of the abnormal wave through a threshold initial screening, and if the amplitude, frequency, or area under the effect curve feature exceeds the determined abnormal wave threshold, it is determined to be an abnormal wave; the recognition unit uses expert feature analysis to perform initial screening based on the features of various typical interference waveforms in the past, and removes eye movement artifacts, blink artifacts, and electrode artifacts. At the same time, a bandpass filter is used to complete the filtering processing of the above-mentioned EEG signals to obtain EEG signals in useful frequency bands, and various interference signals outside the frequency band are filtered out, and a simple scaling method is used to normalize the electrophysiological data, perform segmented storage processing and label processing; then, the preprocessed electrophysiological sample data is input into the sparse autoencoder, and the features of the electrophysiological data are extracted based on the sparse autoencoder. The results output by the sparse autoencoder are input into the long short-term memory recurrent neural network, the extracted data features are analyzed, and the analysis results based on the long short-term memory recurrent neural network are input into the softmax classifier, thereby completing the classification of the electrophysiological data, and finally outputting the final result of automatic interpretation of the electrophysiological data in the form of Chinese text.

4. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition according to claim 3 is characterized by: When the acquisition unit acquires the CT image, the recognition unit segments the hemorrhage focus, edema brain tissue, normal brain tissue, each ventricle and important brain cistern in the brain window of the head CT through U-net++ and extracts the corresponding features. Complete the automatic recognition of HICH patients' head CT images, including side determination, positioning and quantitative recognition, and finally output the final results of automatic interpretation of CT images in the form of Chinese text.

5. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition according to claim 4 is characterized by: It also includes an analysis unit, which is used to compare the final result output by the recognition unit with the result obtained by the doctor to determine the accuracy, specificity and sensitivity of the recognition unit; the analysis unit is also used to compare the number of final results output by the recognition unit with that of the doctor within the same unit time, and simultaneously calculate and compare the accuracy of the two.

6. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition according to claim 5 is characterized by: It also includes a rehabilitation pillow, a medical terminal and a patient terminal. The rehabilitation pillow is communicatively connected to the patient terminal, the medical terminal is communicatively connected to the acquisition module and the decision-making module, and the patient terminal is communicatively connected to the acquisition module and the decision-making module. The rehabilitation pillow is used to acquire the patient's brain oxygen saturation data, and transmit the brain oxygen saturation data as rehabilitation acquisition data to the patient terminal. The patient terminal is used to receive and display the rehabilitation acquisition data, and then transmit the rehabilitation acquisition data as electrophysiological data to the acquisition module. The patient terminal is also used to receive and display the data transmitted from the decision-making module. The medical terminal is used to read the acquisition data from the acquisition terminal, receive data and input clinical text data.

7. The HICH intelligent rehabilitation health system based on multi-round complementation of image and text recognition according to claim 6 is characterized by: The rehabilitation pillow comprises a shell and a plurality of near-infrared light probes, wherein the plurality of near-infrared light probes are embedded in the surface of the shell, wherein the surface of the shell is also embedded with a plurality of pressure sensors, wherein a temperature regulating device is embedded in the shell, wherein a power supply device and a processing device are also fixedly connected in the shell, wherein the processing device is communicatively connected with the near-infrared light probe, the pressure sensor and the temperature regulating device, and wherein the power supply device is electrically connected with the near-infrared light probe, the pressure sensor, the temperature regulating device and the processing device; wherein the near-infrared light probe is used to collect the patient's brain oxygen saturation data and transmit it to the processing device; wherein the pressure sensor is used to collect the pressure values ​​received by each point on the rehabilitation pillow and transmit it to the processing device; wherein the temperature regulating device is used to obtain the shell temperature and transmit it to the processing device; wherein the processing device is used to receive the brain oxygen saturation data, the pressure value and the shell temperature, and analyze the bearing on the rehabilitation pillow according to the pressure value and the pressure point providing the pressure value, and then eliminate the interference data in the brain oxygen saturation data through the bearing, thereby obtaining the brain oxygen saturation collection data, and finally sending the brain oxygen saturation collection data as electrophysiological data to the patient terminal.

8. The HICH intelligent rehabilitation health system based on multi-round complementation of image and text recognition according to claim 7 is characterized by: A plurality of pressure sensors are evenly distributed on the shell, and the processing device stores the received brain oxygen saturation data, pressure value and shell temperature in a cache. When the processing device is used to eliminate interference data in the brain oxygen saturation data, a three-dimensional model of the carrier is established as a temporary model through the pressure value and the pressure point providing the pressure value. A reference model is preset in the processing module, and the reference model is a three-dimensional model of the patient's head and neck entered in advance. The processing module is used to compare the temporary model with the reference model to obtain similarity. When the similarity is less than 90%, the brain oxygen saturation data collected at the same time as the pressure value is deleted from the cache as interference data, and the pressure value and the shell temperature collected at the same time as the pressure value are deleted at the same time. When the similarity is greater than or equal to 90%, the median of the brain oxygen saturation data collected at the same time as the pressure value is selected as the brain oxygen saturation collection data. When the similarity is between 90% and 97%, the brain oxygen saturation collection data is corrected and supplemented by the temporary model, and then the brain oxygen saturation collection data is sent to the patient terminal as electrophysiological data, and then other data outside the temporary model in the cache are cleared.

9. The HICH intelligent rehabilitation health system based on multi-round complementation of image and text recognition according to claim 8 is characterized by: The temperature regulating device is also used to adjust the temperature of the housing. The processing device is preset with the maximum brain oxygen saturation, the minimum brain oxygen saturation, the single waiting time and the maximum waiting time. After sending the brain oxygen saturation acquisition data as electrophysiological data to the patient terminal and before clearing other data in the cache, the processing device performs the following steps: S10: Calculate the difference between the cerebral oxygen saturation collection data and the highest cerebral oxygen saturation; S20a: If the difference value increases continuously after three consecutive waiting times, a first instruction is sent to the patient terminal; S20b: If the patient waits for multiple times in a row and the accumulated waiting time exceeds the maximum waiting time, a second instruction is sent to the patient terminal; S20c: If the brain oxygen saturation collected data is lower than the minimum brain oxygen saturation, a third instruction is sent to the patient terminal; S20d: If the difference is not less than 0 and the last difference is less than 0, the temperature regulating device is controlled to be turned on, and the housing temperature is controlled to be maintained at 34 degrees Celsius. After counting down the single waiting time, the process returns to S10; S20e: If the difference is less than 0 and the last difference is not less than 0, turn off the temperature adjustment device and return to S10; S20f: If the difference between the two most recent calculations is not less than 0, and the difference is greater than the previous difference, the housing temperature is controlled to be maintained at 32 degrees Celsius, and after the countdown of the single waiting time, the system returns to S10; S20g: If the difference between the two most recent calculations is not less than 0, and the difference is not greater than the previous difference, count down the single waiting time and return to S10; The patient terminal is preset with a first prompt, a second prompt, a third prompt and a closing time. After receiving the first instruction, the second instruction or the third prompt, the patient terminal immediately displays the first prompt, the second prompt or the third prompt, and broadcasts the first prompt, the second prompt or the third prompt in a loop. If the time of the loop broadcast exceeds the closing time, an alarm signal is sent to the acquisition terminal; if the patient terminal turns off the loop broadcast within the closing time, and selects a delay time after turning off the loop broadcast, the first instruction, the second instruction and the third instruction will not be received during the delay time; if the patient terminal turns off the loop broadcast within the closing time, and chooses to send an alarm signal after turning off the loop broadcast, an alarm signal is sent to the acquisition terminal.

10. The HICH intelligent rehabilitation health system based on multi-round complementation of image and text recognition according to claim 9 is characterized in that: The temperature adjustment device is also used to send the collected shell temperature and adjustment records to the patient terminal after being turned on. After the patient terminal receives the shell temperature and adjustment records, it matches the shell temperature and adjustment records with the brain oxygen saturation collection data according to the collection time, and converts them into clinical text data and sends them to the collection module; after the collection module receives the alarm signal, it uses the alarm signal sending time as the accident time node, and sends the electrophysiological data, clinical text data and alarm signal one hour before and after the accident time node to the medical terminal and the decision-making module; after receiving the alarm signal, the medical terminal emits an emergency prompt sound and displays the received electrophysiological data and clinical text data. After receiving the alarm signal, the decision module marks the received electrophysiological data and clinical text data as the highest priority, and uses the electrophysiological data and clinical text data to calculate an emergency treatment plan, and sends the emergency treatment plan to the patient terminal. After receiving the emergency treatment plan, the patient terminal plays the emergency treatment plan.

Images