HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition

Abstract

The present invention relates to the field of HICH auxiliary nursing systems, and in particular to a HICH intelligent rehabilitation health system based on multiple rounds of complementary image and text recognition. This solution includes: an acquisition module for collecting electrophysiological data, imaging data, and clinical text data of patients' past treatment events, integrating them into training data and integrating them into first data; a graph construction module for identifying imaging data and electrophysiological data as text data, performing text recognition and semantic extraction on text data, determining entities and entity relationships through multiple verifications of the extracted content, and establishing a HICH rehabilitation knowledge graph; it is also used to perform text recognition and semantic extraction on clinical text data, and use the extracted content to check and supplement the HICH rehabilitation knowledge graph; a decision module, including a treatment weight unit and a Gaussian model unit. This solution solves the problem of low reliability of HICH rehabilitation diagnosis and treatment results given by AI recognition calculations.

Description

Technical Field

[0001] The present invention relates to the field of HICH auxiliary nursing systems, and in particular to a HICH intelligent rehabilitation health system based on multiple rounds of complementary image and text recognition. Background Art

[0002] Hypertensive cerebral hemorrhage (HICH) is a common neurological disease with extremely high mortality and disability rates. Studies have shown that approximately 80% of HICH patients (referred to herein as "patients") will suffer from various disabilities and complications, requiring long-term rehabilitation treatment for most patients, which is not only labor-intensive but also extremely costly.

[0003] However, most patients and their families cannot afford such a huge expense and are forced to choose to recuperate and rehabilitate at home. Without professional guidance and support at home, patients often experience relapses or progressive deterioration, severely impacting their quality of life and potentially even leading to death. Given the current limitations of medical resources, the professionals needed for rehabilitation treatment are in short supply. Furthermore, the training of these professionals is time-consuming, expensive, and slow to produce results, resulting in high economic costs. However, with the continued development of AI technology, more and more information can be rapidly processed using AI.

[0004] Currently, some diseases can be diagnosed and treated with AI technology. Typically, AI technology is trained to label or identify suspected lesions during imaging, improving doctors' diagnostic and treatment efficiency. However, using deep learning (AI) technology to provide more accurate and personalized analysis of patients' conditions, assist in diagnosis and treatment decisions, and provide recommendations for medical rehabilitation programs, ultimately providing more convenient and accurate intelligent rehabilitation treatment plans for HICH patients, still faces certain challenges.

[0005] The main difficulty lies in the fact that HICH requires the collection of imaging and electrophysiological data from patients during rehabilitation treatment, including CT, MRI, and PET image data, as well as electroencephalogram (EEG), electromyography, evoked potentials, and brain oxygen saturation. These data are too large and complex in structure. Using an independent AI model for training, the amount of training data is relatively limited in the initial stage due to the excessive number of input parameters (variables). In addition, there are great individual differences between patients, resulting in low reliability of HICH rehabilitation diagnosis and treatment results given by AI recognition and calculation. Summary of the Invention

[0006] The purpose of this solution is to provide a HICH intelligent rehabilitation health system based on multiple rounds of complementary image and text recognition to solve the problem of low reliability of HICH rehabilitation diagnosis and treatment results given by AI recognition calculations.

[0007] To achieve the above objectives, this solution provides a HICH intelligent rehabilitation health system based on multiple rounds of complementary image and text recognition, including:

[0008] An acquisition module, configured to collect electrophysiological data, imaging data, and clinical text data of a patient's past treatment events, and integrate them into training data and first data;

[0009] The graph construction module is used to identify imaging data and electrophysiological data as text data, perform text recognition and semantic extraction on the text data, determine entities and entity relationships through multi-party verification of the extracted content, and establish the HICH rehabilitation knowledge graph. It is also used to perform text recognition and semantic extraction on clinical text data, and use the extracted content to check and supplement the HICH rehabilitation knowledge graph;

[0010] The decision module includes a treatment weight unit and a Gaussian model unit. The treatment weight unit is a computational model constructed based on TransG-DNN, which combines the HICH rehabilitation guidelines and consensus with the actual situation during rehabilitation treatment. The Gaussian model unit is a Gaussian mixture model established based on the HICH rehabilitation knowledge graph. The decision module is used to place the first data into the Gaussian mixture model to calculate the condition information, and then place the condition information into the treatment weight unit to calculate the treatment plan and treatment expectations.

[0011] The principle and effect of this solution are as follows: This solution extracts entities and verifies entity relationships from the patient's electrophysiological data and imaging data, and then combines the results of extraction and verification with each other, and performs multiple rounds of complementarity with each other. The treatment weight unit of this solution is jointly constructed by specialists and AI researchers from neurosurgery, neurology, rehabilitation, radiology, nuclear medicine, electrophysiology center, mental health center, etc., based on the latest domestic and international HICH rehabilitation guidelines and consensus, combined with the actual situation during rehabilitation treatment, so that the treatment weight unit can output reliable rehabilitation diagnosis and treatment results through the condition. At the same time, the knowledge graph has the advantage of connecting structured and unstructured data. This solution constructs a personalized HICH rehabilitation knowledge graph based on the patient's actual situation, which solves the problem of complex structure of imaging data and electrophysiological data in this solution, and also provides sophisticated and powerful basic technical support for the analysis of complex imaging data and electrophysiological data. This solution is based on the Gaussian mixture model established with the HICH rehabilitation knowledge graph. Taking the constructed HICH rehabilitation knowledge graph as the basis, the Gaussian mixture embedding model (TransG) is used to vectorize each triple in the HICH rehabilitation knowledge graph to complete preliminary classification. On this basis, the DNN training model is used. After obtaining the optimal training model, the optimal training model is applied to the training data to achieve the classification effect and stability of the HICH rehabilitation decision-making model, and finally complete the HICH rehabilitation automatic decision with high prediction accuracy, solving the problems of complex data and too many input parameters (variables).

[0012] In summary, this solution solves the problem of low reliability of HICH rehabilitation diagnosis and treatment results given by AI recognition and calculation.

[0013] Furthermore, the acquisition module also includes a recognition unit, which is a computing model based on U-net++ deep learning and neural network. When the acquisition module acquires multimodal MRI and PET images in the imaging data, the recognition unit mainly automatically recognizes the cranial MRI and PET images during the patient's rehabilitation process. The recognition unit is used to evaluate the integrity of the patient's brain parenchyma and white matter fiber bundles, cerebral blood perfusion, and brain function retention, and finally outputs the final result of the automatic interpretation of MRI and PET images in the form of Chinese text; the recognition unit uses the method of CT image recognition to determine the side, position and quantitatively recognize the MRI and PET image data; the recognition unit uses U-net++ to add a dense block and a convolutional layer between the encoder and the decoder, and adds a redesigned jump path on the basis of the original U-net++ to combine the output of the previous convolutional layer of the same dense block with the output of the lower dense layer. The sampling outputs corresponding to the block are fused to make the semantic level of the encoded features closer to the semantic level of the feature map to be in the decoder; at the same time, dense skip connections are used to realize the jump path between the encoder and decoder to ensure that all prior feature maps are accumulated and reach the current node through the dense convolution block on each jump path, generating feature maps with full resolution at multiple semantic levels, improving segmentation accuracy and improving gradient flow; in addition, the recognition unit is also used to increase deep supervision, adjust the complexity of the model by pruning the model, and adjust the balance between computational inference speed and performance; after the recognition unit performs side-location, positioning and quantitative recognition on the MRI and PET image data, the recognition result obtained from one of the MRI and PET image data is used as the reference result, and the recognition obtained from other MRI and PET image data collected in the same examination is obtained as the complementary result. The multiple recognition results are verified and compared with the reference results in sequence, and the reference results are complemented for multiple rounds, and the results of the multiple rounds of complementation with the reference results are output as the final recognition result.

[0014] Furthermore, when the acquisition module collects electrophysiological data, the recognition unit smoothes the spike signal through half-wave processing, uses the principal component analysis algorithm to perform wavelet decomposition on the electrophysiological data to obtain signal components, then applies the principal component analysis algorithm to the signal components, and uses the independent component analysis algorithm to find the mixed signal matrix of the artifact signal and the EEG signal, thereby separating and processing various artifacts from the mixed composite signal; the recognition unit determines the threshold of the abnormal wave through the threshold screening, and if the amplitude, frequency, or area under the effect curve feature exceeds the determined abnormal wave threshold, it is determined to be an abnormal wave; the recognition unit uses expert feature analysis to perform preliminary screening based on the characteristics of various typical interference waveforms in the past, and removes eye movement artifacts, blink artifacts, electrode artifacts, etc. Artifacts, and at the same time, a bandpass filter is used to complete the filtering processing of the above-mentioned EEG signals to obtain EEG signals in the useful frequency band, and various interference signals outside the frequency band are filtered out, and a simple scaling method is used to normalize the electrophysiological data, perform segmented storage processing and labeling processing; then, the preprocessed electrophysiological sample data is input into the sparse autoencoder, and the features of the electrophysiological data are extracted based on the sparse autoencoder. The results output by the sparse autoencoder are input into the long short-term memory recurrent neural network, the extracted data features are analyzed, and the analysis results based on the long short-term memory recurrent neural network are input into the softmax classifier, thereby completing the classification of the electrophysiological data, and finally outputting the final result of automatic interpretation of the electrophysiological data in the form of Chinese text.

[0015] Furthermore, when the acquisition module acquires a CT image, the recognition unit uses U-net++ to segment the hemorrhage foci, edematous brain tissue, normal brain tissue, each ventricle, and important cisterns in the brain window of the head CT scan and extract their corresponding features, completing the automatic recognition of the head CT images of HICH patients, including lateralization, positioning, and quantitative recognition, and ultimately outputting the final result of the automatic interpretation of the CT image in the form of Chinese text.

[0016] The recognition unit's automatic interpretation of imaging and electrophysiological data not only reduces the workload of medical staff but also quickly provides a massive computational foundation for the atlas construction and decision-making modules, thereby improving the speed and efficiency of the treatment plan output. Furthermore, the recognition unit utilizes different interpretation methods for different examination data, enabling more accurate output of the final automatically interpreted results, further enhancing the reliability of the HICH rehabilitation diagnosis and treatment results provided by this protocol. The recognition unit outputs the final automatically interpreted results in text format, facilitating verification and modification by medical staff. It also unifies the storage and access methods for electrophysiological, imaging, and clinical text data, making them more accessible to the atlas construction and decision-making modules. Due to the complexity of electrophysiological, imaging, and clinical text data, the atlas construction and decision-making modules require numerous input parameters (variables) during the construction or calculation process. This approach also reduces operational errors in the atlas construction and decision-making modules.

[0017] Furthermore, it also includes an analysis unit, which is used to compare the final result output by the recognition unit with the result obtained by the doctor to determine the accuracy, specificity and sensitivity of the recognition unit; the analysis unit is also used to compare the number of final results output by the recognition unit with that of the doctor within the same unit time, and simultaneously calculate and compare the accuracy of the two.

[0018] By comparing the doctor's judgment with the final result output by the recognition unit through the analysis unit, more accurate training data can be provided to the recognition unit, thereby improving the accuracy, specificity and sensitivity of the recognition unit, and further improving the reliability of the HICH rehabilitation diagnosis and treatment results given by this scheme.

[0019] Furthermore, it also includes a rehabilitation pillow, a medical terminal and a patient terminal. The rehabilitation pillow is communicatively connected to the patient terminal, the medical terminal is communicatively connected to the acquisition module and the decision-making module, and the patient terminal is communicatively connected to the acquisition module and the decision-making module; the rehabilitation pillow is used to collect the patient's brain oxygen saturation data and transmit the brain oxygen saturation data as rehabilitation acquisition data to the patient terminal, the patient terminal is used to receive and display the rehabilitation acquisition data, and then transmit the rehabilitation acquisition data as electrophysiological data to the acquisition module, and the patient terminal is also used to receive and display the data transmitted from the decision-making module; the medical terminal is used to read the acquisition data from the acquisition terminal, receive data and input clinical text data.

[0020] By integrating a rehabilitation pillow, a medical terminal, and a patient terminal, the rehabilitation pillow provides reliable monitoring for patients. Patients can use the pillow to understand their physical condition and receive reliable treatment plans from the decision-making module through the patient terminal. This facilitates the collection of electrophysiological data while providing timely treatment plans that are more tailored to the patient's current condition. The medical terminal can also use the collection module to keep abreast of the patient's symptoms and recovery status, allowing medical staff to input clinical text data that is more relevant to the patient's current condition through the terminal, further improving the reliability of the HICH rehabilitation diagnosis and treatment results provided by this solution.

[0021] Furthermore, the rehabilitation pillow includes a shell and multiple near-infrared light probes, the multiple near-infrared light probes are embedded in the surface of the shell, the surface of the shell is also embedded with multiple pressure sensors, the shell is embedded with a temperature adjustment device, the shell is also fixedly connected with a power supply device and a processing device, the processing device is communicatively connected to the near-infrared light probe, the pressure sensor and the temperature adjustment device, and the power supply device is electrically connected to the near-infrared light probe, the pressure sensor, the temperature adjustment device and the processing device; the near-infrared light probe is used to collect the patient's brain oxygen saturation data and transmit it to the processing device; the pressure sensor is used to collect the pressure values ​​of various points on the rehabilitation pillow and transmit them to the processing device; the temperature adjustment device is used to obtain the shell temperature and transmit it to the processing device; the processing device is used to receive the brain oxygen saturation data, pressure value and shell temperature, and analyze the load on the rehabilitation pillow according to the pressure value and the pressure point providing the pressure value, and then eliminate the interference data in the brain oxygen saturation data through the load, thereby obtaining the brain oxygen saturation collection data, and finally sending the brain oxygen saturation collection data as electrophysiological data to the patient terminal.

[0022] Many factors in a patient's living environment can interfere with the collection of brain oxygen saturation data. Collecting brain oxygen saturation data while the patient is sleeping does not affect their daily lives and can also provide more accurate data. The pressure sensor can help determine the data being collected by the rehabilitation pillow, eliminating any interference during the collection process.

[0023] Furthermore, multiple pressure sensors are evenly distributed on the shell, and the processing device stores the received brain oxygen saturation data, pressure values, and shell temperature in a cache. When the processing device is used to eliminate interference data in the brain oxygen saturation data, a three-dimensional model of the carrier is established as a temporary model based on the pressure values ​​and the pressure points providing the pressure values. A reference model is preset in the processing module, and the reference model is a three-dimensional model of the patient's head and neck that has been entered in advance. The processing module is used to compare the temporary model and the reference model to obtain similarity. When the similarity is less than 90%, the brain oxygen saturation data collected simultaneously with the pressure value is deleted from the cache as interference data, and the pressure value and the shell temperature collected simultaneously with the pressure value are also deleted. When the similarity is greater than or equal to 90%, the median of the brain oxygen saturation data collected at the same time as the pressure value is selected as the brain oxygen saturation collected data. When the similarity is between 90% and 97%, the brain oxygen saturation collected data is corrected and supplemented using the temporary model, and then the brain oxygen saturation collected data is sent to the patient terminal as electrophysiological data, and then other data in the cache outside the temporary model is cleared.

[0024] By building and comparing a 3D model of the carrier and the patient's head and neck, we can accurately distinguish between patients and non-patient objects, thereby determining and eliminating environmental interference with brain oxygen saturation data during collection, achieving precise collection results. Furthermore, the temporary model can be used to determine the patient's posture during brain oxygen saturation data collection, thereby determining any interference caused by the patient themselves during the collection process. The brain oxygen saturation data can then be corrected based on the patient's current posture (or condition), further achieving precise collection results.

[0025] Furthermore, the temperature regulating device is also used to regulate the temperature of the housing. The processing device is preset with the maximum brain oxygen saturation, the minimum brain oxygen saturation, the single waiting time, and the maximum waiting time. After the processing device sends the brain oxygen saturation collected data as electrophysiological data to the patient terminal and before clearing other data in the cache, it performs the following steps:

[0026] S10: Calculate the difference between the collected cerebral oxygen saturation data and the maximum cerebral oxygen saturation;

[0027] S20a: If the difference value increases continuously after three consecutive waiting times, a first instruction is sent to the patient terminal;

[0028] S20b: If the patient waits for multiple times in a row and the accumulated waiting time exceeds the maximum waiting time, a second instruction is sent to the patient terminal;

[0029] S20c: If the brain oxygen saturation collected data is lower than the minimum brain oxygen saturation, a third instruction is sent to the patient terminal;

[0030] S20d: If the difference is not less than 0 and the last difference is less than 0, the temperature regulating device is turned on and the housing temperature is controlled to be maintained at 34 degrees Celsius. After counting down the single waiting time, the process returns to S10.

[0031] S20e: If the difference is less than 0 and the previous difference is not less than 0, turn off the temperature adjustment device and return to S10;

[0032] S20f: If the difference between the two most recent calculations is not less than 0 and is greater than the previous difference, the housing temperature is controlled to be maintained at 32 degrees Celsius, and after counting down the single waiting time, the process returns to S10;

[0033] S20g: If the difference between the two most recently calculated values ​​is not less than 0 and is not greater than the previous difference, count down the single waiting time and return to S10;

[0034] The patient terminal is preset with a first prompt, a second prompt, a third prompt and a closing time. After receiving the first instruction, the second instruction or the third prompt, the patient terminal immediately displays the first prompt, the second prompt or the third prompt, and broadcasts the first prompt, the second prompt or the third prompt in a loop. If the time of the loop broadcast exceeds the closing time, an alarm signal is sent to the acquisition terminal; if the patient terminal turns off the loop broadcast within the closing time, and selects a delay time after turning off the loop broadcast, the first instruction, the second instruction and the third instruction will not be received during the delay time; if the patient terminal turns off the loop broadcast within the closing time, and chooses to send an alarm signal after turning off the loop broadcast, an alarm signal is sent to the acquisition terminal.

[0035] Based on the precise collection of brain oxygen saturation data, this solution makes multiple judgments on the patient's brain oxygen saturation data when abnormal data is detected (when the brain oxygen saturation is too high), and verifies the judgment results by lowering the contact temperature during the multiple judgments. This can not only reduce misjudgments, but also reduce the contact temperature of the patient's brain and neck. This method will not only not affect the collection of brain oxygen saturation data, but also reduce the patient's brain oxygen saturation to a certain extent, reduce the risk of sudden accidents in patients, or buy more rescue time for patients.

[0036] The system also interacts with the patient through displays and announcements on the patient terminal to further determine their safety. If the patient does not respond, they are likely isolated and unable to respond (such as being unconscious, unable to speak, or unable to move). In this case, an alarm signal is sent to the acquisition terminal, providing timely assistance and buying more time for rescue. If the patient responds and selects a delay time, they are safe and can clearly determine that the rehabilitation pillow has made an error. In this case, the rehabilitation pillow is damaged. The patient terminal does not receive the first, second, and third commands, allowing the patient to maintain a peaceful sleep. If the patient responds and selects to send an alarm signal, an alarm signal is sent to the acquisition terminal, providing timely assistance.

[0037] Furthermore, the temperature adjustment device is also used to send the collected shell temperature and adjustment records to the patient terminal after being turned on. After the patient terminal receives the shell temperature and adjustment records, it matches the shell temperature and adjustment records with the brain oxygen saturation collection data according to the collection time, and converts them into clinical text data and sends them to the collection module; after the collection module receives the alarm signal, it uses the alarm signal sending time as the accident time node, and sends the electrophysiological data, clinical text data and alarm signal one hour before and after the accident time node to the medical terminal and the decision-making module; after receiving the alarm signal, the medical terminal emits an emergency prompt sound and displays the received electrophysiological data and clinical text data. After receiving the alarm signal, the decision module marks the received electrophysiological data and clinical text data as the highest priority, and uses the electrophysiological data and clinical text data to calculate an emergency treatment plan, and sends the emergency treatment plan to the patient terminal. After receiving the emergency treatment plan, the patient terminal plays the emergency treatment plan.

[0038] When the acquisition module receives an alarm signal, indicating that the patient is in need of assistance, this solution can provide medical staff with the most relevant data on the patient's condition, allowing them and the decision-making unit to provide the most professional assistance, further buying time and opportunities for rescue. Furthermore, this solution plays emergency treatment plans, providing professional first aid guidance to the patient and any caregivers nearby, helping to expedite the patient's escape from danger. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is the logic block diagram of Example 1 of this solution.

[0040] Figure 2 This is the logic block diagram of Example 2 of this solution.

[0041] Figure 3 This is a schematic structural diagram of the rehabilitation pillow according to Example 2 of this scheme.

[0042] The following is a further detailed description through specific implementation methods:

[0043] The reference numerals in the drawings of the specification include: 1. light hole; 2. pressure sensor; 3. housing; 4. power supply device; 5. processing device; 6. near-infrared light probe; 7. temperature adjustment device. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the concept and technical effects of the present invention in conjunction with the embodiments to fully understand the purpose, features and effects of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention:

[0045] Example 1:

[0046] HICH intelligent rehabilitation health system based on multiple rounds of complementary image and text recognition (such as Figure 1 shown), including:

[0047] An acquisition module is used to collect electrophysiological data, imaging data, and clinical text data of the patient's past treatment events and integrate them into training data, and is also used to collect electrophysiological data, imaging data, and clinical text data of the patient's current rehabilitation treatment and integrate them into first data;

[0048] The graph construction module is used to identify imaging data and electrophysiological data as text data, perform text recognition and semantic extraction on text data and clinical text data, determine entities and entity relationships through multi-party verification of the extracted content, and establish a HICH rehabilitation knowledge graph;

[0049] The decision module includes a treatment weight unit and a Gaussian model unit. The treatment weight unit is a computational model constructed based on TransG-DNN, which combines the HICH rehabilitation guidelines and consensus with the actual situation during rehabilitation treatment. The Gaussian model unit is a Gaussian mixture model established based on the HICH rehabilitation knowledge graph. The decision module is used to place the first data into the Gaussian mixture model to calculate the condition information, and then place the condition information into the treatment weight unit to calculate the treatment plan and treatment expectations.

[0050] The acquisition module also includes a recognition unit, which is a computing model based on U-net++ deep learning and neural network. When the acquisition module acquires multimodal MRI and PET images in the imaging data, the recognition unit mainly automatically recognizes the cranial MRI and PET images during the patient's rehabilitation process. The recognition unit is used to evaluate the integrity of the patient's brain parenchyma and white matter fiber bundles, cerebral blood perfusion, and brain function retention, and finally outputs the final results of the automatic interpretation of MRI and PET images in the form of Chinese text; the recognition unit uses the method of CT image recognition to determine the side, position and quantitatively recognize MRI and PET image data; the recognition unit uses U-net++ to add a dense block and a convolutional layer between the encoder and the decoder, and on the basis of the original U-net++, adds a redesigned jump path to combine the output of the previous convolutional layer of the same dense block with the output of the lower dense layer. The sampling outputs corresponding to the block are fused to make the semantic level of the encoded features closer to the semantic level of the feature map to be in the decoder; at the same time, dense skip connections are used to realize the jump path between the encoder and decoder to ensure that all prior feature maps are accumulated and reach the current node through the dense convolution block on each jump path, generating feature maps with full resolution at multiple semantic levels, improving segmentation accuracy and improving gradient flow; in addition, the recognition unit is also used to increase deep supervision, adjust the complexity of the model by pruning the model, and adjust the balance between computational inference speed and performance; after the recognition unit performs side-location, positioning and quantitative recognition on the MRI and PET image data, the recognition result obtained from one of the MRI and PET image data is used as the reference result, and the recognition obtained from other MRI and PET image data collected in the same examination is obtained as the complementary result. The multiple recognition results are verified and compared with the reference results in sequence, and the reference results are complemented for multiple rounds, and the results of the multiple rounds of complementation with the reference results are output as the final recognition result.

[0051] When the acquisition module collects electrophysiological data, the recognition unit smoothes the spike signal through half-wave processing, uses the principal component analysis algorithm to perform wavelet decomposition on the electrophysiological data to obtain signal components, then applies the principal component analysis algorithm to the signal components, and uses the independent component analysis algorithm to find the mixed signal matrix of artifact signals and EEG signals, thereby separating and processing various artifacts from the mixed composite signals; the recognition unit determines the threshold of the abnormal wave through the threshold screening, if the amplitude, frequency, or area under the effective curve exceeds the determined abnormal wave threshold, it is determined to be an abnormal wave; the recognition unit uses expert feature analysis to perform preliminary screening based on the characteristics of various typical interference waveforms in the past, remove eye movement artifacts, blink artifacts, electrode artifacts, and at the same time A bandpass filter is used to complete the filtering processing of the above-mentioned EEG signals to obtain EEG signals in the useful frequency band, and various interference signals outside the frequency band are filtered out. A simple scaling method is used to normalize the electrophysiological data, perform segmented storage processing and label processing; then, the preprocessed electrophysiological sample data is input into a sparse autoencoder, and the features of the electrophysiological data are extracted based on the sparse autoencoder. The results output by the sparse autoencoder are input into a long short-term memory recurrent neural network, the extracted data features are analyzed, and the analysis results based on the long short-term memory recurrent neural network are input into a softmax classifier, thereby completing the classification of the electrophysiological data, and finally outputting the final results of automatic interpretation of the electrophysiological data in the form of Chinese text.

[0052] When the acquisition module collects CT images, the recognition unit uses U-net++ to segment the hemorrhage foci, edematous brain tissue, normal brain tissue, each ventricle and important brain cisterns in the brain window of the head CT scan and extract their corresponding features. It completes the automatic recognition of the head CT images of HICH patients, including side determination, positioning and quantitative recognition, and finally outputs the final results of the automatic interpretation of the CT image in the form of Chinese text.

[0053] Among them, when the recognition unit identifies and interprets electrophysiological data and CT images, one of the acquired data (electrophysiological data and CT images) is used as the reference data, and the other data acquired at the same time is used as the supplementary data. The supplementary data is used to sequentially compare and verify the interpretation results of the reference data, and the interpretation results after multiple rounds of complementarity of the reference data are output as the final interpretation results.

[0054] It also includes an analysis unit, which is used to compare the final result output by the recognition unit with the result obtained by the doctor to determine the accuracy, specificity and sensitivity of the recognition unit; the analysis unit is also used to compare the number of final results output by the recognition unit with that of the doctor within the same unit time, and to calculate and compare the accuracy of the two.

[0055] Among them, the analysis unit uses the first judgment result output by the recognition unit as the reference result, and the judgment results output thereafter as the comparison results, and uses the comparison results to perform multiple rounds of complementation on the reference results. Then, the clinical text data (data recorded by medical staff) is used as the comparison result to perform multiple rounds of complementation on the reference results again. The judgment results after multiple rounds of complementation with the reference results are used for map construction.

[0056] When implementing it specifically,

[0057] The acquisition module collects the electrophysiological data, imaging data and clinical text data of the patient's past treatment events and integrates them into training data, and collects the electrophysiological data, imaging data and clinical text data of the patient's current rehabilitation treatment and integrates them into first data.

[0058] The acquisition module collects CT, MRI, and PET image data (i.e., imaging image data) from HICH patients and stores them in the system of this solution in DICOM format. The recognition unit then identifies and labels the image data, performs denoising on the imaging image data, and labels relevant images. A deep neural network model is then constructed to perform image segmentation and extract image feature information, achieving automatic recognition of the multimodal imaging data of HICH patients. The final results of the automatic interpretation are stored in the form of Chinese text in the training data and the first data.

[0059] After the acquisition module collects electrophysiological data and brain oxygen metabolism data (including electroencephalogram (EEG), electromyogram (EMG), evoked potentials, and brain oxygen saturation, etc.) of HICH patients during rehabilitation treatment, the recognition unit extracts the power spectral density of the electrophysiological data (including EEG, EMG, evoked potentials, etc.), completes the classification of the brain oxygen saturation data set, converts the one-dimensional signal in the data into a two-dimensional time-frequency domain signal, removes some noise and artifacts, then extracts and classifies the electrophysiological data and brain oxygen saturation data, completes feature recognition, and finally stores the final result of automatic interpretation in the form of Chinese text in the training data and the first data.

[0060] The acquisition module collects clinical text data of HICH patients during the treatment and rehabilitation process (including electronic medical records (EMR) during acute hospitalization after onset and rehabilitation-related text data recorded during rehabilitation treatment), and then classifies the clinical text data using the text information rules preset in the system.

[0061] The graph construction module preprocesses the training data and first data using natural language processing techniques such as word segmentation, syntactic analysis, information extraction, word sense disambiguation, and automatic summarization. This involves text classification, keyword extraction, and semantic disambiguation of the training and first data, resulting in preprocessed Chinese medical information strings related to HICH rehabilitation. Based on pre-established HICH rehabilitation-related dictionaries, the obtained Chinese medical information strings are divided into strings with independent semantics and parsed across different dimensions. Next, mechanical word segmentation is used to match the parsed strings within their respective parsed dimensions.

[0062] The HICH rehabilitation knowledge graph is defined as a directed labeled graph consisting of HICH rehabilitation-related entities and their relationships. By constructing a HICH rehabilitation-related "entity-relationship-entity" triple structure, various entities are connected through relationships in the form of subject, predicate, and object, forming a network-like knowledge graph. Entities (i.e., "HICH rehabilitation-related entities") are represented as nodes, and relationships (i.e., medical event relationships between TBI diagnosis and treatment-related entities) are represented as edges. These are defined as follows.

[0063] Ⅰ. Set "HICH rehabilitation related entity" to "H", which is defined as various medical entities related to HICH rehabilitation that can be uniquely identified in HICH rehabilitation related text information records.

[0064] II. Set “HICH rehabilitation medical event relationship” to “R”, which is defined as the medical event relationship that occurs or exists between different HICH rehabilitation-related entities.

[0065] III. Let "HICH rehabilitation knowledge graph" be "K," which is defined as a directed labeled graph. Let T be the vertex set of the knowledge graph, representing the set of all HICH rehabilitation-related entities; let R be the edge set of the knowledge graph, representing the medical event connections between all HICH rehabilitation-related entities.

[0066] Relevant entities in the HICH rehabilitation process were defined and classified, with entities to be identified divided into categories such as "patient, basic information, symptoms, signs, auxiliary examination results, examination plan, prognosis, and rehabilitation plan." A bidirectional long short-term memory-conditional random fields (BiLSTM-CRF) model was then used to construct a corpus, generate a model, and apply the model to complete entity extraction (or named entity recognition).

[0067] The relationships between HICH rehabilitation-related medical events in the EMRs of HICH patients were organized and the entity relationships to be identified were categorized into "belong to," "instance of," "attribute of," "diagnosis," "check," and "treat." Furthermore, since attribute extraction can essentially be considered relationship extraction, the extraction of "HICH rehabilitation-related medical event relationships" in this project will include both entity relationship extraction and attribute extraction. By combining the corresponding kernel functions of a feature-based relationship extraction method and a tree kernel-based method, a combined feature and tree kernel method was developed to extract relationships between HICH rehabilitation-related medical events.

[0068] Then, through steps such as entity disambiguation, reference resolution, and knowledge merging, the TBI-related knowledge integration management is completed.

[0069] Finally, the knowledge processing and management related to TBI diagnosis and treatment are completed through ontology construction and extraction, knowledge reasoning and quality assessment.

[0070] The decision-making module initially involved collaboration between specialists from neurosurgery, neurology, rehabilitation, radiology, nuclear medicine, the electrophysiology center, and the mental health center, along with AI researchers. Based on the latest international and domestic guidelines and consensus for HICH rehabilitation, combined with the actual conditions of rehabilitation treatment and the requirements of AI algorithms, they initially constructed a treatment weighting unit for emergency diagnosis and treatment decisions for HICH patients. Simultaneously, a Gaussian model unit based on TransG-DNN was constructed to achieve automated decision-making with high predictive accuracy, including HICH rehabilitation. Based on the constructed HICH rehabilitation knowledge graph, the Gaussian mixture embedding model (TransG) within the Gaussian model unit was used to vectorize each triple in the HICH rehabilitation knowledge graph for preliminary classification. Based on this, the model was trained using a DNN (using training data). After obtaining the optimal training model, it was applied to the first data set, achieving classification effectiveness and stability in the HICH rehabilitation decision-making model, ultimately completing automated HICH rehabilitation decision-making with high predictive accuracy.

[0071] Treatment options for HICH patients include predictive assessments (such as the Modified and Grading Scale for Injured Heart Disease (ICH) Scale, the New ICH Scale, the ICH Functional Outcome Rating Scale, and the Functional Outcome Scale) as well as actual rehabilitation assessments (such as the mRS, GOS, assessment of consciousness and mental status, intellectual ability, motor function, balance, walking function, and activities of daily living). Treatment options include setting rehabilitation goals (both short-term and long-term), medication, physical therapy, activities of daily living training, exercise therapy, psychotherapy, neuromodulation therapy, brain-computer interface-based rehabilitation training, and traditional Chinese medicine.

[0072] When the analysis unit determines the accuracy, specificity and sensitivity of the recognition unit:

[0073] This tool is used to analyze the accuracy of automatic multimodal imaging image recognition based on U-Net++. The automated multimodal imaging interpretation results generated by this solution are compared with those obtained by doctors to determine the accuracy, specificity, and sensitivity of the imaging results generated by this solution.

[0074] This system is used to analyze the accuracy of automatic recognition of electrophysiological and brain oxygen metabolism data based on neural networks and deep learning. The automated interpretation of electrophysiological and brain oxygen metabolism data output by this solution is compared with the results obtained by doctors to determine the accuracy, specificity, and sensitivity of the results output by this solution.

[0075] This tool is used to analyze the accuracy of rehabilitation decisions based on knowledge graph reasoning. The treatment plan output by this solution is compared with the results obtained by the doctor to determine the accuracy, specificity, and sensitivity of the rehabilitation decision output by this solution.

[0076] This study analyzes the efficiency of automatic multimodal imaging recognition using U-Net++. The accuracy of this method is compared with that of doctors in the same unit time.

[0077] This study analyzes the efficiency of automated recognition of electrophysiological and brain oxygen metabolism data using neural networks and deep learning. The accuracy of this method is compared with that of doctors in the same unit time.

[0078] This tool analyzes the efficiency of completing treatment plans based on knowledge graph reasoning. The number of cases completed by this solution is compared with that of doctors within the same unit time, and the accuracy of the two is calculated and compared.

[0079] Example 2

[0080] like Figure 2As shown, the HICH intelligent rehabilitation health system based on multiple rounds of complementary image and text recognition also includes a rehabilitation pillow, a medical terminal and a patient terminal. The rehabilitation pillow is communicatively connected to the patient terminal, the medical terminal is communicatively connected to the acquisition module and the decision-making module, and the patient terminal is communicatively connected to the acquisition module and the decision-making module; the rehabilitation pillow is used to collect the patient's brain oxygen saturation data and transmit the brain oxygen saturation data as rehabilitation acquisition data to the patient terminal, the patient terminal is used to receive and display the rehabilitation acquisition data, and then transmit the rehabilitation acquisition data as electrophysiological data to the acquisition module, and the patient terminal is also used to receive and display the data transmitted from the decision-making module; the medical terminal is used to read the acquisition data from the acquisition terminal, receive data and input clinical text data.

[0081] like Figure 3 As shown, the rehabilitation pillow includes a shell 3 and multiple near-infrared light probes 6. The surface of the shell 3 that contacts the patient is provided with multiple light holes 1 arranged in a matrix. The near-infrared light probes 6 are fixedly adhered to the bottom of the light holes 1. The near-infrared light probes 6 are embedded in the surface of the shell 3. Multiple pressure sensors 2 are also embedded on the surface of the shell 3. A temperature regulating device 7 is embedded in the shell 3. A power supply device 4 and a processing device 5 are also fixedly adhered in the shell 3. The space between the shell 3 and the power supply device 4 is also filled with flexible and fluffy materials to improve the comfort of use, such as cotton, polyester fiber, foam, etc. The processing device 5 is communicatively connected to the near-infrared light probe 6, the pressure sensor 2 and the temperature regulating device 7, and the power supply device 4 is electrically connected to the near-infrared light probe 6, the pressure sensor 2, the temperature regulating device 7 and the processing device 5; the near-infrared light probe 6 is used to collect the patient's brain oxygen saturation data and transmit it to the processing device 5; the pressure sensor 2 is used to collect the pressure values ​​of each point on the rehabilitation pillow and transmit it to the processing device 5; the temperature regulating device 7 is used to obtain the temperature of the shell 3 and transmit it to the processing device 5; the processing device 5 is used to receive the brain oxygen saturation data, the pressure value and the temperature of the shell 3, and analyze the carrier on the rehabilitation pillow according to the pressure value and the pressure point that provides the pressure value, and then eliminate the interference data in the brain oxygen saturation data through the carrier, thereby obtaining the brain oxygen saturation collection data, and finally sending the brain oxygen saturation collection data as electrophysiological data to the patient terminal.

[0082] like Figure 3As shown, multiple pressure sensors 2 are evenly distributed on the housing 3. The processing device 5 stores the received brain oxygen saturation data, pressure values, and housing 3 temperature in a cache. When the processing device 5 is used to eliminate interference data in the brain oxygen saturation data, it establishes a three-dimensional model of the carrier as a temporary model based on the pressure values ​​and the pressure points providing the pressure values. The processing module is preset with a reference model, which is a pre-entered three-dimensional model of the patient's head and neck. The processing module is used to compare the temporary model with the reference model to obtain similarity. When the similarity is less than 90%, the brain oxygen saturation data collected simultaneously with the pressure value is deleted from the cache as interference data, and the pressure value and the housing 3 temperature collected simultaneously with the pressure value are also deleted. When the similarity is greater than or equal to 90%, the median of the brain oxygen saturation data collected at the same time as the pressure value is selected as the brain oxygen saturation collected data. When the similarity is between 90% and 97%, the brain oxygen saturation collected data is corrected and supplemented using the temporary model. The brain oxygen saturation collected data is then sent to the patient terminal as electrophysiological data, and the other data in the cache outside the temporary model is cleared.

[0083] Among them, when the similarity between the temporary model and the reference model is between 90% and 97%, the patient's body shape has undergone subtle changes. At this time, using the temporary model to supplement the corrective reference model can make the reference model more in line with the patient's current real-time situation. This method can use the real data that best matches the patient's real-time situation to supplement the historical data, so that the historical data can be updated. In the case of multiple updates (that is, using real-time data to supplement historical data for multiple rounds), the rehabilitation pillow's recognition accuracy of the patient (himself) is maintained or even improved, reducing the occurrence of misjudgments and improving the patient's user experience.

[0084] The temperature regulating device 7 is also used to regulate the temperature of the housing 3. The processing device 5 is preset with the maximum brain oxygen saturation, the minimum brain oxygen saturation, the single waiting time and the maximum waiting time. After the processing device 5 sends the brain oxygen saturation collected data as electrophysiological data to the patient terminal and before clearing other data in the buffer, it performs the following steps:

[0085] S10: Calculate the difference between the collected cerebral oxygen saturation data and the maximum cerebral oxygen saturation;

[0086] S20a: If the difference value increases continuously after three consecutive waiting times, a first instruction is sent to the patient terminal;

[0087] S20b: If the patient waits for multiple times in a row and the accumulated waiting time exceeds the maximum waiting time, a second instruction is sent to the patient terminal;

[0088] S20c: If the brain oxygen saturation collected data is lower than the minimum brain oxygen saturation, a third instruction is sent to the patient terminal;

[0089] S20d: If the difference is not less than 0 and the previous difference is less than 0, the temperature regulating device 7 is controlled to be turned on, and the temperature of the housing 3 is controlled to be maintained at 34 degrees Celsius. After counting down the single waiting time, the process returns to S10;

[0090] S20e: If the difference is less than 0 and the previous difference is not less than 0, turn off the temperature adjustment device 7 and return to S10;

[0091] S20f: If the difference between the two most recently calculated values ​​is not less than 0 and is greater than the previous difference, the temperature of the housing 3 is controlled to be maintained at 32 degrees Celsius, and after counting down the single waiting time, the process returns to S10;

[0092] S20g: If the difference between the two most recently calculated values ​​is not less than 0 and is not greater than the previous difference, count down the single waiting time and return to S10;

[0093] The patient terminal is preset with a first prompt, a second prompt, a third prompt and a closing time. After receiving the first instruction, the second instruction or the third prompt, the patient terminal immediately displays the first prompt, the second prompt or the third prompt, and broadcasts the first prompt, the second prompt or the third prompt in a loop. If the time of the loop broadcast exceeds the closing time, an alarm signal is sent to the acquisition terminal; if the patient terminal turns off the loop broadcast within the closing time, and selects a delay time after turning off the loop broadcast, the first instruction, the second instruction and the third instruction will not be received during the delay time; if the patient terminal turns off the loop broadcast within the closing time, and chooses to send an alarm signal after turning off the loop broadcast, an alarm signal is sent to the acquisition terminal.

[0094] When the health pillow collects abnormal data, the abnormal situation does not reflect the actual situation of the patient himself. At this time, the patient or the patient's family needs to verify and supplement the judgment result (made by the health pillow). When the patient or the patient's family fails to provide verification and supplement operations in time, the judgment result (made by the health pillow) can be verified by the patient's other physiological data, that is, the judgment result (made by the health pillow) can be supplemented multiple times using the patient's other physiological data to increase the accuracy of the health pillow's judgment.

[0095] The temperature regulating device 7 is also used to send the collected shell 3 temperature and adjustment records to the patient terminal after being turned on. After receiving the shell 3 temperature and adjustment records, the patient terminal matches the shell 3 temperature and adjustment records with the brain oxygen saturation collection data according to the collection time, and converts them into clinical text data and sends them to the collection module; after receiving the alarm signal, the collection module uses the alarm signal sending time as the accident time node, and sends the electrophysiological data, clinical text data and alarm signal one hour before and after the accident time node to the medical terminal and the decision-making module; after receiving the alarm signal, the medical terminal emits an emergency prompt sound and displays the received electrophysiological data and clinical text data. After receiving the alarm signal, the decision module marks the received electrophysiological data and clinical text data as the highest priority, and uses the electrophysiological data and clinical text data to calculate an emergency treatment plan, and sends the emergency treatment plan to the patient terminal. After receiving the emergency treatment plan, the patient terminal plays the emergency treatment plan.

[0096] During specific implementation, patient A is regarded as the user of the rehabilitation pillow, caregiver A is regarded as the caregiver of patient A, the patient's mobile phone is regarded as the patient terminal, and the doctor's computer is regarded as the medical terminal.

[0097] Before using the rehabilitation pillow, patient A has entered a three-dimensional model of his own head and neck into the processing module as reference model A.

[0098] One day, Patient A, instead of using a rehabilitation pillow to sleep, placed their phone on the pillow. The pillow's pressure sensor 2, which detected pressure, transmitted the pressure value to the processing device 5. Simultaneously, the near-infrared probe 6 and temperature control device 7 activated to collect brain oxygen saturation data and the temperature of the housing 3. The processing device 5 records the location of each pressure sensor 2. After receiving the pressure value, the processing device 5 identifies the corresponding point of the pressure sensor 2 that transmitted the pressure value as the pressure point, and then associates each pressure point with the pressure value. The processing device then creates a three-dimensional model based on all the pressure values ​​and pressure points, serving as a temporary model (the 3D model corresponding to the phone). The processing device 5 compares the reference model A with the temporary model to obtain a similarity 1 of 2%, which is less than 90%. The rehabilitation pillow then deletes the brain oxygen saturation data collected simultaneously with the pressure value from its cache as interference data. It also deletes the pressure value and the temperature of the housing 3 collected simultaneously with the pressure value.

[0099] After a period of time, caregiver A takes a nap using the rehabilitation pillow. The processing device 5 of the rehabilitation pillow compares the reference model A with the temporary model established this time to obtain similarity 2. Similarity 2 is 80%, which is less than 90%. The rehabilitation pillow deletes the brain oxygen saturation data collected at the same time as the pressure value from the cache as interference data, and also deletes the pressure value and the shell 3 temperature collected at the same time as the pressure value.

[0100] That night, patient A used the rehabilitation pillow to fall asleep. The processing device 5 of the rehabilitation pillow compared the reference model A with the temporary model established this time to obtain similarity 3. The similarity 3 was 98%, which was greater than 90%. The rehabilitation pillow selected the median of the brain oxygen saturation data collected at the same time as the pressure value as the brain oxygen saturation collection data (8.09), and then sent the brain oxygen saturation collection data as electrophysiological data to the patient terminal. At the same time, the shell 3 temperature (36.2 degrees Celsius) collected at the same time as the pressure value was transmitted to the patient terminal, and the patient terminal sent the electrophysiological data and the shell 3 temperature to the collection module.

[0101] The processing device 5 is preset with a maximum brain oxygen saturation of 10.09, a minimum brain oxygen saturation of 7.88, a single waiting time of 5 minutes, and a maximum waiting time of 30 minutes.

[0102] The rehabilitation pillow calculates that the difference between the brain oxygen saturation collection data and the maximum brain oxygen saturation is -2, which is less than 0. The rehabilitation pillow continues to collect the patient's brain oxygen saturation data.

[0103] After a period of time, the rehabilitation pillow calculates that the difference between the brain oxygen saturation collection data and the highest brain oxygen saturation is 0.5, which is greater than 0, and controls the temperature adjustment device 7 to turn on (turn it on again to ensure that the temperature adjustment device 7 is in the on state), and controls the temperature of the shell 3 to be maintained at 34 degrees Celsius. After the countdown single waiting time (5 minutes), the difference between the brain oxygen saturation collection data and the highest brain oxygen saturation is calculated again to be 0.2. The differences between the two most recent calculations are not less than 0, and the current difference (0.2) is not greater than the previous difference (0.5). After the countdown single waiting time (5 minutes), the difference between the brain oxygen saturation collection data and the highest brain oxygen saturation is calculated again to be -0.2. The current difference is less than 0 (-0.2) and the previous difference is not less than 0 (0.2). The control device turns off the temperature adjustment device 7, and the rehabilitation pillow continues to collect the patient's brain oxygen saturation data.

[0104] After a period of time, the rehabilitation pillow calculates that the difference between the collected cerebral oxygen saturation data and the maximum cerebral oxygen saturation is 0.4. 5 minutes later, the difference is measured again and is 0.9. The control device controls the housing 3 to maintain its temperature at 32 degrees Celsius. 5 minutes later, the difference is measured again and is 1.3. The processing device 5 determines that the waiting time has continued for three consecutive times and the difference is increasing. The processing device 5 sends a first instruction to the patient terminal (mobile phone). After receiving the instruction, the mobile phone displays a first prompt and repeats the first prompt in a loop. After hearing the first prompt on the mobile phone, caregiver A immediately rushes to check on patient A's condition and finds that patient A is in an abnormal state. Caregiver A closes the prompt and chooses to send an alarm signal. The mobile phone then sends the alarm signal to the acquisition module. After receiving the alarm signal, the acquisition module uses the alarm signal transmission time as the accident time node and sends the electrophysiological data, clinical text data, and alarm signal for one hour before and after the accident time node to the medical terminal and decision module. After receiving the alarm signal, the medical terminal (doctor's computer) emits an emergency prompt sound and displays the received electrophysiological data and clinical text data. After hearing the emergency prompt sound, the doctor checks the data displayed on the computer and enters the clinical text data into the acquisition module. After receiving the alarm signal, the decision module marks the received electrophysiological data and clinical text data as the highest priority, and uses the electrophysiological data and clinical text data to calculate an emergency treatment plan, and sends the emergency treatment plan to the patient's mobile phone. After receiving the emergency treatment plan, the patient's mobile phone plays the emergency treatment plan, and caregiver A performs first aid for the patient according to the emergency treatment plan broadcast by the mobile phone.

[0105] The above is only an embodiment of the present invention, and the specific structure and characteristic common sense known in the scheme are not described in detail here. It should be pointed out that for those skilled in the art, several variations and improvements can be made without departing from the structure of the present invention, and these should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods described in the specification can be used to interpret the content of the claims.

Claims

1. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition is characterized by: include: An acquisition module, configured to collect electrophysiological data, imaging data, and clinical text data of a patient's past treatment events, and integrate them into training data and first data; The graph construction module is used to identify imaging data and electrophysiological data as text data, perform text recognition and semantic extraction on the text data, determine entities and entity relationships through multi-party verification of the extracted content, and establish the HICH rehabilitation knowledge graph. It is also used to perform text recognition and semantic extraction on clinical text data, and use the extracted content to check and supplement the HICH rehabilitation knowledge graph; A decision module includes a treatment weight unit and a Gaussian model unit. The treatment weight unit is a computational model constructed based on TransG-DNN. The treatment weight unit combines the HICH rehabilitation guidelines and consensus with the actual situation during rehabilitation treatment. The Gaussian model unit is a Gaussian mixture model established based on the HICH rehabilitation knowledge graph. The decision module is used to place the first data into the Gaussian mixture model to calculate the condition information, and then place the condition information into the treatment weight unit to calculate the treatment plan and treatment expectations. The acquisition module also includes a recognition unit, which is a computing model based on U-net++ deep learning and neural network. When the acquisition module acquires multimodal MRI and PET images in the imaging data, the recognition unit mainly automatically recognizes the cranial MRI and PET images during the patient's rehabilitation process. The recognition unit is used to evaluate the integrity of the patient's brain parenchyma and white matter fiber bundles, cerebral blood perfusion, and brain function retention, and finally outputs the final result of the automatic interpretation of MRI and PET images in the form of Chinese text; the recognition unit uses the method of CT image recognition to determine the side, position and quantitatively recognize the MRI and PET image data; the recognition unit uses U-net++ to add a dense block and a convolutional layer between the encoder and the decoder, and adds a redesigned jump path on the basis of the original U-net++, and connects the output of the previous convolutional layer in the same dense block with the output of the encoder with a depth layer lower than the current dense layer. The sampled outputs corresponding to the dense block of the block are fused to make the semantic level of the encoded features closer to the semantic level of the feature map to be in the decoder. At the same time, dense skip connections are used to implement the jump path between the encoder and decoder to ensure that all prior feature maps are accumulated and reach the current node through the dense convolution blocks on each jump path, generating full-resolution feature maps at multiple semantic levels, improving segmentation accuracy and gradient flow. In addition, the recognition unit is also used to increase deep supervision, adjust the complexity of the model by pruning the model, and adjust the balance between computational inference speed and performance. After the recognition unit performs lateral, positioning and quantitative recognition on the MRI and PET image data, the recognition result obtained from one of the MRI and PET image data is used as the reference result, and the recognition results obtained from other MRI and PET image data collected in the same examination are obtained as complementary results. The multiple complementary results are verified and compared with the reference results in sequence, and the reference results are complemented for multiple rounds, and the results of the multiple rounds of complementation with the reference results are output as the final recognition result.

2. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition according to claim 1 is characterized by: When the acquisition module collects electrophysiological data, the recognition unit smoothes the spike signal in a half-wave processing manner, performs wavelet decomposition on the electrophysiological data to obtain signal components, then applies the principal component analysis algorithm to the signal components, and uses the independent component analysis algorithm to find the mixed signal matrix of the artifact signal and the EEG signal, thereby separating and processing various artifacts from the mixed composite signal; the recognition unit determines the threshold of the abnormal wave through the threshold initial screening, and if the amplitude, frequency, or area under the effect curve exceeds the determined abnormal wave threshold, it is determined to be an abnormal wave; the recognition unit uses expert feature analysis to perform initial screening based on the characteristics of various typical interference waveforms in the past, removes eye movement artifacts, blink artifacts, and electrode artifacts, and at the same time uses A bandpass filter is used to complete the filtering processing of the above-mentioned EEG signals to obtain EEG signals in the useful frequency band, and various interference signals outside the frequency band are filtered out. A simple scaling method is used to normalize the electrophysiological data, perform segmented storage processing and label processing; then, the preprocessed electrophysiological sample data is input into a sparse autoencoder, and the features of the electrophysiological data are extracted based on the sparse autoencoder. The output of the sparse autoencoder is input into a long short-term memory recurrent neural network, and the extracted data features are analyzed. The analysis results based on the long short-term memory recurrent neural network are input into a softmax classifier, thereby completing the classification of the electrophysiological data, and finally outputting the final result of automatic interpretation of the electrophysiological data in the form of Chinese text.

3. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition according to claim 2 is characterized by: When the acquisition module acquires the CT image, the recognition unit uses U-net++ to segment the brain parenchyma hemorrhage, edematous brain tissue, normal brain tissue, each ventricle and important brain cisterns in the brain window of the head CT and extract their corresponding features. Complete automatic recognition of HICH patient head CT images, including side determination, positioning and quantitative recognition, and finally output the final results of automatic interpretation of CT images in the form of Chinese text.

4. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition according to claim 3 is characterized by: It also includes an analysis unit, which is used to compare the final result output by the recognition unit with the final result output by the doctor to determine the accuracy, specificity and sensitivity of the recognition unit; the analysis unit is also used to compare the number of final results output by the recognition unit with the number of final results output by the doctor within the same unit time, and simultaneously calculate and compare the accuracy of the two.

5. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition according to claim 4 is characterized by: It also includes a rehabilitation pillow, a medical terminal and a patient terminal. The rehabilitation pillow is communicatively connected to the patient terminal, the medical terminal is communicatively connected to the acquisition module and the decision-making module, and the patient terminal is communicatively connected to the acquisition module and the decision-making module. The rehabilitation pillow is used to collect the patient's brain oxygen saturation data and transmit the brain oxygen saturation data as rehabilitation acquisition data to the patient terminal. The patient terminal is used to receive and display the rehabilitation acquisition data, and then transmit the rehabilitation acquisition data as electrophysiological data to the acquisition module. The patient terminal is also used to receive and display the data transmitted from the decision-making module. The medical terminal is used to read the acquisition data from the acquisition terminal, receive data and input clinical text data.

6. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition according to claim 5 is characterized by: The rehabilitation pillow includes a shell and multiple near-infrared light probes, which are embedded in the shell surface. Multiple pressure sensors are also embedded in the shell surface. A temperature adjustment device is embedded in the shell. A power supply device and a processing device are also fixedly connected to the shell. The processing device is communicatively connected to the near-infrared light probe, pressure sensor and temperature adjustment device, and the power supply device is electrically connected to the near-infrared light probe, pressure sensor, temperature adjustment device and processing device. The near-infrared light probe is used to collect the patient's brain oxygen saturation data and transmit it to the processing device. The pressure sensor is used to collect the pressure values ​​of various points on the rehabilitation pillow and transmit them to the processing device. The temperature adjustment device is used to obtain the shell temperature and transmit it to the processing device. The processing device is used to receive the brain oxygen saturation data, pressure values ​​and shell temperature, and analyze the load on the rehabilitation pillow based on the pressure values ​​and the pressure points that provide the pressure values, and then eliminate interference data in the brain oxygen saturation data through the load, thereby obtaining brain oxygen saturation collection data, and finally sending the brain oxygen saturation collection data as electrophysiological data to the patient terminal.

7. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition according to claim 6 is characterized by: Multiple pressure sensors are evenly distributed on the shell, and the processing device stores the received brain oxygen saturation data, pressure values, and shell temperature in a cache. When the processing device is used to eliminate interference data in the brain oxygen saturation data, a three-dimensional model of the carrier is established as a temporary model based on the pressure values ​​and the pressure points providing the pressure values. A reference model is preset in the processing module, and the reference model is a three-dimensional model of the patient's head and neck that is recorded in advance. The processing module is used to compare the temporary model and the reference model to obtain similarity. When the similarity is less than 90%, the brain oxygen saturation data collected simultaneously with the pressure value is deleted from the cache as interference data, and the pressure value and the shell temperature collected simultaneously with the pressure value are also deleted. When the similarity is greater than or equal to 90%, the median of the brain oxygen saturation data collected at the same time as the pressure value is selected as the brain oxygen saturation collected data. When the similarity is between 90% and 97%, the brain oxygen saturation collected data is corrected and supplemented using the temporary model, and then the brain oxygen saturation collected data is sent to the patient terminal as electrophysiological data, and then other data in the cache outside the temporary model is cleared.

8. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition according to claim 7 is characterized by: The temperature regulating device is also used to regulate the temperature of the housing. The processing device is preset with the maximum brain oxygen saturation, the minimum brain oxygen saturation, the single waiting time and the maximum waiting time. After the processing device sends the brain oxygen saturation collected data as electrophysiological data to the patient terminal and before clearing other data in the buffer, it performs the following steps: S10: Calculate the difference between the collected cerebral oxygen saturation data and the maximum cerebral oxygen saturation; S20a: If the difference value increases continuously after three consecutive waiting times, a first instruction is sent to the patient terminal; S20b: If the patient waits for multiple times in a row and the accumulated waiting time exceeds the maximum waiting time, a second instruction is sent to the patient terminal; S20c: If the brain oxygen saturation collected data is lower than the minimum brain oxygen saturation, a third instruction is sent to the patient terminal; S20d: If the difference is not less than 0 and the last difference is less than 0, the temperature regulating device is turned on and the housing temperature is controlled to be maintained at 34 degrees Celsius. After counting down the single waiting time, the process returns to S10. S20e: If the difference is less than 0 and the previous difference is not less than 0, turn off the temperature adjustment device and return to S10; S20f: If the difference between the two most recent calculations is not less than 0 and is greater than the previous difference, the housing temperature is controlled to be maintained at 32 degrees Celsius, and after counting down the single waiting time, the process returns to S10; S20g: If the difference between the two most recently calculated values ​​is not less than 0 and is not greater than the previous difference, count down the single waiting time and return to S10; The patient terminal is preset with a first prompt, a second prompt, a third prompt and a closing time. After receiving the first instruction, the second instruction or the third prompt, the patient terminal immediately displays the first prompt, the second prompt or the third prompt, and broadcasts the first prompt, the second prompt or the third prompt in a loop. If the time of the loop broadcast exceeds the closing time, an alarm signal is sent to the acquisition terminal; if the patient terminal turns off the loop broadcast within the closing time, and selects a delay time after turning off the loop broadcast, the first instruction, the second instruction and the third instruction will not be received during the delay time; if the patient terminal turns off the loop broadcast within the closing time, and chooses to send an alarm signal after turning off the loop broadcast, an alarm signal is sent to the acquisition terminal.

9. The HICH intelligent rehabilitation health system based on multi-round complementary image and text recognition according to claim 8 is characterized by: The temperature adjustment device is also used to send the collected shell temperature and adjustment records to the patient terminal after being turned on. After the patient terminal receives the shell temperature and adjustment records, it matches the shell temperature and adjustment records with the brain oxygen saturation collection data according to the collection time, and converts them into clinical text data and sends them to the collection module; after the collection module receives the alarm signal, it uses the alarm signal sending time as the accident time node, and sends the electrophysiological data, clinical text data and alarm signal one hour before and after the accident time node to the medical terminal and decision-making module; after receiving the alarm signal, the medical terminal emits an emergency prompt sound and displays the received electrophysiological data and clinical text data. After receiving the alarm signal, the decision module marks the received electrophysiological data and clinical text data as the highest priority, and uses the electrophysiological data and clinical text data to calculate an emergency treatment plan, and sends the emergency treatment plan to the patient terminal. After receiving the emergency treatment plan, the patient terminal plays the emergency treatment plan.

Images