Intelligent nursing bed control system and method based on multi-modal information

Abstract

The invention belongs to the technical field of intelligent nursing bed control, and particularly relates to an intelligent nursing bed control system and method based on multi-modal information, and the system comprises an information collection module which is used for monitoring a physiological signal of a user in real time; the physiological signals at least comprise electroencephalogram signals, eye movement signals and body position body pressure signals; the fusion control module is used for fusing the acquired physiological signals to obtain multi-modal information, analyzing the obtained multi-modal information and judging the physiological state and intention of the user; and the driving execution module is used for driving an electric push rod of the intelligent nursing bed to act according to the obtained physiological state and intention of the user and adjusting the posture of the user.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent nursing bed control, and in particular relates to an intelligent nursing bed control system and method based on multimodal information. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the intensification of the aging trend of the population, the incidence of diseases such as stroke and hip fracture has increased significantly, becoming an important problem that needs to be solved urgently in the field of medical care worldwide. According to statistics, about 70% of stroke patients will have varying degrees of limb movement disorders, and the recovery process of patients with hip fractures requires long-term bed rehabilitation training and care. Such patients often face the risk of long-term bed rest, which is prone to complications such as pressure sores and thrombosis. In addition, the existing care beds cannot automatically adjust their posture according to the patient's physiological state, resulting in an increase in the care burden and a significant decrease in the patient's quality of life. Therefore, the development of an intelligent and multifunctional care bed control system is of great significance for improving the care effect and reducing the care burden of patients with stroke and hip fractures.

[0004] Traditional nursing beds have a relatively simple posture adjustment function, mainly relying on manual adjustment. They lack intelligent and automated adjustment mechanisms and cannot respond to patients' physiological changes and personalized needs in real time. Existing intelligent bed control systems face many challenges in physiological signal acquisition, data fusion processing, and control algorithm design. For example, the acquisition and processing of physiological signals such as EEG are susceptible to external interference, signal synchronization is difficult and has low accuracy, resulting in the control system being unable to achieve accurate and real-time posture adjustment, making it difficult to meet personalized rehabilitation care needs.

[0005] Existing research focuses on the application of a single physiological signal, and lacks systematic research on multimodal signal fusion. How to effectively integrate multiple physiological signals, accurately interpret the patient's physiological state and intention, and combine intelligent algorithms to precisely control the bed is a key problem in the development of intelligent care bed systems. At the same time, how to achieve reliable wireless transmission of physiological signals and control data in a complex care environment and ensure data security and real-time performance has also become a major bottleneck in the application and promotion of intelligent care bed systems. Summary of the invention

[0006] To solve the above problems, the present invention proposes an intelligent care bed control system and method based on multimodal information, which realizes patient intention recognition and personalized care adjustment based on the effective fusion of multiple physiological signals such as EEG signals, muscle signals and mattress pressure signals, improves the patient's rehabilitation effect and quality of life, and reduces the workload of caregivers.

[0007] According to some embodiments, a first solution of the present invention provides an intelligent nursing bed control system based on multimodal information, which adopts the following technical solutions:

[0008] An intelligent nursing bed control system based on multimodal information, comprising:

[0009] An information acquisition module, used to monitor the user's physiological signals in real time; the physiological signals at least include EEG signals, eye movement signals, and body position and pressure signals;

[0010] A fusion control module, used to fuse the acquired physiological signals to obtain multimodal information, analyze the obtained multimodal information, and determine the user's physiological state and intention;

[0011] The drive execution module is used to drive the electric push rod of the intelligent care bed and adjust the user's posture according to the user's physiological state and intention.

[0012] As a further technical limitation, the information acquisition module includes an electroencephalogram measuring device for real-time monitoring of the user's brain electrical signals, an eye tracking device for real-time capturing the user's eye movement signals, and a plurality of pressure sensors placed in the mattress for real-time monitoring of the user's body position and pressure signals.

[0013] Furthermore, the user's Steady-State Visual Evoked Potentials (SSVEP) signal is obtained through the electroencephalogram measurement device; the fusion control module performs preprocessing and feature extraction operations on the obtained steady-state visual evoked potential model in sequence to obtain a feature vector, and uses a classification recognition algorithm to convert the obtained feature vector into the user's intention.

[0014] Furthermore, based on the eye movement information of the user captured by the eye tracking device, the eye movement information is sent to the fusion control module when and only when the acquired eye movement information meets the gaze condition, otherwise the signal continues to be captured.

[0015] Furthermore, the pressure sensor is placed inside the mattress to monitor the pressure distribution between the user's body surface and the bed surface in real time to obtain a user's body position and pressure signal; the fusion control module receives and processes the obtained user's body position and pressure signal to obtain a user's body pressure distribution map, and displays the obtained user's body pressure distribution map on a display screen of the fusion control module.

[0016] As a further technical limitation, the execution module includes at least a back electric drive unit, a torso electric drive unit and a leg electric drive unit. The fusion control module sends drive instructions to the drive execution module based on the obtained user physiological state and intention, controls and drives the electric push rods of different electric drive units, and adjusts the user posture.

[0017] According to some embodiments, the second solution of the present invention provides a method for controlling an intelligent nursing bed based on multimodal information, which adopts the intelligent nursing bed control system based on multimodal information provided by the first solution, and adopts the following technical solutions:

[0018] A method for controlling an intelligent nursing bed based on multimodal information, comprising:

[0019] Acquire real-time physiological signals of the user including at least EEG signals, eye movement signals, and body position and pressure signals;

[0020] Integrate and analyze the real-time physiological signals of the user to determine the user's physiological state and intention;

[0021] According to the user's physiological state and intention, the electric push rod of the smart care bed is driven to adjust the user's posture.

[0022] According to some embodiments, a third solution of the present invention provides a computer-readable storage medium, which adopts the following technical solution:

[0023] A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps in the intelligent nursing bed control system based on multimodal information as described in the second solution of the present invention.

[0024] According to some embodiments, a fourth solution of the present invention provides an electronic device, which adopts the following technical solution:

[0025] An electronic device comprises a memory, a processor and a program stored in the memory and running on the processor, wherein when the processor executes the program, the steps in the intelligent nursing bed control system based on multimodal information as described in the second solution of the present invention are implemented.

[0026] According to some embodiments, a fifth solution of the present invention provides a computer program product, which adopts the following technical solution:

[0027] A computer program product includes software codes, wherein the program in the software codes executes the steps in the intelligent nursing bed control system based on multimodal information as described in the second embodiment of the present invention.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention integrates multimodal physiological signal acquisition, multimode control strategy and wireless communication technology to establish an intelligent bed control system for stroke and hip fracture patients, realizes patient intention recognition and personalized care adjustment, and is expected to achieve intelligent and personalized care adjustment based on multimodal physiological signals, improve patients' rehabilitation effects and quality of life, reduce the workload of caregivers, and promote the industrial application of intelligent care equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings in the specification that constitute a part of this embodiment are used to provide a further understanding of this embodiment. The schematic embodiments of this embodiment and their descriptions are used to explain this embodiment and do not constitute improper limitations on this embodiment.

[0031] Figure 1 Schematic diagram of the structure of an intelligent nursing bed control system based on multimodal information in Embodiment 1 of the present invention;

[0032] Figure 2 This is a signal acquisition principle diagram of the pressure sensing device in the first embodiment of the present invention;

[0033] Figure 3 This is a communication principle diagram of the pressure sensing device in the first embodiment of the present invention;

[0034] Figure 4 This is an eye movement signal control logic diagram in Embodiment 1 of the present invention;

[0035] Figure 5 This is a logic diagram of EEG signal control in Embodiment 1 of the present invention;

[0036] Figure 6 This is a flow chart of a smart care bed control method based on multimodal information in Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0038] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0039] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0040] In the present invention, terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are relational words determined only for the convenience of describing the structural relationships of the various parts or elements of the present invention, and do not specifically refer to any part or element in the present invention and should not be understood as limitations on the present invention.

[0041] In the present invention, terms such as "fixed connection", "connected", "connection", etc. should be understood in a broad sense, indicating that it can be fixedly connected, integrally connected or detachably connected; it can be directly connected or indirectly connected through an intermediate medium. For relevant scientific research or technical personnel in this field, the specific meanings of the above terms in the present invention can be determined according to specific circumstances, and they cannot be understood as limitations on the present invention.

[0042] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0043] Embodiment 1

[0044] Embodiment 1 of the present invention introduces an intelligent nursing bed control system based on multimodal information.

[0045] This embodiment collects EEG and eye movement signals from stroke and hip fracture patients, combines them with the pressure signals of the mattress, and designs a multimodal control system to achieve gentle care control for the patients.

[0046] like Figure 1 As shown, this embodiment integrates multiple signal acquisition and feedback modules, including an EEG measurement module, an eye movement module, a pressure measurement module, a main control module, and a drive module;

[0047] The EEG measurement module monitors the patient's EEG signals in real time through an EEG device. These signals can be used to determine the patient's physiological state and intentions, such as whether there is a need to turn over or adjust posture. The EEG measurement module transmits the collected signals to the main control module wirelessly or wiredly for further processing and decision-making.

[0048] The eye movement module captures the patient's eye movements through an eye tracking device and converts these movement data into control commands. Patients can select different operation modes by looking at a certain area, such as turning over, adjusting the bed position, or exiting the current operation mode.

[0049] The pressure measurement module monitors the pressure distribution between the patient's body surface and the bed surface in real time through the 128-channel pressure sensors built into the mattress. These sensors can accurately sense the patient's body position and body pressure changes, thereby preventing problems such as bedsores caused by long-term local pressure. The pressure data is transmitted to the main control module, which generates a body pressure distribution map of the patient by analyzing the data and displays it on the industrial computer for medical staff to view.

[0050] The main control module comprehensively processes data from EEG, eye movement and pressure sensors, and makes reasonable decisions through intelligent algorithms. For example, by combining EEG signals and pressure signals, the system can determine whether the patient needs to turn over and actively adjust the bed position. At the same time, the industrial computer also sends control instructions to the drive module to control the movement of the bed.

[0051] The drive module completes specific mechanical actions through electric push rods. The drive module includes electric drive units responsible for the back, torso and legs, which can accurately adjust the patient's posture to meet personalized care needs. When the main control module issues an instruction, the drive module controls the corresponding electric push rod to move, completing actions such as lifting the back and turning over. This design based on multi-sensor fusion and intelligent control significantly improves the system's response speed and control accuracy.

[0052] In order to measure multiple pressure sensor signals simultaneously, this embodiment adopts F407 series STM32, uses ADC digital-to-analog conversion module to read external pressure sensor signals, adopts multi-channel multiplexing module (CD74HC4067M96) to expand the number of measurable channels, and uses ESP8266 module to realize TCP / IP communication with industrial computer, and transmits 128 pressure sensor signals to industrial computer in real time, and the industrial computer displays them in real time.

[0053] like Figure 2 and Figure 3 As shown, this embodiment uses the MCP6004-E / SL operational amplifier to measure the external pressure sensor. 16 pressure signals are simultaneously used through the CD74HC4067M96 multi-channel multiplexing module. It is worth noting that each acquisition module can simultaneously acquire up to 32 pressure sensor signals. Figure 3 This is the circuit schematic diagram of the communication mainboard. By planning the circuit, the pins of the communication mainboard are configured as the ADC input of the STM32 and connected to the pressure measurement output of the daughter board, thereby performing 128-channel pressure sensor measurement.

[0054] In this embodiment, data collection and processing of a multi-channel pressure sensor are realized from data reception, verification, splitting to graphical display, specifically:

[0055] This embodiment defines a gradient color mapping, which gradually transitions from blue to green, yellow, and then to red, for generating a heat map. The heat map dynamically displays the real-time data of 128 pressure sensors and provides visual sensor status feedback. The TCP server part establishes a connection through the specified IP and port, and continuously receives sensor data transmitted from the client. After the data is received, the program splits the data packet through the frame header and frame tail, and uses the CRC16 check algorithm to check the data integrity. The verified data is further parsed, and the data is classified according to different flags (such as A1, A2, etc.) and mapped to the corresponding storage unit. The data is rearranged during the mapping process to adapt to the actual hardware configuration, and the disordered sensor data is restored to the original order through the predefined disordered index. For some data groups (such as A3, A4, etc.), array transformation operations are further performed to extend the data by inserting zero elements to match specific matrix or graphic display requirements. Finally, the processed data will be dynamically displayed in the form of a heat map, providing users with real-time multi-channel pressure data visualization. The entire program also includes some auxiliary functions, such as removing empty data items and rearranging data, to ensure the completeness and accuracy of data processing.

[0056] This embodiment uses a Tobii eye tracker to identify the patient's eye movement information and gaze points to capture the patient's eye movement information to control the movement of the nursing bed. The control logic is as follows: Figure 4 As shown, specifically:

[0057] Eye movement information indicates the position (x, y coordinates) of the patient's viewpoint mapped to the screen. Before using the device, the patient needs to perform a simple calibration to ensure that the eye tracker can accurately identify the position of his gaze. After that, the eye tracker will capture the patient's eye movements in real time. When certain conditions are met, it will send instructions to the controller of the care bed to control the care bed to perform corresponding actions.

[0058] like Figure 5As shown in the EEG signal logic diagram, this embodiment collects the user's SSVEP signal through an EEG measurement device, and captures the electrophysiological response of the user's brain under visual stimulation of a specific frequency. These electrophysiological responses contain key information about the user's intention; the collected EEG signal is then preprocessed to improve the signal quality by removing noise and interference, so as to accurately analyze the signal characteristics; features are extracted from the preprocessed signal to identify the key links of the user's intention, and feature vectors that can characterize the user's intention are extracted by analyzing specific patterns or parameters in the EEG signal. These feature vectors will serve as input for subsequent classification and recognition; the extracted features are mapped to specific control instructions using a classification and recognition algorithm. The classification and recognition stage is based on machine learning or deep learning algorithms, and the user's intention is recognized by training the model, and the recognition results are converted into instructions that can control the movement of the medical bed. These instructions may include adjusting the height and tilt angle of the bed.

[0059] The control instruction generation module generates specific control instructions based on the results of classification and recognition, and sends them to the medical bed control system. After receiving the instructions, the medical bed control system drives the medical bed to perform corresponding actions, such as lifting, tilting, etc. At the same time, the system also has status feedback and monitoring functions, which can monitor the status of the medical bed and the user in real time to ensure the safety and effectiveness of the entire control process.

[0060] This embodiment identifies the steady-state visual evoked potential (SSVEP) frequency by analyzing the power spectral density (PSD) of the acquired EEG signal; first, a function simulate_eeg_data is defined, which generates simulated EEG data including a sine wave signal and random noise according to the duration, sampling rate and SSVEP frequency specified by the user; then the calculate_psd function and the Welch method are used to calculate the PSD of this simulated EEG data to provide power distribution information of the signal at different frequencies; to identify the SSVEP frequency from the PSD, this embodiment defines the identify_ssvep_frequency function, sets a threshold (if not specified by the user, the default is 50% of the maximum value of the PSD), and uses the find_peaks function to detect peaks in the PSD that exceed this threshold; to focus on the typical range of SSVEP frequencies (e.g., 15 Hz to 60 Hz), the function further filters these peaks and retains only those that fall within this range. Finally, the function returns the frequency with the highest PSD value among these correlation peaks as the identified SSVEP frequency; simulated EEG data is generated by setting simulation parameters (such as duration, sampling rate, and SSVEP frequency), and the above functions are called in sequence to calculate PSD and identify SSVEP frequency.

[0061] This embodiment integrates multimodal physiological signal acquisition, multimode control strategy and wireless communication technology to establish an intelligent bed control system for stroke and hip fracture patients, realize patient intention recognition and personalized care adjustment, and is expected to achieve intelligent and personalized care adjustment based on multimodal physiological signals, improve patients' rehabilitation effects and quality of life, reduce the workload of caregivers, and promote the industrial application of intelligent care equipment.

[0062] Embodiment 2

[0063] The second embodiment of the present invention introduces a method for controlling an intelligent nursing bed based on multimodal information, and adopts the intelligent nursing bed control system based on multimodal information introduced in the first embodiment.

[0064] like Figure 6 A method for controlling an intelligent nursing bed based on multimodal information is shown, comprising:

[0065] Acquire real-time physiological signals of the user including at least EEG signals, eye movement signals, and body position and pressure signals;

[0066] Integrate and analyze the real-time physiological signals of the user to determine the user's physiological state and intention;

[0067] According to the user's physiological state and intention, the electric push rod of the smart care bed is driven to adjust the user's posture.

[0068] The detailed steps are the same as the working principle of the intelligent nursing bed control system based on multimodal information provided in Example 1, and will not be repeated here.

[0069] Embodiment 3

[0070] Embodiment 3 of the present invention provides a computer-readable storage medium.

[0071] A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps in the intelligent care bed control system based on multimodal information as described in the second embodiment of the present invention.

[0072] The detailed steps are the same as those of the intelligent nursing bed control system based on multimodal information provided in Example 2, and will not be repeated here.

[0073] Embodiment 4

[0074] A fourth embodiment of the present invention provides an electronic device.

[0075] An electronic device comprises a memory, a processor and a program stored in the memory and running on the processor, wherein when the processor executes the program, the steps in the intelligent nursing bed control system based on multimodal information as described in the second embodiment of the present invention are implemented.

[0076] The detailed steps are the same as those of the intelligent nursing bed control system based on multimodal information provided in Example 2, and will not be repeated here.

[0077] Embodiment 5

[0078] Embodiment 5 of the present invention provides a computer program product.

[0079] A computer program product includes software codes, wherein the program in the software codes executes the steps in the intelligent nursing bed control system based on multimodal information as described in the second embodiment of the present invention.

[0080] The detailed steps are the same as those of the intelligent nursing bed control system based on multimodal information provided in Example 2, and will not be repeated here.

[0081] The above description is only a preferred embodiment of the present embodiment and is not intended to limit the present embodiment. For those skilled in the art, the present embodiment may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present embodiment shall be included in the protection scope of the present embodiment.

Claims

1. An intelligent nursing bed control system based on multimodal information, characterized in that: include: Information collection module, used to monitor the user's physiological signals in real time; The physiological signals at least include brain wave signals, eye movement signals, and body position and pressure signals; A fusion control module, used to fuse the acquired physiological signals to obtain multimodal information, analyze the obtained multimodal information, and determine the user's physiological state and intention; The drive execution module is used to drive the electric push rod of the intelligent care bed and adjust the user's posture according to the user's physiological state and intention.

2. The intelligent nursing bed control system based on multimodal information as claimed in claim 1, characterized in that: The information acquisition module includes an electroencephalogram measuring device for real-time monitoring of the user's brain electrical signals, an eye tracking device for real-time capturing the user's eye movement signals, and a plurality of pressure sensors placed in the mattress for real-time monitoring of the user's body position and pressure signals.

3. The intelligent nursing bed control system based on multimodal information as described in claim 2, characterized in that: The user's Steady-State Visual Evoked Potentials (SSVEP) signal is obtained through the electroencephalogram measurement device; the fusion control module sequentially performs preprocessing and feature extraction operations on the obtained Steady-State Visual Evoked Potentials model to obtain a feature vector, and uses a classification recognition algorithm to convert the obtained feature vector into the user's intention.

4. The intelligent nursing bed control system based on multimodal information as described in claim 2, characterized in that: Based on the eye movement information of the user captured by the eye tracking device, the eye movement information is sent to the fusion control module when and only when the acquired eye movement information meets the gaze condition, otherwise the signal continues to be captured.

5. The intelligent nursing bed control system based on multimodal information as described in claim 2, characterized in that: The pressure sensor is placed inside the mattress and is used to monitor the pressure distribution between the user's body surface and the bed surface in real time to obtain the user's body position and pressure signal; the fusion control module receives and processes the obtained user's body position and pressure signal to obtain a user's body pressure distribution map, and displays the obtained user's body pressure distribution map on the display screen of the fusion control module.

6. The intelligent nursing bed control system based on multimodal information as claimed in claim 1, characterized in that: The drive execution module includes at least a back electric drive unit, a torso electric drive unit and a leg electric drive unit. The fusion control module sends a drive instruction to the drive execution module based on the obtained user's physiological state and intention, controls and drives the electric push rods of different electric drive units, and adjusts the user's posture.

7. A method for controlling an intelligent nursing bed based on multimodal information, using the intelligent nursing bed control system based on multimodal information as claimed in any one of claims 1 to 6, characterized in that: include: Acquire real-time physiological signals of the user including at least EEG signals, eye movement signals, and body position and pressure signals; Integrate and analyze the real-time physiological signals of the user to determine the user's physiological state and intention; According to the user's physiological state and intention, the electric push rod of the smart care bed is driven to adjust the user's posture.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, the steps of the intelligent care bed control method based on multimodal information as described in claim 7 are implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of the intelligent care bed control method based on multimodal information as described in claim 7 are implemented.

10. A computer program product comprising software code, characterized in that The program in the software code executes the steps of the intelligent care bed control method based on multimodal information as described in claim 7.

Images