Visual man-machine interaction system

Abstract

The utility model provides a visual man-machine interaction system, which is used for monitoring neuromuscular functions and sleep states of a human body after a cervical vertebra operation, and comprises a plurality of electromyographic signal collectors, a plurality of electromyographic signal acquisition units, a plurality of electromyographic signal processing units and a plurality of human body monitoring units, a camera for capturing three-dimensional (3D) image data of a specific activity performed by a human body; a plurality of inertial sensors, wherein each inertial sensor is fixed at a specific position of a human body; the interaction terminal comprises a processing module and an interaction module which are in communication connection, the processing module is used for analyzing the captured data to obtain a monitoring analysis result, and the interaction module is used for visually displaying the monitoring analysis result. Various sensors are conveniently connected with the interactive terminal, so that the interactive terminal can monitor 3D videos and real-time actions of a nerve regulation patient in real time, evaluate nerve and muscle functions and visually display early warning intervention measures and personalized suggestions.

Description

Technical Field

[0001] The utility model relates to a visualized human-computer interaction system, in particular to a visualized human-computer interaction system for monitoring neuromuscular function and sleep state after cervical vertebrae surgery. Background Art

[0002] After cervical spine surgery, the assessment of patients' neurological and muscle function is a key part of the rehabilitation process. Existing assessment methods, such as face-to-face clinical examinations, although effective in some cases, are difficult to implement in resource-limited or remote areas. In addition, these methods usually lack the ability to continuously monitor the patient's functional recovery in daily life, and it is difficult for patients to obtain immediate feedback on their physical status and posture.

[0003] Currently, the field of neuromodulation lacks a comprehensive remote monitoring system that is easy for patients to install, can simultaneously monitor the patient's electromyographic signal analysis and sleep status, and provide immediate warnings and / or visualize the patient's condition and guidance. Most of the related technologies are used for preoperative and intraoperative navigation, and do not involve the postoperative rehabilitation process. Utility Model Content

[0004] The purpose of the utility model is to overcome the deficiencies of the prior art and provide a visualized human-computer interaction system for monitoring the neuromuscular function and sleep state of a human body after cervical vertebrae surgery, characterized in that the visualized human-computer interaction system comprises: a plurality of electromyographic signal collectors, each of which is placed on a specific muscle group of the human body to capture the electromyographic signals generated on the specific muscle group; a camera, which is used to capture three-dimensional (3D) image data of a human body performing a specific activity; a plurality of inertial sensors, each of which is fixed at a specific position of the human body to capture data reflecting the relative position and motion information of various parts of the human body; and an interactive terminal, comprising a processing module and an interactive module connected in communication, the processing module being used to analyze the data captured by the plurality of electromyographic signal collectors, the camera and the plurality of inertial sensors to obtain monitoring and analysis results, and the interactive module being used to visually display the monitoring and analysis results.

[0005] The beneficial effects of the utility model are: integrating various types of easy-to-install sensors, conveniently connecting to the processing module of the interactive terminal, so that the interactive terminal can instantly monitor the 3D video and real-time movements of the neuromodulation patient, evaluate nerve and muscle function based on various sensor signals, and visually display early warning intervention measures and personalized suggestions according to the patient's condition.

[0006] Optionally, the interactive terminal further includes a communication module for communicating with multiple electromyographic signal collectors, cameras, and multiple inertial sensors to obtain captured data. Optionally, the communication module is also used to transmit instructions from the processing module to the multiple electromyographic signal collectors, cameras, and multiple inertial sensors. The communication module can be a wireless communication module.

[0007] The beneficial effect of adopting the above further solution is: further facilitating the deployment of various types of sensors in the system, with strong security and high reliability.

[0008] Optionally, the interactive terminal is used to control the operation of multiple electromyographic signal collectors, cameras and multiple inertial sensors. The communication module is also used to transmit instructions from the processing module to the multiple electromyographic signal collectors, cameras and multiple inertial sensors.

[0009] The beneficial effect of adopting the above further solution is: facilitating the adjustment and flexible deployment of the system.

[0010] Optionally, the processing module in the interactive terminal further includes a memory, which stores a mapping table of various types of emergency warning information and corresponding warning commands.

[0011] The beneficial effect of adopting the above further solution is: by using the pre-stored mapping table, the triggering of the early warning command is further accelerated.

[0012] Optionally, the visual human-computer interaction system further includes a display, a speech synthesis circuit, a sound amplifier and an audio device which are communicatively connected to the interaction module, the speech synthesis circuit is connected to the processing module and the sound amplifier, and the sound amplifier is connected to the audio device.

[0013] The beneficial effect of adopting the above further plan is to further improve the convenience for patients to receive early warning intervention and personalized advice information.

[0014] Optionally, the visual human-computer interaction system is communicatively connected to a server to send monitoring and analysis results and various types of emergency warning information received to the server for storage.

[0015] The beneficial effect of adopting the above further solution is: improving the reliability of the system.

[0016] Optionally, the visual human-computer interaction system includes multiple interactive terminals, and multiple electromyographic signal collectors, the camera, the multiple inertial sensors and the multiple interactive terminals form a wireless sensor network.

[0017] The beneficial effect of adopting the above further solution is: improving the reliability of the system.

[0018] Optionally, the interactive terminal further includes a power module, which is electrically connected to the processing module, the interactive module and the communication module to supply power to the processing module, the interactive module and the communication module. The power module may be a battery.

[0019] The beneficial effect of adopting the above further solution is: further improving the convenience of power supply for each part of the equipment and reducing the cost.

[0020] Optionally, the visual human-computer interaction system is communicatively connected to the doctor's communication terminal to send the monitoring and analysis results and various types of emergency warning information received to the doctor's communication terminal.

[0021] The beneficial effect of adopting the above further solution is: connecting the system with the doctor's communication terminal makes it easier for patients to obtain the doctor's guidance and for doctors to understand the patient's condition. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 The block diagram of the visual human-computer interaction system of the embodiment of the utility model is shown in FIG.

[0023] Figure 2 The block diagram is a communication connection between the visual human-computer interaction system and related devices in an embodiment of the utility model. DETAILED DESCRIPTION

[0024] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the drawings are in a very simplified form and are not in exact proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In addition, the structures shown in the drawings are often part of the actual structure.

[0025] Combination Figure 1 and Figure 2 The visual human-computer interaction system 201 of the embodiment of the utility model includes multiple electromyographic signal collectors 110, cameras 137, multiple inertial sensors 116, an interactive terminal 118, a display 125, a speech synthesis circuit 126, a sound amplifier 127 and an audio device 128, etc.

[0026] The interactive terminal 118 integrates multiple modules to realize data processing, display and communication functions. The interactive terminal includes an interactive module 124, a processing module 120, a communication module 129, a power module 132, etc.

[0027] The processing module 120 of the interactive terminal 118 is composed of a microprocessor, a memory and an interface circuit. The microprocessor executes data analysis algorithms / applications known in the industry, such as signal processing, feature extraction and pattern recognition. The memory 121 is used to store data in the processing process, early warning information mapping tables, etc. The interface circuit realizes communication with other modules. The memory 122 may further include volatile storage (DRAM) and non-volatile storage (FLASH). DRAM is used to store runtime data, while FLASH stores firmware, known application code and early warning information mapping tables. These memories are directly connected to the microprocessor via a data bus.

[0028] The interactive module 124 of the interactive terminal 118 is connected to the display 125, the speech synthesis circuit 126, the sound amplifier 127 and the audio device 128. The display 125 can be directly connected to the processing module for the output of graphics and text. The speech synthesis circuit 126 converts the text information into speech signals, the sound amplifier 127 enhances these signals, and finally outputs them through the audio device 128. These components are connected to the interactive module through a standard interface. Specifically, the display 125 can display the monitoring and analysis results according to the instructions. It is connected to the interactive module through a digital video interface, receives graphic data and displays it to the user. The speech synthesis circuit 126 can be implemented by a digital signal processor (DSP), which receives text information and generates corresponding speech signals through algorithms / applications known in the industry. The sound amplifier 127 is an analog circuit that receives the output of the DSP and enhances the signal strength. The audio device 128 converts the enhanced signal into sound. This process is connected through an audio interface (such as a 3.5mm audio jack or Bluetooth).

[0029] The power module 132 of the interactive terminal 118 includes a power management chip, a battery, a charging circuit, and a power conversion circuit. It is responsible for converting the external power into various voltage levels required by the system and supplying power to each module through a power distribution network. The power management chip monitors and controls power usage to optimize energy consumption and extend battery life.

[0030] The communication module 129 of the interactive terminal 118 exchanges data with data acquisition equipment (multiple electromyographic signal collectors 110, cameras 137, multiple inertial sensors 116, etc.) and optional external systems (for example, server 202 and doctor communication terminal 203) through a wireless communication interface. The wireless communication interface allows the system to be flexibly deployed. The components of the visual human-computer interaction system are interconnected and collaborated through various wireless communication interfaces and communication protocols to form an efficient and reliable system to achieve comprehensive monitoring and management of patients after cervical spine surgery. Through the communication connection between the interactive terminal 118 and the above-mentioned various data acquisition devices, a wireless sensor network (Wireless Sensor Networks, WSN) is essentially formed. In some embodiments, the visual human-computer interaction system may include multiple interactive terminals 118, so that some of the interactive terminals act as relay nodes, while other interactive terminals act as ordinary nodes. In this way, when there may be a communication failure between the ordinary interactive terminal and each data acquisition device, the communication between the above-mentioned ordinary interactive terminal and the data acquisition device is ensured by the relay node.

[0031] The interactive terminal 118 can be connected to the doctor communication terminal 203 via a wireless network via the communication module 129. The doctor communication terminal 203 runs an application known in the industry to receive, display and analyze data sent from the interactive terminal 118, allowing doctors to remotely monitor the patient's condition and respond.

[0032] All components of the interactive terminal 118 are powered by the power module 132 to ensure continuous data monitoring and processing capabilities. Solutions for the power module 132 include but are not limited to battery, charger or cable power supply to adapt to different usage scenarios and needs.

[0033] The visualized human-computer interaction system 201 includes a plurality of myoelectric signal collectors 110, each of which is used to be placed on a specific muscle group of the human body (e.g., neck, shoulder and upper limb muscles), and the electrical signals generated during muscle activity are captured by placing electrodes on the specific muscle groups. These signals can reflect the activity state of the muscles and the functional status of the nervous system. Each myoelectric signal collector 110 is composed of a plurality of key components, including electrodes, signal amplifiers, filters and wireless modules for signal transmission. The electrodes are directly attached to the surface of the skin and are responsible for capturing the tiny electrical signals generated during muscle activity. These electrical signals are then amplified by the signal amplifier and any possible noise interference is removed by the filter to ensure the clarity and accuracy of the signal. The filter can use a known digital filter, such as a Butterworth filter or a Chebyshev filter. Finally, the processed myoelectric signal is transmitted to the processing module 120 of the interactive terminal 118 through the communication module 129 for further analysis and interpretation. This wireless connection method not only reduces the patient's discomfort and the complexity of use, but also facilitates the integration and analysis of real-time data. The processing module 120 in the interactive terminal 118 of the visual human-computer interaction system can use the algorithms / applications such as spectrum analysis and pattern recognition known in the industry to conduct in-depth analysis of the captured electromyographic signals, and apply known machine learning models such as neural networks, decision trees, etc. to perform multi-category classification to identify abnormal muscle activity. Through this highly integrated approach, accurate muscle and nerve function assessment can be provided, which is of great significance for early diagnosis of diseases and monitoring of the rehabilitation process.

[0034] The visual human-computer interaction system includes multiple cameras 137. The data acquired by the camera 137 can be transmitted to the processing module 120 in real time through a communication module (such as Bluetooth or Wi-Fi). The image data captured by the camera 137 is processed by the processing module 120 in the visual human-computer interaction system in combination with known human body detection, joint recognition, limb tracking and other technologies, which can accurately analyze and evaluate the patient's motor function, generate a 3D skeleton representation and estimate the mass and inertia torque of the limbs, thereby providing a personalized rehabilitation plan for the patient. In a further embodiment, all or part of the above processing work is handed over to the server 202 that is connected to the visual human-computer interaction system 201 for communication.

[0035] The visual human-computer interaction system includes a plurality of inertial sensors 116 for collecting muscle activity and body movement data in real time. These sensors 116 can be fixed at specific locations to capture subtle body movement and posture change data, thereby achieving high-precision monitoring of the patient's status (especially sleep status). The plurality of inertial sensors 116 are fixed by adjustable belts to adapt to the body sizes of different patients and provide a comfortable wearing experience. The internal structure of each inertial sensor 116 includes a plurality of components, such as accelerometers and gyroscopes, for measuring and recording body movement and orientation changes. The acquired data can be transmitted to the processing module 120 in real time via a communication module (such as Bluetooth or Wi-Fi). These components are interconnected through electronic circuits and work in conjunction with the processing module 120 in the interactive terminal of the visual human-computer interaction system to process and analyze data in real time using algorithms / applications known in the industry. The processing module 120 in the interactive terminal of the visual human-computer interaction system can also send the collected data to the server 202 in real time through the communication module 129, and the server 202 uses the deep learning and artificial intelligence algorithms / applications known in the industry to further analyze the received data stream to achieve real-time recognition and monitoring of the patient's sleep stage. The design of the multiple inertial sensors 116 fully considers the efficient connection relationship between components and seamless signal transmission with other components of the system, ensuring high efficiency and accuracy of data processing.

[0036] The processing module 120 of the interactive terminal 118 is responsible for receiving and analyzing the signals from the data acquisition terminal. The processed data is stored in the memory 121, and provided to the patient through the speech synthesis circuit 126, the sound amplifier 127 and the audio device 128 to provide auditory feedback, such as sleep improvement suggestions or warning information.

[0037] The visual human-computer interaction system may include a display 125, which provides the patient with intuitive data display and interactive functions. The patient can view his or her sleep analysis results, including information such as sleep stages, abnormal sleep patterns, etc., and receive a personalized sleep improvement plan.

[0038] Through the organic combination and collaborative work of the above components, this embodiment is easy for users to install and use, and brings significant benefits to the sleep quality and rehabilitation process of patients after cervical spine surgery. The system can not only automatically monitor and analyze sleep status, but also provide timely warning feedback and intervention measures through the interactive terminal 118, supporting patients and doctors to formulate and implement personalized sleep improvement plans.

[0039] The visual human-computer interaction system 201 can be connected to the doctor communication terminal 203, allowing the doctor to remotely access the patient's sleep data and analysis results. The doctor can provide professional medical advice to the patient through this terminal 203 to help adjust the sleep intervention plan.

[0040] The visual human-computer interaction system 201 can be connected to the server 202 , allowing the processing work known in the industry to be handed over to the server 202 , and key data can also be backed up on the server 202 .

[0041] The working principle of the utility model is as follows:

[0042] After the user installs multiple electromyographic signal collectors, cameras, and multiple inertial sensors according to the instructions, the multiple electromyographic signal collectors, cameras, and multiple inertial sensors are started, and the interactive terminal is started at the same time, and the power module of the interactive terminal begins to power various parts of the interactive terminal. Multiple electromyographic signal collectors, cameras, and multiple inertial sensors automatically establish communication connections with the communication module of the interactive terminal, and the communication module of the interactive terminal also establishes connections with the server and the doctor's communication terminal. After that, multiple electromyographic signal collectors, cameras, and multiple inertial sensors transmit signals to the interactive terminal in a time sequence. After the processing module of the interactive terminal performs necessary processing using known algorithms / applications, information related to the user's physical condition is displayed on the display, and when the warning information is triggered, the voice synthesis circuit, sound amplifier, and audio equipment provide auditory feedback to the patient, such as sleep improvement suggestions or warning information.

[0043] The above description is only a description of the preferred embodiment of the utility model, and is not any limitation on the scope of the utility model. Any changes and modifications made by ordinary technicians in the field of the utility model based on the above disclosure shall fall within the scope of protection of the claims.

Claims

1. A visual human-computer interaction system for monitoring the neuromuscular function and sleep state of the human body after cervical spine surgery, characterized in that: The visual human-computer interaction system comprises: A plurality of myoelectric signal collectors, each of which is placed on a specific muscle group of the human body to capture the myoelectric signals generated by the specific muscle group; A camera for capturing three-dimensional (3D) image data of a human body performing a specific activity; A plurality of inertial sensors, each of which is fixed at a specific position of the human body to capture data reflecting relative position and movement information of various parts of the human body; and The interactive terminal includes a processing module and an interactive module that are communicatively connected, wherein the processing module is used to analyze the data captured by the multiple electromyographic signal collectors, the cameras, and the multiple inertial sensors to obtain monitoring and analysis results, and the interactive module is used to visually display the monitoring and analysis results.

2. The visual human-computer interaction system according to claim 1, characterized in that: The interactive terminal further includes a communication module, which is used to communicate with multiple electromyographic signal collectors, the cameras and the multiple inertial sensors to obtain captured data.

3. The visual human-computer interaction system according to claim 2, characterized in that: The interactive terminal is used to control the operation of the multiple electromyographic signal collectors, the cameras and the multiple inertial sensors.

4. The visual human-computer interaction system according to claim 2, characterized in that: The processing module includes a memory, and the memory stores a mapping table of various types of emergency warning information and corresponding warning commands.

5. The visual human-computer interaction system according to claim 4, characterized in that: The visual human-computer interaction system further includes a display, a speech synthesis circuit, a sound amplifier and an audio device which are communicatively connected to the interaction module, the speech synthesis circuit is connected to the processing module and the sound amplifier, and the sound amplifier is connected to the audio device.

6. The visual human-computer interaction system according to claim 2, characterized in that: The visual human-computer interaction system is connected to the server for communication, and is used to send the monitoring and analysis results and various types of emergency warning information received to the server for storage.

7. The visual human-computer interaction system according to claim 2, characterized in that: The visual human-computer interaction system comprises a plurality of interaction terminals, and the plurality of myoelectric signal collectors, the cameras, the plurality of inertial sensors and the plurality of interaction terminals form a wireless sensor network.

8. The visual human-computer interaction system according to claim 3, characterized in that: The communication module is also used to transmit the instructions of the processing module to the multiple electromyographic signal collectors, the cameras and the multiple inertial sensors.

9. The visual human-computer interaction system according to claim 2, characterized in that: The interactive terminal further includes a power module, which is electrically connected to the processing module, the interactive module and the communication module to supply power to the processing module, the interactive module and the communication module.

10. The visual human-computer interaction system according to claim 5, characterized in that: The visual human-computer interaction system is communicatively connected to the doctor communication terminal, and is used to send the monitoring and analysis results and various types of emergency warning information received to the doctor communication terminal.

Images