Life auxiliary system based on artificial intelligence

Abstract

The invention discloses a life assistance system based on artificial intelligence, belongs to the technical field of artificial intelligence, integrates five modules of vital sign monitoring and hearing assistance into a single device through modular integration and AI cooperative control, reduces the weight by 60% compared with a traditional scheme, and solves the problems of function fragmentation and cumbersome carrying. The response speed is improved through voice / gesture interaction, the multi-language translation accuracy is high, convenience is provided for people, and meanwhile convenience conditions are provided for old people and disabled people. The emergency response bidirectional authentication mechanism reduces the false alarm rate, and the cross-scene adaptability is high. Module cooperation and an open interface realize multi-device linkage, and the system can be in butt joint with a hospital monitoring system or a smart home. The system realizes environment identification and state evaluation through multi-modal analysis, provides intelligent and efficient life assistance for aged people and handicapped people, and improves the use frequency and convenience.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to an AI-based life assistance system that integrates vital sign monitoring, sensory function compensation, motor assistance, and multilingual interaction. Background Technology

[0002] Traditional assistive devices for daily living are unable to meet the complex needs of users.

[0003] Firstly, the low level of functional integration makes them inconvenient to use. Most existing assistive devices adopt a single-scenario design. For example, medical heart rate monitors can only collect physiological indicators, traditional hearing aids only provide sound amplification, and assistive glasses for the visually impaired only have optical correction. This fragmented design requires users to carry multiple devices at the same time, resulting in significant issues of weight accumulation and cumbersome operation, thereby reducing the frequency of use or abandoning the use of assistive functions. Secondly, the interaction methods have limitations. For example, traditional button and touch screen operation modes are not user-friendly for the elderly and people with physical disabilities, have high learning costs, and are difficult to operate. Moreover, it is difficult to achieve smooth interaction when facing users or scenarios with different languages, and the functionality is limited. In summary, existing technologies have shortcomings in terms of functional integration, cross-module collaboration, and interactive experience. There is an urgent need for an AI-based life assistance system that integrates vital sign monitoring, sensory function compensation, motor assistance, and multilingual interaction to meet the diverse needs of an aging society and people with disabilities. Summary of the Invention

[0004] The purpose of this invention is to provide an artificial intelligence-based life assistance system to overcome the above-mentioned technical problems. Through modular integration and AI-assisted control, it achieves a breakthrough in assistive devices from single-function to intelligent composite.

[0005] The present invention solves the above-mentioned technical problems through the following technical solution: Provides an artificial intelligence-based life assistance system, including: The life monitoring module monitors heart rate, respiratory rate, body temperature and weight to draw a data graph of the human body's normal state, and determines whether to activate emergency measures through two-way authentication of heart rate and respiratory rate.

[0006] The auditory assistance module includes an AI hub, a loudspeaker, bionic conductive headphones, and a camera. The camera, combined with the SLAM algorithm, constructs a 3D map, and the AI ​​hub generates obstacle avoidance paths based on the Dijkstra algorithm.

[0007] A visual compensation module compensates visual defects in combination with a CNN image restoration algorithm, and also implements real-time translation of five kinds of language texts based on a Transformer architecture NMT model and converts target speech through TTS speech synthesis technology.

[0008] A motion control module includes an electromyographic signal acquisition module that captures muscle electrical signals, maps them into mechanical prosthesis control instructions using an RNN model, supports Bluetooth protocol docking of external devices, and realizes device action preview based on VR simulation.

[0009] An interaction hub module realizes gesture control and voice control through an artificial intelligence auxiliary system, a camera and a microphone, realizes instruction recognition in a noisy environment through a DeepSpeech speech recognition model, and analyzes 18 kinds of gesture semantics using an OpenPose algorithm.

[0010] Further, the life monitoring module includes a PPG heart rate sensor, a thermal imaging body temperature sensor, a respiratory frequency piezoelectric sensor, and a pressure type body weight sensor; the life monitoring module is electrically connected to the AI hub; the emergency response mechanism of the life monitoring module sends a distress signal to external devices through an MQTT protocol when physiological abnormalities are detected.

[0011] Further, the AI hub is built-in with an NVIDIA Jetson AGX Orin chip.

[0012] Further, the obstacle avoidance path planning result is synchronized to the motion control module through a RESTAPI interface to drive the mechanical prosthesis or exoskeleton to perform obstacle avoidance actions.

[0013] Further, the multilingual interaction process of the visual compensation module is that the camera performs OCR text recognition through a Tesseract engine, then a Transformer model translates, and then TTS speech synthesis is performed to transmit to a bionic conduction earphone for playing.

[0014] Further, the external device connected to the motion control module is one or more of a mechanical prosthesis, an exoskeleton, a drone, and a car.

[0015] Further, the artificial intelligence-based life assistance system inputs visual images, auditory signals and physiological indicators after Kalman filter denoising into the AI hub for multi-modal analysis to realize environment recognition and user state evaluation.

[0016] Further, the hardware of each module of the artificial intelligence-based life assistance system is distributed as follows: the hardware of the system is a virtual helmet, which includes a life monitoring sensor array integrated on both sides of the neck, an electromyographic signal acquisition module arranged on the neck, a bionic conduction earphone arranged on the lower side of the ear, two loudspeakers arranged on both sides of the respirator, a Bluetooth communication module integrated inside, four cameras arranged on the outside, a protective lens arranged in front of the eyes, and a flexible screen arranged on the inner side of the protective lens. The outside is the protective lens, and the inside is the double-sided flexible screen. The protective lens and the double-sided flexible screen are two independent individuals.

[0017] Further, the system supports third-party device access, and opens motion control instruction interfaces and physiological data interfaces through RESTAPI, so that hospital monitoring systems or smart home devices can be linked.

[0018] Further, the system is equipped with a 5000mAh lithium polymer battery and supports Qi protocol wireless fast charging. Beneficial effects

[0019] The present application solves the problems of functional fragmentation and inconvenient interaction of traditional devices through modular integration and AI collaborative control. In terms of functional integration, a single device integrates five modules such as vital sign monitoring and hearing assistance, which reduces the weight of the device by 60% compared to traditional solutions, reduces the cumbersome of carrying multiple devices for users, and improves the use frequency.

[0020] Through the interaction module of the present application, the response speed is improved through voice / gesture, and the multi-language translation accuracy is high, effectively solving the operation difficulty problem of the elderly and the disabled, and expanding the range of application population.

[0021] An emergency response two-way authentication mechanism is adopted to further reduce the false positive rate, and the cross-scene adaptability is strong. In addition, the system realizes multi-device linkage through module collaboration and open interface, meets diversified assistance needs, and provides more intelligent and efficient convenient life collaborative assistance functions for the aging population and the disabled. DETAILED DESCRIPTION

[0022] The present application provides an artificial intelligence-based life assistance system, which comprises: The life monitoring module draws a human normal state data graph through the indexes of heart rate, respiratory rate, body temperature and weight, and inputs the weight parameter to facilitate the measurement of the human normal state data graph, and judges whether to start emergency measures through two-way authentication of heart rate and respiratory rate.

[0023] The hearing assistance module includes an AI hub, a loudspeaker, a bionic conduction earphone and a camera, constructs a three-dimensional map through the camera combined with a SLAM algorithm, and generates an obstacle avoidance path based on a Dijkstra algorithm.

[0024] The visual compensation module compensates for visual defects in combination with a CNN image restoration algorithm, and also realizes real-time translation of five kinds of language text based on a Transformer architecture NMT model, and converts into target speech through TTS speech synthesis technology.

[0025] The motion control module includes an electromyographic signal acquisition module, which captures muscle electrical signals, maps them into mechanical prosthesis control instructions using an RNN model, supports Bluetooth protocol interfacing with external devices, and realizes device action preview based on VR simulation.

[0026] The interaction hub module realizes gesture control and voice control through an artificial intelligence auxiliary system, a camera and a microphone, realizes instruction recognition in a noisy environment through a DeepSpeech speech recognition model, and analyzes 18 kinds of gesture semantics using an OpenPose algorithm.

[0027] Specifically, the life monitoring module includes a PPG heart rate sensor, a thermal imaging body temperature sensor, a respiratory frequency piezoelectric sensor, and a pressure type body weight sensor; the life monitoring module is electrically connected to the AI hub; the emergency response mechanism of the life monitoring module sends a distress signal to external devices through the MQTT protocol when physiological abnormalities are detected.

[0028] Specifically, the AI hub is built-in with an NVIDIA Jetson AGX Orin chip.

[0029] Specifically, the obstacle avoidance path planning result is synchronized to the motion control module through the RESTAPI interface to drive the mechanical prosthesis or exoskeleton to perform obstacle avoidance actions.

[0030] Specifically, the multilingual interaction process of the visual compensation module is as follows: the camera performs OCR text recognition through the Tesseract engine, then the Transformer model translates, and finally the TTS speech synthesis is transmitted to the bionic conduction earphones for playback.

[0031] Specifically, the external device interfaced by the motion control module is one or more of a mechanical prosthesis, an exoskeleton, a drone, and a car.

[0032] Specifically, the life assistance system based on artificial intelligence inputs visual images, auditory signals, and physiological indicators after Kalman filter denoising into the AI hub for multi-modal analysis, realizing environment recognition and user state assessment.

[0033] Specifically, the hardware integration distribution of each module of the artificial intelligence-based life assistance system is as follows: the hardware of the system is a virtual helmet, the virtual helmet includes a life monitoring sensor array integrated on the left and right sides of the neck, an electromyographic signal acquisition module arranged on the neck, a bionic conduction earphone arranged on the lower side of the ear, two loudspeakers arranged on the two sides of the respirator, an internally integrated Bluetooth communication module, four cameras arranged on the outside, a protective lens arranged in front of the eyes, a double-sided flexible screen arranged on the inner side of the protective lens, the protective lens on the outside, the double-sided flexible screen on the inner side, the protective lens and the double-sided flexible screen being two independent individuals.

[0034] Specifically, the system supports third-party device access, and opens a motion control instruction interface and a physiological data interface through a REST API, so that a hospital monitoring system or a smart home device can be linked.

[0035] Specifically, the system is equipped with a 5000mAh lithium polymer battery and supports Qi protocol wireless fast charging.

[0036] 1. Hardware integration distribution The patch type sensor array is as follows: PPG heart rate sensor (model: Maxim MAX30102, sampling rate 100Hz); Thermal imaging body temperature sensor (model: FLIR Lepton 3.5, accuracy ±0.5℃); Piezoelectric respiration frequency sensor (sampling rate 20Hz); Miniature pressure body weight sensor (accuracy ±1kg).

[0037] Bionic conduction earphone integrated on the lower side of the ear (model: AfterShokz Aeropex, noise reduction depth 30dB); Two loudspeakers (frequency response 20Hz-20kHz); Four cameras (fish-eye lens + standard lens combination, fish-eye field of view angle 180°); Protective lens and ergonomically designed double-sided flexible screen arranged on the inner side; Electromyographic signal acquisition module (sampling rate 1000Hz); Bluetooth module (model: Nordic nRF52840).

[0038] The AI central processor is built-in NVIDIA Jetson AGX Orin chip with 16G memory; Front microphone; 5000mAh lithium polymer battery, supporting 15W Qi wireless fast charging.

[0039] 2. Software algorithm and process implementation Life monitoring module processing flow: Sensors acquire data in real time → Kalman filter noise reduction (noise standard deviation reduced by 80%); An individual physiological baseline model was constructed based on an LSTM neural network (3 layers, 128 neurons per layer). The training set contained 24-hour data of 200 users of different ages and genders. Anomaly detection: When HRV < 50ms and respiratory rate < 8 breaths / minute, an emergency response is triggered with a response time of < 1.5 seconds.

[0040] Emergency measures: The voice alarm says, "An abnormal heart rate has been detected. Emergency services have been contacted for you." At the same time, a distress signal containing physiological data is sent to the preset emergency contacts via the MQTT protocol.

[0041] Obstacle avoidance path planning: The camera captures environmental images → the SLAM algorithm (ORB-SLAM3) constructs a 3D map at a speed of 15 frames per second; The Dijkstra algorithm generates obstacle avoidance paths, which are then smoothed and synchronized to the motion control module. Mechanical prosthetic control: Electromyographic signals are mapped into joint movement commands through an RNN model, such as the "elbow flexion" command, with a recognition accuracy of 92%.

[0042] Multilingual interaction process: Visual compensation module OCR recognition (Tesseract engine, supports 120 fonts) → Transformer translation model (8-layer encoder + 8-layer decoder, vocabulary size 32000) → TTS synthesis (timbre similarity 4.5 / 5) → Bionic conductive headphones playback, with a total latency of 480ms.

[0043] 3. Typical work scenarios Scenario 1: An elderly person experiences a sudden abnormal heart rate outdoors Life monitoring module: When the sensor detects a sudden drop in heart rate to 42 beats / minute, heart rate variability (HRV) below 45ms, and respiratory rate less than 10 breaths / minute, the two-way authentication mechanism is triggered. The alarm was then played through bionic conductive headphones, and a distress signal with real-time heart rate and respiratory rate curves, along with GPS location, was sent to the family doctor platform via the MQTT protocol.

[0044] Scenario 2: Obstacle Avoidance Path Planning for Visually Impaired Individuals The system achieves obstacle avoidance navigation through multi-sensor fusion and AI algorithms. On the hardware side, the environment is collected in real time by cameras, and then a 3D map is built using the SLAM algorithm. The AI ​​hub generates obstacle avoidance paths based on the Dijkstra algorithm, and the mechanical prosthesis receives control commands via Bluetooth.

[0045] The algorithm flow is: the camera image is identified by the obstacle, ORB-SLAM3 constructs a three-dimensional map, and Dijkstra algorithm generates an initial path. When a dynamic obstacle is encountered, a buffer for obstacle avoidance is reserved by the inflation coefficient 1.2 of the grid map.

[0046] The visually impaired person receives voice prompts such as "2 meters ahead of the stairs" through the bionic conduction earphone; the physically disabled person can preset the "clenching gesture to detour" through the electromyographic signal acquisition module.

[0047] As in a typical scenario, when a visually impaired user walks in a mall, the system detects a promotion booth and generates a right detour path, dynamically adjusts the route when a pedestrian approaches, and synchronously guides through voice.

[0048] Scenario 3: Use demand in cross-language scenarios When the user is traveling abroad and ordering food in a restaurant, the head-mounted camera performs OCR text recognition on the menu through the Tesseract engine, transmits the recognized text information to the Transformer architecture NMT model of the visual compensation module, which quickly translates the menu content into a language familiar to the user, and the translation result is converted into voice through TTS speech synthesis technology and played to the user through the bionic conduction earphone on the shoulder, helping them understand the menu content.

[0049] When the user is lost on the street and asks a passerby for directions, the microphone collects the passerby's voice, and the DeepSpeech speech recognition model realizes instruction recognition in a noisy environment, and after translation by the Transformer model, it is fed back to the user in the form of voice and text. In addition, if the user needs gesture assistance when communicating with others, the OpenPose algorithm analyzes 18 gesture semantics, combined with voice translation, to achieve more efficient cross-language communication and fully meet the user's interactive needs in a multilingual environment.

[0050] The above is only a preferred specific embodiment of the present application; however, the protection scope of the present application is not limited thereto. Any skilled person in the art can make equivalent substitutions or changes within the technical scope disclosed by the present application according to the technical solution and improvement concept of the present application, which should be covered within the protection scope of the present application.

Claims

1. An artificial intelligence-based life assistance system, comprising: a life monitoring module that draws a human normal state data graph through indicators of heart rate, respiratory rate, body temperature, and body weight, and determines whether to initiate emergency measures through two-way authentication of heart rate and respiratory rate; an auditory assistance module that includes an AI hub, a loudspeaker, a bionic conduction earphone, and a camera, constructs a three-dimensional map through the camera combined with a SLAM algorithm, and generates an obstacle avoidance path based on a Dijkstra algorithm; a visual compensation module that compensates for visual defects combined with a CNN image repair algorithm, realizes real-time translation of no less than five languages and texts based on a Transformer architecture NMT model, and converts into target voice and text translation through TTS speech synthesis technology; a motion control module that includes an electromyographic signal acquisition module, captures muscle electrical signals through the electromyographic signal acquisition module, maps mechanical prosthesis control instructions using an RNN model, supports Bluetooth protocol docking of external devices, and realizes device action preview and simulation control based on VR simulation; an interaction hub module that realizes gesture control and voice control through an artificial intelligence assistance system, a camera, and a microphone, realizes instruction recognition in a noisy environment through a DeepSpeech speech recognition model, and analyzes no less than 18 gesture semantics using an OpenPose algorithm.

2. The life assistance system based on artificial intelligence according to claim 1, characterized in that, The life monitoring module includes a PPG heart rate sensor, a thermal imaging body temperature sensor, a respiratory rate piezoelectric sensor, and a pressure type body weight sensor; the life monitoring module is electrically connected to the AI hub; the emergency response mechanism of the life monitoring module sends a distress signal to external devices through an MQTT protocol when physiological abnormalities are detected. 3.The life assistant system based on artificial intelligence according to claim 1, characterized in that, The AI hub is built-in with an NVIDIA Jetson AGX Orin chip. 4.The life assistant system based on artificial intelligence according to claim 1, wherein, The obstacle avoidance path planning result is synchronized to the motion control module through a RESTAPI interface, driving the mechanical prosthesis or exoskeleton to execute the obstacle avoidance scheme and action. 5.The life assistant system based on artificial intelligence according to claim 1, wherein, The language interaction process of the visual compensation module is that the camera performs OCR text recognition through a Tesseract engine, then a Transformer model translates, and then TTS speech synthesis is performed, and the result is transmitted to the bionic conduction earphone for playing. 6.The life assistant system based on artificial intelligence according to claim 1, wherein, The motion control module docks external devices through a wireless transmission mode, which is one of WiFi, Bluetooth, and 5G signal transmission modes; the external devices are one or more of a mechanical prosthesis, an exoskeleton, a drone, and a car. 7.The life assistant system based on artificial intelligence according to claim 1, wherein, The artificial intelligence-based life assistance system inputs visual images, auditory signals, and physiological indicators after Kalman filter denoising into the AI hub for multi-modal analysis, realizing environment recognition and user state evaluation. 8.The life assistant system based on artificial intelligence according to claim 1, wherein, The hardware of each module of the artificial intelligence-based life assistance system is distributed as follows: the hardware of the system is a virtual helmet, the virtual helmet includes a life monitoring sensor array integrated on the left and right sides of the neck, an electromyographic signal acquisition module arranged on the neck, a bionic conduction earphone arranged on the lower side of the ear, two loudspeakers arranged on both sides of the respirator, a Bluetooth communication module integrated inside, four cameras arranged on the outside, two cameras built-in for scanning the pupils, a protective lens arranged in front of the eyes, and a flexible screen arranged on the inner side of the protective lens. 9.The life assistant system based on artificial intelligence according to claim 1, wherein, The system supports third-party device access, opens a motion control instruction interface and a physiological data interface through a RESTAPI, and can be linked with a hospital monitoring system or a smart home device.

10. The life assisting system based on artificial intelligence according to any one of claims 1-9, characterized in that, The system is equipped with a 5000mAh or larger capacity lithium polymer battery and supports Qi protocol wireless fast charging.