Remote monitoring robot for health monitoring of old people

Abstract

The invention discloses a remote monitoring robot for health monitoring of old people, and relates to the technical field of monitoring robots, the remote monitoring robot comprises a mobile robot module, and the mobile robot module is in communication connection with a deep learning algorithm module, a voice module, an anomaly detection module, a positioning module and an emergency safety guarantee module. According to the remote monitoring robot for health monitoring of the old people, by integrating a deep learning algorithm and a mobile robot technology, a safe living space is provided for the old people, activities of the old people are monitored in real time, and it is ensured that the old people can be timely helped when encountering health crisis through a fall-down detection and emergency situation feedback mechanism; when it is detected that the old man falls down through a YOLOv5 algorithm, help seeking information is immediately sent to a preset guardian, in addition, the mobile following function of the robot provides continuous monitoring when the old man goes out, and the robot can closely follow the old man at home or outdoors to ensure that no dead angle exists in monitoring.

Description

Technical Field

[0001] The invention relates to the technical field of monitoring robots, and in particular to a remote monitoring robot used for health monitoring of the elderly. Background Art

[0002] As the population ages, the burden of social care for the elderly increases, and it is very important to properly handle the elderly care issues of the whole society. Globally, the health of the elderly is threatened by many fatal diseases, and the elderly living alone lack family care and psychological comfort, resulting in a higher incidence of illness. At the same time, accidents cannot be discovered and rescued in time. As young people are busy with work in their daily lives, the care of the elderly and real-time monitoring of the home environment have become emerging needs.

[0003] However, traditional home cameras are restricted by fixed positions, which limits the monitoring effect and cannot take into account real-time monitoring of the elderly when they go out. From the perspective of caring for the health of the elderly, in response to the above shortcomings, a remote monitoring robot for health monitoring of the elderly is proposed. The robot can monitor the elderly in real time and has mobile functions, realizing a system that continuously follows and monitors the elderly. Summary of the invention

[0004] To achieve the above objectives, the present invention is implemented through the following technical solutions: A remote monitoring robot for health monitoring of the elderly, comprising a mobile robot module, wherein the mobile robot module is communicatively connected with a deep learning algorithm module, a voice module, an anomaly detection module, a positioning module and an emergency safety guarantee module, wherein the modules are electrically connected with each other; The deep learning algorithm module collects video streams in real time through a USB camera, uses the YOLOv5 algorithm to detect falls, intelligently analyzes video content, and identifies the body posture of the elderly. Once a fall is detected, subsequent safety measures will be immediately triggered, thereby improving the detection accuracy and response speed of elderly fall events and effectively preventing serious consequences that may be caused by falls. The mobile robot module integrates CSRT_Tracker tracking technology, and autonomously adjusts the moving direction and speed according to the target position information provided by the deep learning algorithm module, so as to maintain a safe distance from the elderly and continuously monitor them, thus realizing real-time monitoring of the elderly when they go out. Whether the elderly are at home or out, the robot can closely follow them to ensure that there are no blind spots in monitoring. The mobile robot module includes a mobile robot, a development board, a USB camera and a microphone; The voice module is used to deploy a large language model on the development board of the mobile robot module, extract and recognize the voice of the elderly in the ambient sound, including dialect recognition, so that the elderly can send instructions to the robot in the form of voice dialogue, such as "start following", "stop following", etc. In addition, a corresponding mobile phone APP is also provided, which is convenient for the elderly to send instructions from the mobile phone APP, simplifies the operation process, and improves the user experience of the elderly. The elderly do not need to learn complex operation interfaces or buttons, and can control the robot only by voice, realizing convenient human-computer interaction; The anomaly detection module is used to monitor the behavior of the elderly by combining the results of voice recognition and visual recognition, and to detect and warn the occurrence of elderly falls and emergencies; The positioning module enables instant communication between the robot and the emergency contact, as well as precise positioning of the robot, ensuring the rapid delivery of help information and the accurate arrival of rescuers, improving rescue efficiency, and reducing the risks faced by the elderly. Even if an accident occurs when the elderly are out, they can still get help in time; The emergency safety assurance module is used to combine the abnormal behavior detection results of the abnormal detection module, immediately send help and positioning information to the emergency contact, and play help voice in the surrounding area, activate the safety assurance mechanism in an emergency, and provide timely and effective assistance to the elderly. At the same time, through voice recognition technology, the intelligence level of the robot is enhanced, making it more adaptable to the actual needs of the elderly.

[0005] Preferably, in the deep learning algorithm module, the process of identifying the body posture of the elderly includes: A USB camera is installed on the mobile robot module to collect real-time video streams of the elderly's activities, continuously monitor the elderly's activity area, and capture dynamic images; Collect a batch of fall and non-fall action samples in advance, build a fall detection model with a ratio of training set: test set = 9:1, learn the unique body posture change pattern when falling, and deploy the fall detection model in the deep learning algorithm module; Segment the captured video stream into separate video frames, each of which is a static image, and preprocess the segmented video frame images, including resizing to match the model input size, normalizing pixel values, etc., to improve model processing efficiency and accuracy; The preprocessed video frames are sent to the YOLOv5 algorithm for target detection, detecting the human body contour of the elderly in the video frame image and distinguishing the elderly from other objects or backgrounds; The deep learning algorithm module further analyzes the human posture in the human body contour, detects the position and relative relationship of the key points of the elderly's body (such as head, shoulders, knees, etc.), and combines the preset fall detection model to determine whether the elderly has fallen.

[0006] Preferably, the process of determining whether the elderly person has fallen down includes: Run the YOLOv5 algorithm on the video frame, detect the human body in the video frame based on the deep learning algorithm, and output the bounding box of the human body; In the output human body bounding box area, a posture estimation algorithm (such as OpenPose, HRNet, etc.) is applied to identify the key points of the human body, such as the head, shoulders, knees, ankles, etc., and generate two-dimensional coordinates The key points are represented in the image in the form of The coordinates of the key points are ,in Indicates the index of the key point (such as head is 1, left shoulder is 2, ..., right ankle is N); The fall features are extracted from the detected key points, wherein the fall features include the distance, angle, and relative position between the key points. According to the preset fall detection model, whether the fall features conform to the behavior pattern when falling is analyzed. The distance between the key points is calculated using the Euclidean distance formula, and its expression is: , For key points and The distance between For key points The coordinates of For key points The angles between key points are calculated using the vector dot product formula, which is expressed as: , For key points , and The angle formed, For key points The coordinates of For key points and The distance between Thresholds are preset for different fall features. When the fall detection node detects that the feature value is lower than the preset threshold for five consecutive times, the fall detection model is combined to determine whether the fall feature meets the behavior pattern during a fall.

[0007] Preferably, in the mobile robot module, the process of real-time monitoring of the elderly when they go out includes: The elderly person turns on the robot before going out and starts the mobile robot through voice control or mobile phone APP operation. The deep learning algorithm module is used to process the video frame data and output the key points of the elderly person's body. Use the posture estimation algorithm to output key point information, initialize the CSRT_Tracker tracking algorithm, use CSRT_Tracker to track the position of the elderly in consecutive video frames, and update its bounding box coordinates in real time; Get the elderly person's real-time location information from CSRT_Tracker, including the center coordinates of the bounding box. The mobile robot uses its own sensors (such as wheel encoders, IMU, etc.) to determine its location. After identifying the target, calculate the pixel center coordinates of the target. If the coordinates appear on the right side of the USB camera, control the mobile robot to rotate right. The same goes for the left side. By constantly fine-tuning the rotation direction and speed, the target is always in the center of the picture. According to the real-time position of the elderly and the current position of the mobile robot, the path planning algorithm is used to calculate the best path for the mobile robot to reach the target location. Based on the path planning results, the speed and direction that the mobile robot needs to adjust are calculated to maintain a safe distance from the elderly. The driving system of the mobile robot performs corresponding movement operations according to the calculation results, including adjusting the speed and steering. During the movement of the mobile robot, it continuously uses the USB camera and CSRT_Tracker to track the position of the elderly. The mobile robot dynamically adjusts the speed and position based on the elderly's movement speed and direction to ensure stability and continuity during the following process and avoid losing the target.

[0008] Preferably, in the voice module, the process of extracting and identifying the elderly person's voice from the ambient sound includes: Pre-enter the command vocabulary preset by the elderly and store the preset command vocabulary to match the elderly's voice commands, such as "start following" or "stop following"; The microphone equipped with the mobile robot module is used to continuously collect environmental sounds, and the collected environmental sounds are pre-processed, including noise reduction, echo cancellation, gain control and other pre-processing operations to improve the accuracy of speech recognition; Through voice activity detection technology, the effective voice part in the environmental sound is identified, and the large language model deployed on the development board is used to recognize the effective voice part, process different dialects and accents, recognize the voice text, improve the recognition accuracy, and ensure that the elderly can communicate with the robot in a natural and accustomed language; Perform natural language processing on the voice text, including semantic understanding and command classification, to ensure that the elderly's instructions are correctly understood, and match the recognized voice text with the preset command vocabulary to analyze the elderly's intentions; The parsed instructions are sent to the development board of the mobile robot module, which controls the mobile robot to perform corresponding actions. After the robot executes the instructions, it provides feedback to the elderly through voice and display screen to confirm that the instructions have been executed.

[0009] Preferably, in the abnormality detection module, the process of monitoring the behavior of the elderly includes: Collect audio and video data of normal and abnormal behaviors (such as falling, calling for help, etc.) of the elderly under different environments, lighting conditions, and background noises, and pre-process the collected audio and video data to mark abnormal behavior data; Analyze the preprocessed audio and video data, extract sound features and visual features therefrom, and mark abnormal sound features and abnormal action features in the sound features and visual features respectively, to obtain a sound feature set and a visual feature set; Using the abnormal sound features in the sound feature set, the sound intensity and frequency analysis of the audio data are performed, the decibel value and frequency deviation of the abnormal sound features are calculated, and the sound feature weighted summation function is obtained. The weights of all sampling points are calculated, and the benchmark value in the long-term trend is found to obtain the benchmark adjustment index function. The abnormal sound is weighted to obtain the abnormal sound index, and the degree of overall abnormal sound is evaluated. Using the abnormal action features in the visual feature set, the normal behavior benchmark value is preset, and the deviation between a single abnormal action feature and the normal behavior benchmark value is calculated to obtain the behavior deviation weighted summation function. The abnormal scores of all video frames at all time points are calculated, and the maximum value in the long-term trend is found to obtain the time series maximum weighted abnormal score function. Multiple abnormal action features are weighted to obtain the abnormal behavior index, and the overall degree of abnormal behavior is evaluated. The training data set is constructed by using the associated data of the sound feature set and the visual feature set, and the anomaly detection model is trained through the neural network model to identify the characteristics of normal and abnormal behaviors; Based on the anomaly detection model, combined with the abnormal behavior data in the sound feature set and the visual feature set, the abnormal sound index and the abnormal behavior index are comprehensively analyzed to obtain the abnormal warning coefficient, analyze the detected abnormal behavior, set the warning threshold, and determine the node that triggers the warning mechanism; The microphone equipped with the mobile robot module is used to continuously collect environmental sounds, and the USB camera installed on the robot is used to collect video streams in real time. The sound and visual features are extracted and input into the trained anomaly detection model. According to the output of the anomaly detection model and the anomaly warning coefficient, it is determined whether the warning mechanism should be triggered.

[0010] Preferably, the abnormal sound index is obtained based on a sound feature weighted sum function and a benchmark adjustment index function, and its expression is: in, is the abnormal sound index, is the weighted sum function of sound features, The exponential function is adjusted for the benchmark, For the The decibel value of the sampling point represents the intensity of the sound. For the The frequency deviation of the sampling points indicates the deviation of the sound frequency from normal speech. is the total number of sampling points, is a preset reference constant used to adjust the impact of the exponential function. For the The weight of the samples, The value is between 0 and 1, 0 means no abnormality, 1 means high abnormality. and increases (i.e., increases in sound intensity and frequency deviation), Increases, indicating an increased likelihood of an abnormality.

[0011] Preferably, the abnormal behavior index is obtained based on a behavior deviation weighted sum function and a time series maximum weighted abnormal scoring function, and its expression is: in, is the abnormal behavior index, is the weighted sum function of behavioral deviation, is the maximum weighted anomaly scoring function for time series, For the The action feature is The measurements in the video frames, For the The normal behavior baseline value of the action feature, is the total number of frames of a single action feature, For the The standard deviation of the action features, For the The standard deviation of normal behavior of action features, is the number of action features, For the The weight of the video frame, For the The anomaly score of each video frame in the first evaluation, The value of is between 0 and 1, where 0 indicates no abnormality and 1 indicates high abnormality. As the action characteristics deviate from normal behavior, Increases, indicating an increased likelihood of an abnormality.

[0012] Preferably, the expression of the abnormal warning coefficient is: in, Abnormal warning coefficient, For in time The abnormal behavior index, is the abnormal sound index at time t, T is the total number of time points, and is the weight coefficient, which is used to adjust the influence of abnormal behavior index and abnormal sound index in the abnormal warning coefficient. is a preset reference constant used to adjust the impact of the exponential function. is the weight at the rth time point, is the abnormal behavior index at the rth time point, is the abnormal sound index at the rth time point, The value is between 0 and 1, where 0 indicates no abnormality and 1 indicates a high abnormality.

[0013] Preferably, in the emergency safety assurance module, the process of starting the safety assurance mechanism includes: Use deep learning algorithm modules and anomaly detection modules to continuously analyze sound and visual data, monitor the behavior of the elderly, and trigger warning signals when abnormal behavior is identified; When the warning signal is triggered, it automatically sends a help message to the preset emergency contact, including text messages, phone calls or app push notifications, and uses the built-in GPS of the mobile robot module to obtain the precise location of the elderly; After sending the help message, a pre-recorded help voice is played in the environment around the elderly to attract the attention of nearby people and seek help, monitor the elderly's condition, and provide real-time updates to emergency contacts. After detecting that the abnormal condition is lifted, a message to lift the alarm is sent.

[0014] The present invention provides a remote monitoring robot for health monitoring of the elderly. It has the following beneficial effects: 1. This remote monitoring robot used for health monitoring of the elderly provides a safe living space for the elderly by integrating deep learning algorithms and mobile robot technology, monitors the activities of the elderly in real time, and ensures that the elderly can get timely help when they encounter health crises through fall detection and emergency feedback mechanisms. When the YOLOv5 algorithm detects that the elderly have fallen, it immediately sends a message for help to the preset guardian. In addition, the robot's mobile following function provides continuous monitoring when the elderly go out. Whether at home or outdoors, the robot can follow closely to ensure that there are no blind spots in monitoring, which significantly improves the safety of the elderly.

[0015] 2. The remote monitoring robot used for health monitoring of the elderly analyzes video information and sound data to evaluate the elderly's behavior patterns and issues warnings when abnormal behavior is detected. It comprehensively considers sound and visual information to provide more accurate abnormal behavior assessments. If the elderly make abnormal sounds or their behavior patterns are significantly different from normal patterns, the warning mechanism will be triggered and the guardian will be notified through the preset communication channels, so that the guardian can understand the elderly's condition in a timely manner and take necessary intervention measures to prevent potential risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is an overall flow chart of a remote monitoring robot for health monitoring of the elderly according to the present invention; Figure 2 It is a module structure diagram of the present invention; Figure 3 The present invention is a flow chart of monitoring the behavior of the elderly. DETAILED DESCRIPTION

[0017] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention are provided for the purpose of illustration and description, and are not intended to be exhaustive or to limit the present invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention, and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.

[0018] The first embodiment, as Figure 1 , Figure 2 As shown, the present invention provides a technical solution: a remote monitoring robot for health monitoring of the elderly, comprising a mobile robot module, the mobile robot module is communicatively connected with a deep learning algorithm module, a voice module, an anomaly detection module, a positioning module and an emergency safety guarantee module, wherein the modules are connected by electrical signals; The deep learning algorithm module collects video streams in real time through a USB camera, uses the YOLOv5 algorithm to detect falls, intelligently analyzes video content, and identifies the body posture of the elderly. Once a fall is detected, subsequent safety measures will be triggered immediately, which improves the detection accuracy and response speed of falls of the elderly and effectively prevents the serious consequences that may be caused by falls. A USB camera is installed on the mobile robot module, and the video stream of the elderly's activities is collected in real time through the USB camera. The activity area of ​​the elderly is continuously monitored, dynamic images are captured, and batches of fall and non-fall action samples are collected in advance. A fall detection model is constructed according to the ratio of training set: test set = 9:1, and the unique body posture change pattern when falling is learned. The fall detection model is deployed in the deep learning algorithm module, and the collected video stream is segmented into single Separate video frames, each of which is a static image, and preprocess the segmented video frame images, including resizing to match the model input size, normalizing pixel values, etc., to improve the model processing efficiency and accuracy, and send the preprocessed video frames to the YOLOv5 algorithm for target detection, detect the human body contour of the elderly in the video frame image, and distinguish the elderly from other objects or backgrounds. The deep learning algorithm module further analyzes the human posture in the human body contour, detects the position and relative relationship of the key points of the elderly's body (such as head, shoulders, knees, etc.), and combines the preset fall detection model to determine whether the elderly have fallen. The YOLOv5 algorithm is run on the video frame, and the human body bounding box is output according to the deep learning algorithm's detection of the human body in the video frame. In the output human body bounding box area, the posture estimation algorithm (such as OpenPose, HRNet, etc.) is applied to identify the key points of the human body, such as head, shoulders, knees, ankles, etc., and express them in two-dimensional coordinates. The key points are represented in the image in the form of The coordinates of the key points are ,in Represents the index of the key point (such as 1 for the head, 2 for the left shoulder, ..., N for the right ankle), extracts the fall features from the detected key points, where the fall features include the distance, angle, and relative position between the key points, and analyzes whether the fall features conform to the behavior pattern when falling according to the preset fall detection model. The distance between the key points is calculated using the Euclidean distance formula, which is expressed as: , For key points and The distance between For key points The coordinates of For key points The angles between key points are calculated using the vector dot product formula, which is expressed as: , For key points , and The angle formed, For key points The coordinates of For key points and The distance between Preset thresholds for different fall features. When the fall detection node detects that the feature value is lower than the preset threshold for five consecutive times, the fall detection node is combined with the preset fall detection model to determine that the fall feature meets the behavior pattern of falling. The mobile robot module integrates CSRT_Tracker tracking technology. According to the target position information provided by the deep learning algorithm module, it can autonomously adjust the moving direction and speed, maintain a safe distance from the elderly and continuously monitor them, and realize real-time monitoring of the elderly when they go out. Whether the elderly are at home or out, the robot can follow closely to ensure that there are no blind spots in the monitoring. The mobile robot module includes a mobile robot, a development board, a USB camera and a microphone. Among them, the development board: uses the Fibocom sc171 development board as the hardware platform. The SC171 series is a multi-network standard 5G smart module launched by Fibocom. It is designed based on the Qualcomm SM6490 platform and uses an 8-core high-performance processor (1*KryoGold plus 2.7GHz +3*KryoGold 2.4GHz+4*KryoSliver 1.95GHz), built-in VDSP, and integrated high-performance graphics engine. It can play 4K videos smoothly. The SC171 series supports 5G NSA and SA modes, is backward compatible with 4G / 3G networks, and supports 5GNRsub-6Ghz, DL4x4 MIMO, UL 2x2 MIMO, supports 2.4G+5GWLAN wireless communication, supports GPS (L1+L5) / Beidou / GLONASS wireless positioning technology, SC171 is equipped with Android12 operating system, has MIPI / USB / UART / SPI / I2C and other expansion interfaces, and deploys aidlux as a deep learning platform in the environment. The native environment drives the camera and positioning module for aidlux to call and read data. CSTR_Tracker sends a drive signal to the chassis stm32 drive motor through serial communication. The elderly turn on the robot before going out, and start the mobile robot through voice control or mobile phone APP operation. The deep learning algorithm module is used to process the video frame data, output the key points of the elderly, and use the posture estimation algorithm to output the key point information. The CSRT_Tracker tracking algorithm is initialized, and the CSRT_Tracker is used to track the position of the elderly in continuous video frames. The coordinates of its bounding box are updated in real time, and the real-time position information of the elderly is obtained from the CSRT_Tracker, including the coordinates of the center point of the bounding box. The mobile robot uses its own sensors (such as wheel encoders, IMU, etc.) to determine the At the location, after identifying the target, calculate the pixel center coordinates of the target. If the coordinates appear on the right side of the USB camera, control the mobile robot to rotate right. The same is true for the left side. By constantly fine-tuning the rotation direction and speed, the target is always in the center of the picture. According to the real-time position of the elderly and the current position of the mobile robot, the path planning algorithm is used to calculate the best path for the mobile robot to reach the target position. According to the path planning results, the speed and direction that the mobile robot needs to adjust are calculated to maintain a safe distance from the elderly. The driving system of the mobile robot performs corresponding mobile operations according to the calculation results, including adjusting the speed and steering. During the movement of the mobile robot, the USB camera and CSRT_Tracker are continuously used to track the position of the elderly. The mobile robot combines the moving speed and direction of the elderly to dynamically adjust the speed and position to ensure the stability and continuity of the following process and avoid losing the target. The voice module is used to deploy a large language model on the development board of the mobile robot module, extract and recognize the voice of the elderly in the ambient sound, including dialect recognition, so that the elderly can send instructions to the robot through voice dialogue, such as "start following", "stop following", etc. In addition, it is also equipped with a corresponding mobile phone APP, which is convenient for the elderly to issue instructions from the mobile phone APP, simplifying the operation process and improving the user experience of the elderly. The elderly do not need to learn complex operation interfaces or buttons, and can control the robot only by voice to achieve convenient human-computer interaction. The command vocabulary preset by the elderly is pre-entered and stored to match the voice commands of the elderly. Voice commands such as "start following" or "stop following" use the microphone equipped with the mobile robot module to continuously collect ambient sounds and pre-process the collected ambient sounds, including noise reduction and echo. Eliminate, gain control and other pre-processing operations to improve the accuracy of speech recognition. Through voice activity detection technology, identify the effective speech part in the environmental sound, use the large language model deployed on the development board to recognize the effective speech part, process different dialects and accents, recognize the speech text, improve the recognition accuracy, and ensure that the elderly can communicate with the robot in a natural and accustomed language. Perform natural language processing on the speech text, including semantic understanding and command classification, to ensure that the elderly’s instructions are correctly understood, and match the recognized speech text with the preset command vocabulary to parse the elderly’s intentions. Send the parsed instructions to the development board of the mobile robot module, which controls the mobile robot to perform the corresponding actions. After the robot executes the instructions, it provides feedback to the elderly through voice and display screen to confirm that the instructions have been executed; The anomaly detection module is used to combine the results of voice recognition and visual recognition to monitor the behavior of the elderly and detect and warn of falls and emergency events; The positioning module enables instant communication between the robot and emergency contacts, as well as precise positioning of the robot, ensuring the rapid delivery of help information and the accurate arrival of rescuers, improving rescue efficiency and reducing the risks faced by the elderly. Even if an accident occurs while the elderly are out, they can still get help in time. The emergency safety assurance module is used to combine the abnormal behavior detection results of the abnormal detection module, immediately send help and positioning information to the emergency contact, and play the help voice in the surrounding area, activate the safety assurance mechanism in an emergency, and provide timely and effective assistance to the elderly. At the same time, through voice recognition technology, the robot's intelligence level is enhanced, making it more adaptable to the actual needs of the elderly.

[0019] The second embodiment is based on the first embodiment. Figure 3 As shown in the figure, in the anomaly detection module, the process of monitoring the behavior of the elderly includes: The audio and video data of the normal and abnormal behaviors (such as falling, calling for help, etc.) of the elderly under different environments, lighting conditions, and background noises are collected, and the collected audio and video data are preprocessed, abnormal behavior data is marked, and the preprocessed audio and video data are analyzed to extract sound features and visual features therefrom, and abnormal sound features and abnormal action features in the sound features and visual features are marked respectively to obtain a sound feature set and a visual feature set. The abnormal sound features in the sound feature set are used to perform sound intensity and frequency analysis on the audio data, and the decibel value and frequency deviation of the abnormal sound features are calculated to obtain a weighted summation function of the sound features, the weights of all sampling points are calculated, the benchmark value in the long-term trend is found, the benchmark adjustment index function is obtained, and the abnormal sound is weightedly calculated to obtain an abnormal sound index to evaluate the degree of overall abnormal sound. The abnormal action features in the visual feature set are used to preset a normal behavior benchmark value, and the deviation between a single abnormal action feature and the normal behavior benchmark value is calculated to obtain a behavior deviation weighted index. Sum function, calculate the anomaly score of all video frames at all time points, find the maximum value in the long-term trend, obtain the maximum weighted anomaly score function of the time series, weightedly calculate multiple abnormal action features, obtain the abnormal behavior index, evaluate the overall abnormal behavior degree, use the associated data of the sound feature set and the visual feature set to build a training data set, train the anomaly detection model through the neural network model, identify the characteristics of normal and abnormal behaviors, combine the abnormal behavior data in the sound feature set and the visual feature set based on the anomaly detection model, conduct a comprehensive analysis of the abnormal sound index and the abnormal behavior index to obtain the abnormal warning coefficient, analyze the detected abnormal behavior, set the warning threshold, determine the node that triggers the warning mechanism, use the microphone equipped with the mobile robot module to continuously collect environmental sounds, and use the USB camera installed on the robot to collect video streams in real time, extract the sound features and visual features, input the trained anomaly detection model, and determine whether to trigger the warning mechanism based on the output of the anomaly detection model and the abnormal warning coefficient; Furthermore, the abnormal sound index is obtained based on the sound feature weighted sum function and the benchmark adjustment index function, and its expression is:

[0020] in, is the abnormal sound index, is the weighted sum function of sound features, The exponential function is adjusted for the benchmark, For the The decibel value of the sampling point indicates the intensity of the sound. For the The frequency deviation of the sampling points indicates the deviation of the sound frequency from normal speech. is the total number of sampling points, is a preset reference constant used to adjust the impact of the exponential function. For the The weight of the samples, The value is between 0 and 1, 0 means no abnormality, 1 means high abnormality. and increases (i.e., increases in sound intensity and frequency deviation), Increase, indicating an increased likelihood of abnormality, It means calculating the decibel value and frequency deviation of each sound sampling point, and adjusting it with the logarithmic function to evaluate the abnormality of the sound. By summing and summing the abnormality of all sampling points, an overall sound abnormality score is obtained. It means calculating the weights of all sampling points and finding the benchmark value in the long-term trend. Through the exponential function and the limit function, the benchmark value of the sound feature in the long-term trend is focused on to adjust the sensitivity of the overall abnormal sound index. The abnormal behavior index is obtained based on the weighted sum function of behavioral deviation and the maximum weighted anomaly scoring function of time series, and its expression is:

[0021] in, is the abnormal behavior index, is the weighted sum function of behavioral deviation, is the maximum weighted anomaly scoring function for time series, For the The action feature is The measurements in the video frames, For the The normal behavior baseline value of the action feature, is the total number of frames of a single action feature, For the The standard deviation of the action features, For the The standard deviation of normal behavior of action features, is the number of action features, For the The weight of the video frame, For the The anomaly score of each video frame in the first evaluation, The value of is between 0 and 1, where 0 indicates no abnormality and 1 indicates high abnormality. As the action characteristics deviate from normal behavior, Increase, indicating an increased likelihood of abnormality, It means calculating the deviation of each action feature in each video frame and adjusting it with the standard deviation to evaluate the abnormality of the action. By summing and using the exponential function, the abnormality of all action features is combined to obtain an overall abnormal behavior score. It means calculating the anomaly scores of all video frames at all time points and finding the maximum value in the long-term trend. Through the limit function and exponential function, it focuses on the maximum anomaly score in the long-term trend to evaluate the overall abnormal behavior risk. Furthermore, the expression of the abnormal warning coefficient is:

[0022] in, Abnormal warning coefficient, For in time The abnormal behavior index, is the abnormal sound index at time t, T is the total number of time points, and is the weight coefficient, which is used to adjust the influence of abnormal behavior index and abnormal sound index in the abnormal warning coefficient. is a preset reference constant used to adjust the impact of the exponential function. is the weight at the rth time point, is the abnormal behavior index at the rth time point, is the abnormal sound index at the rth time point, The value of is between 0 and 1, where 0 indicates no abnormality and 1 indicates a high abnormality. and increase, Increase, indicating that the possibility of abnormality increases, which may trigger the early warning mechanism. For each time point, the abnormal behavior index and abnormal sound index The combination of the root and logarithmic functions is to reduce the amplification effect of outliers while maintaining sensitivity to outliers. and The effects of behavioral and sound indices were adjusted to reflect their importance in early warning, and the adjusted scores at all time points were weighted summed to assess the overall abnormal early warning level; In the emergency safety assurance module, the process of starting the safety assurance mechanism includes: The deep learning algorithm module and the anomaly detection module are used to continuously analyze the sound and visual data and monitor the behavior of the elderly. When abnormal behavior is identified, an early warning signal is triggered. When the early warning signal is triggered, a help message is automatically sent to the preset emergency contact, including text messages, phone calls or application push notifications, and the built-in GPS of the mobile robot module is used to obtain the precise location of the elderly. After sending the help message, the pre-recorded help voice is played in the environment around the elderly to attract the attention of nearby people and seek help. The elderly's status is monitored and real-time updates are provided to the emergency contact. After detecting that the abnormal status is lifted, a message to lift the alarm is sent.

[0023] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field and related fields without creative work should fall within the scope of protection of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention are implemented according to the conventional means in the field unless otherwise specified and limited.

Claims

1. A remote monitoring robot for health monitoring of the elderly, characterized in that: It includes a mobile robot module, which is communicatively connected to a deep learning algorithm module, a voice module, an anomaly detection module, a positioning module, and an emergency safety assurance module, wherein electrical signals are connected between the modules; The deep learning algorithm module collects video streams in real time through a USB camera, uses the YOLOv5 algorithm to detect falls, intelligently analyzes video content, and recognizes the body posture of the elderly; The mobile robot module integrates CSRT_Tracker tracking technology, and autonomously adjusts the moving direction and speed according to the target position information provided by the deep learning algorithm module, so as to maintain a safe distance from the elderly and continuously monitor them, thus realizing real-time monitoring of the elderly when they go out. The mobile robot module includes a mobile robot, a development board, a USB camera and a microphone; The voice module is used to deploy a large language model on the development board of the mobile robot module, extract and recognize the voice of the elderly in the ambient sound, including dialect recognition, so that the elderly can send instructions to the robot in the form of voice dialogue; The anomaly detection module is used to monitor the behavior of the elderly by combining the results of voice recognition and visual recognition, and to detect and warn the occurrence of elderly falls and emergencies; The positioning module enables instant communication between the robot and the emergency contact, as well as accurate positioning of the robot; The emergency safety guarantee module is used to combine the abnormal behavior detection result of the abnormal detection module, immediately send help and positioning information to the emergency contact, and play help voice in the surrounding area.

2. A remote monitoring robot for health monitoring of the elderly according to claim 1, characterized in that: In the deep learning algorithm module, the process of identifying the body posture of the elderly includes: A USB camera is installed on the mobile robot module to collect real-time video streams of the elderly's activities, continuously monitor the elderly's activity area, and capture dynamic images; Collect a batch of fall and non-fall action samples in advance, build a fall detection model with a ratio of training set: test set = 9:1, learn the unique body posture change pattern when falling, and deploy the fall detection model in the deep learning algorithm module; Segment the captured video stream into separate video frames, each frame being a static image, and preprocess the segmented video frame images; The preprocessed video frames are sent to the YOLOv5 algorithm for target detection, detecting the human body contour of the elderly in the video frame image and distinguishing the elderly from other objects or backgrounds; The deep learning algorithm module further analyzes the human posture in the human body contour, detects the position and relative relationship of the key points of the elderly's body, and combines the preset fall detection model to determine whether the elderly has fallen.

3. A remote monitoring robot for health monitoring of the elderly according to claim 2, characterized in that: The process of determining whether the elderly person has fallen down includes: Run the YOLOv5 algorithm on the video frame, detect the human body in the video frame based on the deep learning algorithm, and output the bounding box of the human body; In the output human body bounding box area, the posture estimation algorithm is applied to identify the key points of the human body and calculate them in two-dimensional coordinates. The key points are represented in the image in the form of The coordinates of the key points are in Indicates the index of the key point; The fall features are extracted from the detected key points, wherein the fall features include the distance, angle, and relative position between the key points. According to the preset fall detection model, whether the fall features conform to the behavior pattern when falling is analyzed. The distance between the key points is calculated using the Euclidean distance formula, and its expression is: , For key points and The distance between For key points The coordinates of For key points The angles between key points are calculated using the vector dot product formula, which is expressed as: , For key points , and The angle formed, For key points The coordinates of For key points and The distance between Thresholds are preset for different fall features. When the fall detection node detects that the feature value is lower than the preset threshold for five consecutive times, the fall detection model is combined to determine whether the fall feature meets the behavior pattern during a fall.

4. A remote monitoring robot for elderly health monitoring according to claim 3, characterized in that: In the mobile robot module, the process of real-time monitoring of the elderly when they go out includes: The elderly person turns on the robot before going out and starts the mobile robot through voice control or mobile phone APP operation. The deep learning algorithm module is used to process the video frame data and output the key points of the elderly person's body. Use the posture estimation algorithm to output key point information, initialize the CSRT_Tracker tracking algorithm, use CSRT_Tracker to track the position of the elderly in consecutive video frames, and update its bounding box coordinates in real time; The real-time location information of the elderly is obtained from CSRT_Tracker, including the coordinates of the center point of the bounding box. The mobile robot uses its own sensors to determine its location. After the target is identified, the pixel center coordinates of the target are calculated. If the coordinates appear on the right side of the USB camera, the mobile robot is controlled to rotate right. The same is true for the left side. By constantly fine-tuning the rotation direction and speed, the target is always in the center of the picture. According to the real-time position of the elderly and the current position of the mobile robot, the path planning algorithm is used to calculate the best path for the mobile robot to reach the target location. Based on the path planning results, the speed and direction that the mobile robot needs to adjust are calculated to maintain a safe distance from the elderly. The driving system of the mobile robot performs corresponding movement operations according to the calculation results, including adjusting the speed and steering. During the movement of the mobile robot, the USB camera and CSRT_Tracker are continuously used to track the position of the elderly. The mobile robot dynamically adjusts the speed and position based on the movement speed and direction of the elderly.

5. A remote monitoring robot for elderly health monitoring according to claim 4, characterized in that: In the voice module, the process of extracting and identifying the elderly person's voice from the ambient sound includes: Pre-enter the command vocabulary preset by the elderly and store the preset command vocabulary to match the elderly's voice commands; The microphone equipped with the mobile robot module is used to continuously collect environmental sounds and pre-process the collected environmental sounds; Through voice activity detection technology, the effective speech part in the environmental sound is identified, and the large language model deployed on the development board is used to recognize the effective speech part, process different dialects and accents, and recognize the speech text; Perform natural language processing on the voice text, including semantic understanding and command classification, and match the recognized voice text with the preset command vocabulary to analyze the elderly's intentions; The parsed instructions are sent to the development board of the mobile robot module, which controls the mobile robot to perform corresponding actions. After the robot executes the instructions, it provides feedback to the elderly through voice and display screen to confirm that the instructions have been executed.

6. A remote monitoring robot for health monitoring of the elderly according to claim 5, characterized in that: In the anomaly detection module, the process of monitoring the behavior of the elderly includes: Collect audio and video data of normal and abnormal behaviors of the elderly under different environments, lighting conditions, and background noises, pre-process the collected audio and video data, and mark abnormal behavior data; Analyze the preprocessed audio and video data, extract sound features and visual features therefrom, and mark abnormal sound features and abnormal action features in the sound features and visual features respectively, to obtain a sound feature set and a visual feature set; Using the abnormal sound features in the sound feature set, the sound intensity and frequency analysis of the audio data are performed, the decibel value and frequency deviation of the abnormal sound features are calculated, and the sound feature weighted summation function is obtained. The weights of all sampling points are calculated, and the benchmark value in the long-term trend is found to obtain the benchmark adjustment index function. The abnormal sound is weighted to obtain the abnormal sound index, and the degree of overall abnormal sound is evaluated. Using the abnormal action features in the visual feature set, the normal behavior benchmark value is preset, and the deviation between a single abnormal action feature and the normal behavior benchmark value is calculated to obtain the behavior deviation weighted summation function. The abnormal scores of all video frames at all time points are calculated, and the maximum value in the long-term trend is found to obtain the time series maximum weighted abnormal score function. Multiple abnormal action features are weighted to obtain the abnormal behavior index, and the overall degree of abnormal behavior is evaluated. The training data set is constructed by using the associated data of the sound feature set and the visual feature set, and the anomaly detection model is trained through the neural network model to identify the characteristics of normal and abnormal behaviors; Based on the anomaly detection model, combined with the abnormal behavior data in the sound feature set and the visual feature set, the abnormal sound index and the abnormal behavior index are comprehensively analyzed to obtain the abnormal warning coefficient, analyze the detected abnormal behavior, set the warning threshold, and determine the node that triggers the warning mechanism; The microphone equipped with the mobile robot module is used to continuously collect environmental sounds, and the USB camera installed on the robot is used to collect video streams in real time. The sound and visual features are extracted and input into the trained anomaly detection model. According to the output of the anomaly detection model and the anomaly warning coefficient, it is determined whether the warning mechanism should be triggered.

7. A remote monitoring robot for health monitoring of the elderly according to claim 6, characterized in that: The abnormal sound index is obtained based on the sound feature weighted sum function and the reference adjustment index function, and its expression is: in, is the abnormal sound index, is the weighted sum function of sound features, The exponential function is adjusted for the benchmark, For the The decibel value of the sampling point represents the intensity of the sound. For the The frequency deviation of the sampling points indicates the deviation of the sound frequency from normal speech. is the total number of sampling points, is a preset reference constant used to adjust the effect of the exponential function. For the The weight of the samples, The value is between 0 and 1, where 0 indicates no abnormality and 1 indicates a high degree of abnormality.

8. A remote monitoring robot for elderly health monitoring according to claim 7, characterized in that: The abnormal behavior index is obtained based on the weighted sum function of the behavior deviation and the maximum weighted abnormal scoring function of the time series, and its expression is: in, is the abnormal behavior index, is the weighted sum function of behavioral deviation, is the maximum weighted anomaly scoring function for time series, For the The action feature is The measurements in the video frames, For the The normal behavior baseline value of the action feature, is the total number of frames of a single action feature, For the The standard deviation of the action features, For the The standard deviation of normal behavior of action features, is the number of action features, For the The weight of the video frame, For the The anomaly score of each video frame in the first evaluation, The value is between 0 and 1, where 0 indicates no abnormality and 1 indicates a high abnormality.

9. A remote monitoring robot for elderly health monitoring according to claim 8, characterized in that: The expression of the abnormal warning coefficient is: in, Abnormal warning coefficient, For in time The abnormal behavior index, is the abnormal sound index at time t, T is the total number of time points, and is the weight coefficient, which is used to adjust the influence of abnormal behavior index and abnormal sound index in the abnormal warning coefficient. is a preset reference constant used to adjust the effect of the exponential function. is the weight at the rth time point, is the abnormal behavior index at the rth time point, is the abnormal sound index at the rth time point.

10. A remote monitoring robot for elderly health monitoring according to claim 9, characterized in that: In the emergency safety assurance module, the process of starting the safety assurance mechanism includes: Use deep learning algorithm modules and anomaly detection modules to continuously analyze sound and visual data, monitor the behavior of the elderly, and trigger warning signals when abnormal behavior is identified; When the warning signal is triggered, it automatically sends a help message to the preset emergency contact, including text messages, phone calls or app push notifications, and uses the built-in GPS of the mobile robot module to obtain the precise location of the elderly; After sending the help message, a pre-recorded help voice is played in the environment around the elderly to attract the attention of nearby people and seek help, monitor the elderly's condition, and provide real-time updates to emergency contacts. After detecting that the abnormal condition is lifted, a message to lift the alarm is sent.

Images