Fall detection system and method for accompanying robot
By using a logical judgment algorithm combined with Yolov8 model and human body detection model in the escort robot, combined with non-maximum suppression and AES encryption technology, the problems of fall detection and privacy protection of the escort robot are solved, and the detection accuracy and security are improved. Reduces cost and complexity.
Patent Information
- Application Number
- CN202410970694.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-12-03
AI Technical Summary
Existing escort robots have insufficient accuracy in fall detection and privacy protection, are susceptible to environmental interference, and have high equipment costs, complex use, and have great privacy and security risks.
The target data set is constructed using the data set module, and the fall detection model and the human detection model are trained through the Yolov8 model, and the non-maximum suppression algorithm and the SAM model are combined for logical judgment to realize fall detection and AES encryption is performed during the transmission process.
It improves the accuracy and robustness of fall detection, reduces false positives, ensures the security of user privacy, and reduces equipment costs and complexity of use.
Smart Images

Figure CN119068543A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of accompanying robots, and in particular to a fall detection system and method for accompanying robots. Background Art
[0002] As an innovative solution for home care, companion robots have received widespread attention in recent years. These robots are usually equipped with a variety of sensors and intelligent algorithms, and can move autonomously at home to monitor the daily activities of the elderly. Although existing companion robots have achieved automated monitoring to a certain extent, they still have shortcomings in real-time detection of falls, privacy protection, and system intelligence.
[0003] First, as a key link in the safety monitoring of the elderly, the existing technical means of fall detection mainly rely on simple sensors, such as Doppler radar technology, gyroscopes, etc. These methods are easily affected by environmental interference in practical applications, have low detection accuracy, and are difficult to provide real-time and comprehensive monitoring effects. Although the introduction of computer vision technology has improved the accuracy of fall detection, traditional visual algorithms have poor robustness to changes in ambient light and object occlusion, and are difficult to operate stably in complex home environments.
[0004] Secondly, although the real-time uploading and processing of video data improves the intelligence level of the system, it also brings serious privacy and security risks. Users' daily lives at home are continuously monitored. If these video data are not effectively protected, user privacy may be leaked. Therefore, how to protect user privacy while achieving efficient monitoring is a key issue that needs to be solved in the design of companion robots.
[0005] The existing public document (publication number: CN111175743A) discloses an elderly fall alarm platform and method based on Doppler radar technology. The invention uses Doppler radar technology, the accuracy of which depends on the signal processing algorithm. However, in a complex home environment, factors such as furniture, pets, and light changes may affect the stability and accuracy of the radar signal, and false alarms or missed alarms may still occur. For example, when an elderly person sits down or lies down, the system may mistake it for a fall event. This requires further optimization of the algorithm to improve the accuracy of detection.
[0006] In addition, the application of Doppler radar and high-performance processing units (such as FPGA, DSP, ARM) improves detection performance, but also increases the cost of the system. For some home users, the high cost of the equipment may become a barrier to use. In addition, the daily maintenance and management of the equipment may require the intervention of professionals, which also increases the complexity and cost of use.
[0007] Although the system solves the privacy issue to a certain extent, more comprehensive privacy protection measures are still needed, such as further data encryption or complete anonymization, which may enhance users' trust. Summary of the invention
[0008] In order to accurately detect a fall event, the present invention proposes a fall detection system for a companion robot, comprising:
[0009] The data set module is used to construct a data set, and to label each image in the data set with event categories and target recognition labels to obtain a target data set; the event categories include: fall event categories and non-fall event categories; the target recognition labels include: faller position labels;
[0010] The training module is used to train the Yolov8 model using the target data set to obtain a fall detection model;
[0011] A fall logic judgment module is used to predict the image to be predicted by using a fall detection model and a target detection model, and output a fall prediction image containing a fall detection frame and a human body prediction image containing a human body detection frame, respectively; the fall detection frame corresponds to a potential fall event one by one; the human body detection frame corresponds to a human body one by one; the image to be predicted is a patrol image acquired by the accompanying robot;
[0012] The fall logic judgment module is also used to execute a fall logic judgment algorithm using the detection frames corresponding to the fall prediction image and the human body prediction image as operation data to determine whether the current image to be predicted belongs to the fall event category.
[0013] Further, the detection frames corresponding to the fall prediction image and the human body prediction image are used as operation data to execute the fall logic judgment algorithm, specifically:
[0014] Obtain all detection frames in the fall prediction image and the human body prediction image, remove duplicate detection frames in the fall prediction image and the human body prediction image through a non-maximum suppression algorithm, and retain detection frames whose confidence exceeds a preset value;
[0015] Combining the fall detection frames retained in the fall prediction image with the human body detection frames retained in the human body prediction image in pairs to obtain multiple detection frame pairs;
[0016] Obtain the absolute difference in aspect ratio corresponding to each detection frame pair; the calculation formula for the absolute difference in aspect ratio is: ΔR = |R A -R B |; In the formula, R A Indicates the aspect ratio of the fall detection frame, R Brepresents the aspect ratio corresponding to the human body detection frame, and ΔR represents the absolute difference in aspect ratio; the detection frame pair whose absolute difference in aspect ratio is less than the preset value is marked as the first target category;
[0017] Calculate the distance between the center points of two detection frames in each detection frame pair, and mark the detection frame pairs whose center point distance is greater than a preset value as the second target category;
[0018] Calculate the intersection-and-union ratio corresponding to each detection frame pair, and mark the detection frame pairs whose intersection-and-union ratio is greater than a preset value as the third target category;
[0019] It is determined whether the same detection frame pair belongs to the first target category, the second target category, and the third target category at the same time. If so, the image to be predicted is marked as a fall event category.
[0020] Furthermore, the fall detection system further comprises:
[0021] A privacy processing module, for segmenting the image to be predicted by a SAM model and performing occlusion processing on the segmented privacy area when the image to be predicted is judged to be a fall event category; the input of the SAM model is the image to be predicted containing a fall detection frame and a human body detection frame;
[0022] The alarm module is used to send the image to be predicted after occlusion processing to the preset customer.
[0023] Furthermore, the fall detection system also includes: an encryption transmission module arranged in the accompanying robot, which is used to perform AES encryption on the predicted image and then transmit it to the fall logic judgment module.
[0024] Furthermore, the target detection model is a yolov8m model trained by the coco dataset.
[0025] The present invention also proposes a fall detection method for a companion robot, comprising:
[0026] Constructing a data set, and performing event category labeling and target recognition labeling on each image in the data set to obtain a target data set; the event categories include: fall event categories and non-fall event categories; the target recognition labeling includes: faller position labeling;
[0027] The fall detection model is obtained by training the Yolov8 model with the target dataset;
[0028] The image to be predicted is predicted by the fall detection model and the target detection model respectively, and a fall prediction image containing a fall detection frame and a human body prediction image containing a human body detection frame are output respectively; the fall detection frame corresponds to a potential fall event one by one; the human body detection frame corresponds to a human body one by one; the image to be predicted is a patrol image obtained by the accompanying robot;
[0029] The detection frames corresponding to the fall prediction image and the human body prediction image are used as operation data to execute the fall logic judgment algorithm to determine whether the current image to be predicted belongs to the fall event category.
[0030] Further, the detection frames corresponding to the fall prediction image and the human body prediction image are used as operation data to execute the fall logic judgment algorithm, specifically:
[0031] Obtain all detection frames in the fall prediction image and the human body prediction image, remove duplicate detection frames in the fall prediction image and the human body prediction image through a non-maximum suppression algorithm, and retain detection frames whose confidence exceeds a preset value;
[0032] Combining the fall detection frames retained in the fall prediction image with the human body detection frames retained in the human body prediction image in pairs to obtain multiple detection frame pairs;
[0033] Obtain the absolute difference in aspect ratio corresponding to each detection frame pair; the calculation formula for the absolute difference in aspect ratio is: ΔR = |R A -R B |; In the formula, R A Indicates the aspect ratio of the fall detection frame, R B represents the aspect ratio corresponding to the human body detection frame, and ΔR represents the absolute difference in aspect ratio; the detection frame pair whose absolute difference in aspect ratio is less than the preset value is marked as the first target category;
[0034] Calculate the distance between the center points of two detection frames in each detection frame pair, and mark the detection frame pairs whose center point distance is greater than a preset value as the second target category;
[0035] Calculate the intersection-and-union ratio corresponding to each detection frame pair, and mark the detection frame pairs whose intersection-and-union ratio is greater than a preset value as the third target category;
[0036] It is determined whether the same detection frame pair belongs to the first target category, the second target category, and the third target category at the same time. If so, the image to be predicted is marked as a fall event category.
[0037] Furthermore, the fall detection method further comprises:
[0038] When the image to be predicted is judged as a fall event category, the image to be predicted is segmented by the SAM model, and the segmented privacy area is occluded, and the image to be predicted after occlusion is sent to the preset customer; the input of the SAM model is the image to be predicted containing a fall detection frame and a human body detection frame.
[0039] Furthermore, the fall detection method further comprises:
[0040] The image to be predicted is encrypted by AES through an encryption transmission module arranged in the accompanying robot.
[0041] Furthermore, the target detection model is a yolov8m model trained by the coco dataset.
[0042] Compared with the prior art, the present invention has at least the following beneficial effects:
[0043] (1) In the present invention, the image to be predicted is predicted by the fall detection model and the target detection model respectively, and a fall prediction image including a fall detection frame and a human body prediction image including a human body detection frame are output respectively; the fall detection frame corresponds to a potential fall event one by one; the human body detection frame corresponds to a human body one by one; the fall prediction image and the detection frame corresponding to the human body prediction image are used as operation data to execute a fall logic judgment algorithm to determine whether the current image to be predicted belongs to the fall event category, that is, the present invention improves the accuracy and robustness of fall event detection by combining the prediction results of the fall detection model and the target detection model and executing the fall logic judgment algorithm;
[0044] (2) The present invention calculates the absolute difference in aspect ratio, the distance between the center points of the detection frames and the intersection-and-union ratio for the prediction results of the dual models (fall detection model and target detection model) in sequence, and determines whether the same detection frame pair belongs to the first target category, the second target category and the third target category at the same time. That is, the detection frame pair is triple calculated and judged at the same time. Only when the same detection frame pair belongs to the first target category, the second target category and the third target category at the same time, the image to be predicted is marked as a fall event category, which greatly improves the accuracy of fall detection, effectively reduces false alarms in complex home environments, and improves the robustness of fall detection.
[0045] (3) The present invention improves the security of data transmission by encrypting the predicted image with AES through the encryption transmission module set in the accompanying robot and then transmitting it to the fall logic judgment module;
[0046] (4) When the image to be predicted is judged as a fall event category, the present invention segments the image to be predicted through the SAM model and performs occlusion processing on the segmented privacy area, thereby ensuring that the user's privacy is effectively protected under any circumstances and improving the user's privacy security;
[0047] (5) The present invention sends the image to be predicted after occlusion processing to the preset customer through the alarm module, ensuring that family members can be informed of the fall event in time, thereby improving the timeliness and accuracy of the alarm information. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 This is a module structure diagram of a fall detection system for a companion robot;
[0049] Figure 2 The present invention is a flow chart of a fall detection method for a companion robot. DETAILED DESCRIPTION
[0050] The following are specific embodiments of the present invention and the accompanying drawings to further describe the technical solution of the present invention, but the present invention is not limited to these embodiments.
[0051] Embodiment 1
[0052] In order to improve the accuracy of fall detection, Figure 1 As shown, the present invention proposes a fall detection system for a companion robot, comprising:
[0053] The data set module is used to construct a data set, and to label each image in the data set with event categories and target recognition labels to obtain a target data set; the event categories include: fall event categories and non-fall event categories; the target recognition labels include: faller position labels;
[0054] It should be noted that the data in the data sets such as fall behavior classification in video classification and fall recognition in the field of 3D motion recognition can be collected to construct the data set of this embodiment. At the same time, this embodiment also collects image data of about 100 people covering different lighting conditions, home environments and activity modes of family members for constructing the data set. It should be explained that:
[0055] The images captured by the camera will vary greatly under different lighting conditions, including natural light, indoor lighting, backlight, low light, etc., ensuring that the model can work effectively in any lighting environment.
[0056] Home layout, decoration style, furniture arrangement, etc. may affect the accuracy of fall detection. Collecting image data containing various furniture, floor types (such as carpet, tile, wooden floor), obstacles, etc. can help the model learn to distinguish between falls and normal activities in different environments.
[0057] Considering that the activities of the elderly may be different from those of the young, the collected images include slow movements, sitting, lying down, and other actions that may be similar to falling (such as squatting to pick up something). At the same time, it also includes collecting scenes of multiple people moving at the same time to simulate the actual situation in a home environment. This provides diversity in the image data in the dataset and improves the generalization ability of the model.
[0058] The training module is used to train the Yolov8 model using the target data set to obtain a fall detection model;
[0059] The Yolov8 model is currently widely regarded as one of the strongest object detection models in the field of deep learning. The model is known for its excellent detection accuracy and fast processing speed, and can achieve real-time detection while maintaining high precision. By adopting the Yolov8 model, the system can efficiently identify and locate various target objects, significantly improving detection performance. Its advanced architecture and optimization algorithm not only enhance the robustness of detection, but also reduce false detection and missed detection, ensuring stable operation in complex environments.
[0060] A fall logic judgment module is used to predict the image to be predicted by using a fall detection model and a target detection model, and output a fall prediction image containing a fall detection frame and a human body prediction image containing a human body detection frame, respectively; the fall detection frame corresponds to a potential fall event one by one; the human body detection frame corresponds to a human body one by one; the image to be predicted is a patrol image acquired by the accompanying robot;
[0061] The fall detection system also includes: an encryption transmission module arranged in the accompanying robot, which is used to encrypt the predicted image with AES and then transmit it to the fall logic judgment module to prevent the data from being stolen or tampered with during the transmission process.
[0062] When the companion robot starts the patrol function, it will automatically develop an intelligent patrol route based on the home environment to ensure that the companion robot can cover all important areas. At the same time, it will dynamically adjust the path to avoid insufficient coverage problems caused by fixed routes, and use the encrypted transmission module to encrypt the captured patrol images with AES and then transmit them to the fall logic judgment module.
[0063] The fall logic judgment module is also used to execute a fall logic judgment algorithm using the detection frames corresponding to the fall prediction image and the human body prediction image as operation data to determine whether the current image to be predicted belongs to the fall event category.
[0064] It should be noted that the traditional vision-based fall logic usually uses a posture estimation model for posture detection, and judges whether a fall occurs based on the positional logical relationship of key points. However, this method is prone to missed detection or false detection in complex home environments due to lighting problems and occlusion problems. Using a single detection model or classification model can also result in misidentification, which often occurs in actual tests. Therefore, in order to solve the above problems, the fall logic judgment module of the present invention uses two detection models (fall detection model and target detection model), and formulates three main logic judgment algorithms (absolute difference calculation of aspect ratio, detection frame center point distance calculation and intersection ratio calculation). After a series of calculations on the detection frame, it is determined whether a fall event occurs, which greatly improves the accuracy of detection.
[0065] The fall prediction image and the detection frame corresponding to the human body prediction image are used as operation data to execute the fall logic judgment algorithm, specifically:
[0066] Obtain all detection frames in the fall prediction image and the human body prediction image, remove duplicate detection frames in the fall prediction image and the human body prediction image through a non-maximum suppression algorithm, and retain detection frames whose confidence exceeds a preset value;
[0067] Combining the fall detection frames retained in the fall prediction image with the human body detection frames retained in the human body prediction image in pairs to obtain multiple detection frame pairs;
[0068] Obtain the absolute difference in aspect ratio corresponding to each detection frame pair; the calculation formula for the absolute difference in aspect ratio is: ΔR = |R A -R B |; In the formula, R A Indicates the aspect ratio of the fall detection frame, R B represents the aspect ratio corresponding to the human body detection frame, and ΔR represents the absolute difference in aspect ratio; the detection frame pair whose absolute difference in aspect ratio is less than the preset value is marked as the first target category;
[0069] It should be noted that the width and height of each detection frame in the detection frame pair are calculated, and then the aspect ratio of each detection frame is calculated to make an initial judgment, because a person who falls is usually taller than wide. Then the absolute difference between the two aspect ratios is used to judge the similarity of the two detection frames, so that it can be determined whether the two detection frames show the same person or object.
[0070] Calculate the distance between the center points of the two detection frames in each detection frame pair, and mark the detection frame pairs whose center point distance is greater than a preset value (center point threshold) as the second target category;
[0071] It should be explained that calculating the distance between the center points of the two detection frames can effectively eliminate some false positives. In some environments with occlusion (invisible legs), the human detection frame and the fall detection frame are usually not very similar. At this time, a center point threshold is used to constrain some extreme cases.
[0072] Calculate the intersection-and-union ratio corresponding to each detection frame pair, and mark the detection frame pairs whose intersection-and-union ratio is greater than a preset value as the third target category;
[0073] In this embodiment, by calculating the intersection-union ratio (ratio of intersection and union) of two detection frames in the detection frame pair, the overlap degree between the two detection frames can be fully indicated, and thus can also be used as an important discriminant indicator for determining whether a fall event has occurred.
[0074] It is determined whether the same detection frame pair belongs to the first target category, the second target category, and the third target category at the same time. If so, the image to be predicted is marked as a fall event category.
[0075] In actual tests, it is found that in a complex environment at home, there will be some false detections, such as human-shaped objects. If such false detections are filtered out by setting a higher confidence level, some real fall cases with lower confidence levels due to light, occlusion, etc. in real scenes will also be filtered out. Therefore, the present invention adds a target detection model (i.e., a model for detecting people), so that when setting the confidence level of the fall detection model, the threshold can be lowered to retain most of the real fall cases and a small number of false detection cases.
[0076] The target detection model is a yolov8m model trained by the coco data set. It should be noted that a small number of false positive cases retained by the fall detection model will be filtered out by the target detection model. However, there are still some false positives when judging the fall event only by the detection results of the two detection models. For example, the fall detection model outputs a false positive frame, which is not a real person, and there happens to be a person who does not fall in the figure. At this time, after the judgment of the two models, a false positive will be output. For this, the present invention uses the prediction results of the dual models (fall detection model and target detection model) to calculate the absolute difference of the aspect ratio, the distance between the center points of the detection frame and the intersection and union ratio in turn, that is, the detection frame pair is triple calculated and judged at the same time. Only when the same detection frame pair belongs to the first target category, the second target category and the third target category at the same time, the image to be predicted is marked as a fall event category, which greatly improves the accuracy of fall detection, effectively reduces the false alarm phenomenon in a complex home environment, and improves the robustness of fall detection.
[0077] The fall detection system further comprises:
[0078] A privacy processing module, for segmenting the image to be predicted by a SAM model and performing occlusion processing on the segmented privacy area when the image to be predicted is judged to be a fall event category; the input of the SAM model is the image to be predicted containing a fall detection frame and a human body detection frame;
[0079] The alarm module is used to send the image to be predicted after occlusion processing to the preset customer.
[0080] In this embodiment, the alarm module will send text message reminders to family members (i.e., preset customers) in real time and broadcast voice reminders in real time. The text message will contain an address of the image to be predicted, and the image in this address can be used by the customer to judge the fall situation in real time.
[0081] In the present invention, the image to be predicted is predicted respectively by a fall detection model and a target detection model, and a fall prediction image including a fall detection frame and a human body prediction image including a human body detection frame are output respectively; the fall detection frame corresponds one-to-one to a potential fall event; the human body detection frame corresponds one-to-one to a human body; the fall prediction image and the detection frame corresponding to the human body prediction image are used as operation data to execute a fall logic judgment algorithm to determine whether the current image to be predicted belongs to the fall event category, that is, the present invention improves the accuracy and robustness of fall event detection by combining the prediction results of the fall detection model and the target detection model to execute the fall logic judgment algorithm.
[0082] Embodiment 2
[0083] like Figure 2 As shown, the present invention also proposes a fall detection method for a companion robot, comprising:
[0084] Constructing a data set, and performing event category labeling and target recognition labeling on each image in the data set to obtain a target data set; the event categories include: fall event categories and non-fall event categories; the target recognition labeling includes: faller position labeling;
[0085] The fall detection model is obtained by training the Yolov8 model with the target dataset;
[0086] The image to be predicted is predicted by the fall detection model and the target detection model respectively, and a fall prediction image containing a fall detection frame and a human body prediction image containing a human body detection frame are output respectively; the fall detection frame corresponds one-to-one to a potential fall event; the human body detection frame corresponds one-to-one to a human body; the image to be predicted is a patrol image obtained by the accompanying robot; the target detection model is the yolov8m model trained by the coco dataset.
[0087] The detection frames corresponding to the fall prediction image and the human body prediction image are used as operation data to execute the fall logic judgment algorithm to determine whether the current image to be predicted belongs to the fall event category.
[0088] The fall prediction image and the detection frame corresponding to the human body prediction image are used as operation data to execute the fall logic judgment algorithm, specifically:
[0089] Obtain all detection frames in the fall prediction image and the human body prediction image, remove duplicate detection frames in the fall prediction image and the human body prediction image through a non-maximum suppression algorithm, and retain detection frames whose confidence exceeds a preset value;
[0090] Combining the fall detection frames retained in the fall prediction image with the human body detection frames retained in the human body prediction image in pairs to obtain multiple detection frame pairs;
[0091] Obtain the absolute difference in aspect ratio corresponding to each detection frame pair; the calculation formula for the absolute difference in aspect ratio is: ΔR = |R A -R B |; In the formula, R A Indicates the aspect ratio of the fall detection frame, R B represents the aspect ratio corresponding to the human body detection frame, and ΔR represents the absolute difference in aspect ratio; the detection frame pair whose absolute difference in aspect ratio is less than the preset value is marked as the first target category;
[0092] Calculate the distance between the center points of two detection frames in each detection frame pair, and mark the detection frame pairs whose center point distance is greater than a preset value as the second target category;
[0093] Calculate the intersection-and-union ratio corresponding to each detection frame pair, and mark the detection frame pairs whose intersection-and-union ratio is greater than a preset value as the third target category;
[0094] It is determined whether the same detection frame pair belongs to the first target category, the second target category, and the third target category at the same time. If so, the image to be predicted is marked as a fall event category.
[0095] The fall detection method further comprises:
[0096] When the image to be predicted is judged as a fall event category, the image to be predicted is segmented by the SAM model, and the segmented privacy area is occluded, and the image to be predicted after occlusion is sent to the preset customer; the input of the SAM model is the image to be predicted containing a fall detection frame and a human body detection frame.
[0097] The fall detection method further comprises:
[0098] The image to be predicted is encrypted by AES through an encryption transmission module arranged in the accompanying robot.
[0099] The present invention calculates the absolute difference in aspect ratios, the distance between the center points of the detection frames and the intersection and union ratio of the prediction results of the dual models (the fall detection model and the target detection model) in sequence, and judges whether the same detection frame pair belongs to the first target category, the second target category and the third target category at the same time, that is, triple calculation and judgment are performed on the detection frame pair at the same time. Only when the same detection frame pair belongs to the first target category, the second target category and the third target category at the same time, the image to be predicted is marked as the fall event category, which greatly improves the accuracy of fall detection, effectively reduces false alarms in complex home environments, and improves the robustness of fall detection.
[0100] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.
[0101] In addition, in the present invention, descriptions such as "first", "second", "one", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0102] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0103] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.
Claims
1. A fall detection system for a companion robot, characterized in that: include: The data set module is used to construct a data set and annotate each image in the data set with event categories and target recognition to obtain a target data set. The event categories include: fall event categories and non-fall event categories; the target identification annotation includes: faller position annotation; The training module is used to train the Yolov8 model using the target data set to obtain a fall detection model; A fall logic judgment module is used to predict the image to be predicted by using a fall detection model and a target detection model, and output a fall prediction image containing a fall detection frame and a human body prediction image containing a human body detection frame, respectively; the fall detection frame corresponds to a potential fall event one by one; the human body detection frame corresponds to a human body one by one; the image to be predicted is a patrol image acquired by the accompanying robot; The fall logic judgment module is also used to execute a fall logic judgment algorithm using the detection frames corresponding to the fall prediction image and the human body prediction image as operation data to determine whether the current image to be predicted belongs to the fall event category.
2. A fall detection system for a companion robot according to claim 1, characterized in that: The fall prediction image and the detection frame corresponding to the human body prediction image are used as operation data to execute the fall logic judgment algorithm, specifically: Obtain all detection frames in the fall prediction image and the human body prediction image, remove duplicate detection frames in the fall prediction image and the human body prediction image through a non-maximum suppression algorithm, and retain detection frames whose confidence exceeds a preset value; Combining the fall detection frames retained in the fall prediction image with the human body detection frames retained in the human body prediction image in pairs to obtain multiple detection frame pairs; Obtain the absolute difference in aspect ratio corresponding to each detection frame pair; the calculation formula for the absolute difference in aspect ratio is: ΔR = |R A -R B |; In the formula, R A Indicates the aspect ratio of the fall detection frame, R B represents the aspect ratio of the human body detection frame, and ΔR represents the absolute difference in the aspect ratio; Mark the detection frame pairs whose absolute difference of aspect ratio is less than a preset value as the first target category; Calculate the distance between the center points of two detection frames in each detection frame pair, and mark the detection frame pairs whose center point distance is greater than a preset value as the second target category; Calculate the intersection-and-union ratio corresponding to each detection frame pair, and mark the detection frame pairs whose intersection-and-union ratio is greater than a preset value as the third target category; It is determined whether the same detection frame pair belongs to the first target category, the second target category, and the third target category at the same time. If so, the image to be predicted is marked as a fall event category.
3. A fall detection system for a companion robot according to claim 2, characterized in that: The fall detection system further comprises: A privacy processing module, for segmenting the image to be predicted by a SAM model and performing occlusion processing on the segmented privacy area when the image to be predicted is judged to be a fall event category; the input of the SAM model is the image to be predicted containing a fall detection frame and a human body detection frame; The alarm module is used to send the image to be predicted after occlusion processing to the preset customer.
4. A fall detection system for a companion robot according to claim 3, characterized in that: The fall detection system further includes: an encryption transmission module disposed in the accompanying robot, for performing AES encryption on the predicted image and then transmitting the encrypted image to the fall logic judgment module.
5. A fall detection system for a companion robot according to claim 1, characterized in that: The target detection model is a yolov8m model trained with the coco dataset.
6. A fall detection method for a companion robot, characterized in that: include: Construct a data set, and annotate each image in the data set with event categories and target recognition to obtain the target data set; The event categories include: fall event categories and non-fall event categories; the target identification annotation includes: faller position annotation; The fall detection model is obtained by training the Yolov8 model with the target dataset; The image to be predicted is predicted by the fall detection model and the target detection model respectively, and a fall prediction image containing a fall detection frame and a human body prediction image containing a human body detection frame are output respectively; the fall detection frame corresponds to a potential fall event one by one; the human body detection frame corresponds to a human body one by one; the image to be predicted is a patrol image obtained by the accompanying robot; The detection frames corresponding to the fall prediction image and the human body prediction image are used as operation data to execute the fall logic judgment algorithm to determine whether the current image to be predicted belongs to the fall event category.
7. A fall detection method for a companion robot according to claim 6, characterized in that: The fall prediction image and the detection frame corresponding to the human body prediction image are used as operation data to execute the fall logic judgment algorithm, specifically: Obtain all detection frames in the fall prediction image and the human body prediction image, remove duplicate detection frames in the fall prediction image and the human body prediction image through a non-maximum suppression algorithm, and retain detection frames whose confidence exceeds a preset value; Combining the fall detection frames retained in the fall prediction image with the human body detection frames retained in the human body prediction image in pairs to obtain multiple detection frame pairs; Obtain the absolute difference in aspect ratio corresponding to each detection frame pair; the calculation formula for the absolute difference in aspect ratio is: ΔR = |R A -R B |; In the formula, R A Indicates the aspect ratio of the fall detection frame, R B represents the aspect ratio of the human body detection frame, and ΔR represents the absolute difference in the aspect ratio; Mark the detection frame pairs whose absolute difference of aspect ratio is less than a preset value as the first target category; Calculate the distance between the center points of two detection frames in each detection frame pair, and mark the detection frame pairs whose center point distance is greater than a preset value as the second target category; Calculate the intersection-and-union ratio corresponding to each detection frame pair, and mark the detection frame pairs whose intersection-and-union ratio is greater than a preset value as the third target category; It is determined whether the same detection frame pair belongs to the first target category, the second target category, and the third target category at the same time. If so, the image to be predicted is marked as a fall event category.
8. A fall detection method for a companion robot according to claim 7, characterized in that: The fall detection method further comprises: When the image to be predicted is judged as a fall event category, the image to be predicted is segmented by the SAM model, and the segmented privacy area is occluded, and the image to be predicted after occlusion is sent to the preset customer; the input of the SAM model is the image to be predicted containing a fall detection frame and a human body detection frame.
9. A fall detection method for a companion robot according to claim 8, characterized in that: The fall detection method further comprises: The image to be predicted is encrypted by AES through an encryption transmission module arranged in the accompanying robot.
10. A fall detection method for a companion robot according to claim 6, characterized in that: The target detection model is a yolov8m model trained with the coco dataset.
Citation Information
Cited By
Visual positioning method for intelligent health care and accompanying robot
CN120902025A
Falling detection method and device for accompanying robot
CN122135258A
