Fall prevention method, device and robot

Abstract

The application discloses a fall prevention method and device and a robot. The method comprises the following steps: in the process of following a target object, acquiring multi-modal data of a scene in which the target object is located; evaluating a risk parameter of the target object based on the multi-modal data; the risk parameter is used for reflecting a fall risk of the target object; generating a behavior strategy of the robot based on the risk parameter; and the behavior strategy is used for assisting the target object when the target object is about to fall.

Description

Technical Field

[0001] This application relates to the field of robotics, and more particularly to a method, device, and robot for preventing falls. Background Technology

[0002] With the increasing aging of the population and growing societal focus on the quality of life for the elderly, effectively ensuring their safety and health has become a pressing issue. In scenarios such as home-based care and community nursing, falls due to declining physical function, illness, or accidents are frequent among the elderly. This not only seriously threatens their lives and health but also places a heavy burden on families and society. Traditional fall prevention solutions employ single sensor technologies, such as accelerometers, gyroscopes, or infrared sensors. While these can achieve some degree of real-time fall detection, their limited data dimensions lead to a high false alarm rate. Furthermore, traditional fall prevention solutions only provide detection capabilities and lack the ability to actively intervene, failing to provide timely assistance when an elderly person is about to fall. Summary of the Invention

[0003] In view of this, embodiments of this application provide at least one method, device, robot, storage medium, and program product for preventing falls.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] On one hand, embodiments of this application provide a fall prevention method, the method comprising: acquiring multimodal data of the scene in which the target object is located during the process of following a target object; evaluating risk parameters of the target object based on the multimodal data; the risk parameters being used to reflect the fall risk of the target object; generating a behavior strategy of the robot based on the risk parameters; and the behavior strategy being used to assist the target object when it is about to fall.

[0006] On the other hand, this application provides a fall prevention device for a robot. The device includes: an acquisition module for acquiring multimodal data of the scene in which the target object is located during the process of following a target object; an evaluation module for evaluating risk parameters of the target object based on the multimodal data; and a generation module for generating a behavior strategy for the robot based on the risk parameters. The behavior strategy is used to help the target object when it is about to fall.

[0007] In another aspect, embodiments of this application provide a robot, including a multimodal sensor, a memory, and a processor, wherein the multimodal sensor is used to collect real-time multimodal data; the memory stores a computer program that can run on the processor; and the processor executes the program to implement some or all of the steps in the above method.

[0008] In another aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the above-described method.

[0009] In another aspect, embodiments of this application provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement some or all of the steps in the above-described method.

[0010] In this embodiment, by acquiring multimodal data during the process of following the target object, information about the target object and its surrounding environment can be obtained in real time and comprehensively, including visual images, sound, radar signals, infrared information, etc., which provides a rich data foundation for subsequent risk assessment. At the same time, assessing the risk parameters of the target object based on this multimodal data can accurately reflect the fall risk of the target object, providing a key basis for the robot's behavioral decisions. Furthermore, generating the robot's behavior strategy based on the risk parameters can quickly take support measures when the target object is about to fall, effectively preventing the occurrence of fall events.

[0011] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this application. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0013] Figure 1 A schematic diagram of the implementation process of a fall prevention method provided in this application embodiment. Figure 1 ;

[0014] Figure 2 A schematic diagram of the implementation process of a fall prevention method provided in this application embodiment. Figure 2 ;

[0015] Figure 3 A schematic diagram of the implementation process of a fall prevention method provided in this application embodiment. Figure 3 ;

[0016] Figure 4 A schematic diagram of the implementation process of a fall prevention method provided in this application embodiment. Figure 4 ;

[0017] Figure 5 A schematic diagram of the implementation process of a fall prevention method provided in this application embodiment. Figure 5 ;

[0018] Figure 6 A schematic diagram of the implementation process of a fall prevention method provided in this application embodiment. Figure 6 ;

[0019] Figure 7 A schematic diagram of the implementation process of a fall prevention method provided in this application embodiment. Figure 7 ;

[0020] Figure 8 This is a schematic diagram of the composition of an anti-fall device provided in an embodiment of this application;

[0021] Figure 9 This is a schematic diagram of the hardware entity of a robot provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. The terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application.

[0025] This application provides a fall prevention method, which can be executed by a robot's processor. The robot can be a device with autonomous mobility, integrated sensors, and data processing capabilities. These devices can perform complex tasks, including but not limited to navigation, path planning, object recognition and manipulation, and environmental perception.

[0026] In some embodiments, the robot can be a device that not only possesses basic data processing capabilities but also integrates sensors (such as cameras, lidar, ultrasonic sensors, gyroscopes, accelerometers, etc.), actuators (such as motors, wheels, robotic arms, etc.), and algorithms and software for controlling and optimizing these components. Exemplarily, the robot can be a service robot (such as a household cleaning robot, a restaurant delivery robot, a hospital care robot, etc.), an industrial robot (a robot that performs repetitive, high-precision processing, assembly, and handling tasks on a production line), an exploratory robot (for exploration and reconnaissance tasks in extreme environments or dangerous areas), or a mobile robot (autonomous vehicles, drones, etc.).

[0027] Figure 1 A schematic diagram of the implementation process of a fall prevention method provided in this application embodiment. Figure 1 ,like Figure 1 As shown, the method includes the following steps S101 to S103:

[0028] Step S101: During the process of following the target object, acquire multimodal data for the scene where the target object is located.

[0029] In some embodiments, the aforementioned multimodal data refers to multiple types of information acquired by the robot simultaneously through multimodal sensors (including different types of sensors), such as radar signals, infrared information, visual images, and sound. In the embodiments of this application, this multimodal data is integrated to more comprehensively understand and respond to the environment. It is understood that, since the target object is located in the environment, the multimodal data reflects not only environmental information but also information about the target object.

[0030] In some embodiments, the target object is an entity that the robot needs to monitor, track, or interact with. For example, the target object may be an individual that needs to be monitored to prevent falls, such as an elderly person, a patient, or a worker in a specific work environment. The scenario refers to the physical environment in which the target object is located; for example, the scenario may include a home environment, a medical facility, a workplace, etc.

[0031] In some embodiments, to comprehensively and accurately acquire multimodal data of the scene where the target object is located, the robot can integrate multiple sensors and data acquisition devices, such as cameras, microphones, radar, and infrared sensors, to capture various information about the target object and its scene in real time. In other embodiments, the aforementioned multiple sensors and data acquisition devices can exist independently of the robot. By establishing a communication channel between the robot and the sensors and data acquisition devices, various information about the target object and its scene can be acquired.

[0032] In some embodiments, all integrated sensors (and data acquisition devices) can be activated and data can be collected in real time as the robot follows the target object. The camera captures visual information of the target object, the microphone collects sound information, and the radar and infrared sensors provide position, speed and depth information of the target object, thereby obtaining the multimodal data.

[0033] For example, in a home environment, a robot is following an elderly person. The robot's camera captures the elderly person's posture data, the microphone collects the elderly person's voice and ambient noise, and radar and infrared sensors monitor the elderly person's position and movement speed. All of this data is collected in real time as multimodal data and transmitted to the robot's processing unit.

[0034] Step S102: Evaluate the risk parameters of the target object based on the multimodal data; the risk parameters are used to reflect the fall risk of the target object.

[0035] The risk parameters can include quantitative values ​​reflecting the likelihood of a target object (such as an elderly person) falling in its current state. These risk parameters are derived from in-depth analysis and processing of multimodal data and provide crucial information for the robot's behavioral decisions. Understandably, the level of the risk parameters directly reflects the severity of the target object's fall risk. Of course, fall risk refers to the possibility that a target object may fall due to various reasons (such as declining physical function, environmental factors, etc.). This application can assess the risk of falling due to declining physical function using relevant information about the target object in the multimodal data, and it can also assess the risk of falling due to environmental factors using relevant information about the environment in the multimodal data.

[0036] In some embodiments, the body contour and key point information of the target object can be extracted from the visual images in the multimodal data; a posture recognition algorithm is applied to analyze whether the body posture of the target object is stable, such as whether there are signs of swaying or loss of balance; based on the posture analysis results, the risk of the target object falling due to its own factors is assessed.

[0037] In some embodiments, by analyzing the object recognition results in visual images, risk factors in the environment that may cause a fall, such as obstacles and slippery surfaces, can be identified; by combining data such as radar signals, infrared information, and sound, the distance and relative position of the target object to surrounding objects can be monitored, and the risk of the target object falling due to environmental factors can be assessed.

[0038] In some embodiments, the risk parameters of the target object can be generated based on any one of the fall risks caused by its own factors or the fall risks caused by environmental factors, or a comprehensive risk parameter can be generated by combining the two types of fall risks.

[0039] Step S103: Generate the robot's behavior strategy based on the risk parameters; the behavior strategy is used to help the target object when it is about to fall.

[0040] In some embodiments, the behavioral strategy may include a sequence of actions formulated by the robot based on risk parameters, encompassing sub-tasks such as path planning, contact force control, and balance compensation. Generally, this behavioral strategy needs to meet real-time and safety constraints, such as rapidly adjusting its own posture and providing physical support when the target object becomes unbalanced.

[0041] In some embodiments, the process of a robot assisting a target object may include: the robot applying a controllable external force to the target object to restore its balance through an actuator such as a robotic arm, torso, or exoskeleton.

[0042] In this embodiment, by acquiring multimodal data during the process of following the target object, information about the target object and its surrounding environment can be obtained in real time and comprehensively, including visual images, sound, radar signals, infrared information, etc., which provides a rich data foundation for subsequent risk assessment. At the same time, assessing the risk parameters of the target object based on this multimodal data can accurately reflect the fall risk of the target object, providing a key basis for the robot's behavioral decisions. Furthermore, generating the robot's behavior strategy based on the risk parameters can quickly take support measures when the target object is about to fall, effectively preventing the occurrence of fall events.

[0043] Figure 2 This is a schematic diagram of the implementation process of a fall prevention method provided in an embodiment of this application. Figure 2 .based on Figure 1 , Figure 1 Step S102 can be updated to steps S201 to S202, combining Figure 2 The steps shown are explained.

[0044] Step S201: Detect the center of gravity change data of the target object based on the multimodal data.

[0045] In some embodiments, the center of gravity change data refers to a record of the change in the center of gravity position of the target object in space over time; the center of gravity change data is used to assess the stability, balance ability, and potential motion risks of the target object.

[0046] In some embodiments, multimodal data may include at least one of the following: laser data, depth images, and visual images. The laser data is acquired by a lidar device and contains three-dimensional coordinate information of object surfaces in the environment. The laser data is presented in the form of a point cloud, accurately reflecting the shape and position of objects (including target objects) in the environment. The value of each pixel in the depth image represents the depth of the corresponding point in the scene (i.e., its distance from the depth camera). The visual image is used to record the appearance features of objects in the scene, such as color, texture, and shape.

[0047] In some implementation scenarios, LiDAR can be used to collect laser data of the target object and its surrounding environment to generate a point cloud map. Simultaneously, depth images and visual images are acquired using depth cameras and conventional cameras, respectively. Then, the laser data is filtered and denoised to improve the accuracy of the point cloud, and the depth image is corrected and smoothed to reduce noise and errors. The visual image undergoes preprocessing, such as grayscale conversion and edge detection, to facilitate subsequent target recognition and tracking. In the visual image, computer vision algorithms (such as target detection and tracking algorithms) are used to identify the target object and track its positional changes within the image. Furthermore, combining the laser data and depth image, the position of the target object is further confirmed. Based on the identified target object, point cloud data belonging to that object is extracted from the laser data, and geometric algorithms (such as centroid algorithms) are used to calculate the current centroid position of the target object. Alternatively, depth data belonging to that object is extracted from the depth image, and a 3D model of the target object is constructed to calculate its current centroid position. Here, at least one of the two centroid positions mentioned above can be used to determine the final centroid position. The center of gravity position of the target object is continuously calculated over time to form a center of gravity change dataset, which is the center of gravity change data of the target object.

[0048] Step S202: Predict the risk parameters of the target object based on the center of gravity change data of the target object.

[0049] In some embodiments, risk parameters of a target object can be predicted from at least one of the following aspects: stability parameters, balancing ability parameters, and potential risk prediction parameters.

[0050] By analyzing the fluctuation amplitude and frequency of the center of gravity change data, the stability parameters of the target object can be evaluated.

[0051] Among them, the balance ability parameter is used to characterize the ability of a target object to maintain body stability when subjected to external disturbances. The balance ability of the target object can be evaluated by analyzing the trend of the change of center of gravity data over time, thus obtaining the balance ability parameter. For example, when the center of gravity of the target object shifts significantly during walking or standing, it may indicate that its balance ability is impaired.

[0052] In this way, historical behavioral data of the target object (such as walking speed, gait characteristics, etc.) and current center of gravity change data can be combined to build a prediction model using machine learning or deep learning algorithms, predict the future movement trend of the target object, and obtain the potential risk prediction parameters.

[0053] In some embodiments, the aforementioned risk parameters can be predicted using a time-series analysis-based prediction model. This prediction model can be, but is not limited to, vector machines (SVM), random forests (RF), neural networks (such as LSTM), etc.

[0054] In some embodiments, the prediction model can be trained using historical center of gravity change data and known risk parameters (such as fall records, stability scores, etc.) as a training set.

[0055] In some embodiments, after obtaining the trained prediction model, new center of gravity change data can be input into the trained prediction model to predict the risk parameters of the target object. The prediction model may include a stability parameter prediction network, a balance capability parameter prediction network, and a potential risk prediction parameter prediction network, as well as a fusion module that weights and fuses the outputs of each prediction network.

[0056] In this embodiment, a detection method based on multimodal data (which may include laser data, depth images, visual images, etc.) can comprehensively and accurately capture the center of gravity change data of a target object in a dynamic environment. This allows for the acquisition of data reflecting the stability and motion characteristics of the target object under different states. Simultaneously, using this center of gravity change data as input, a model can predict risk parameters, enabling a quantitative assessment of the target object's potential risks. Since the prediction considers not only changes in the center of gravity position but also the rate and direction of these changes, it provides a more comprehensive reflection of the target object's motion state and risk level.

[0057] Figure 3 This is a schematic diagram of the implementation process of a fall prevention method provided in an embodiment of this application. Figure 3 .based on Figure 1 , Figure 1 Step S102 can be updated to steps S301 to S303, combining Figure 3 The steps shown are explained.

[0058] Step S301: Identify obstacle information in the scene where the target object is located based on the multimodal data.

[0059] In some embodiments, the multimodal data includes depth images, laser point cloud data, and visual image data. In scenarios where a robot follows a target object, the target object is an object relatively close to the robot, while obstacles in the environment surrounding the target object are objects relatively far away from the robot. To fully utilize the advantages of different sensors, in this embodiment, a depth camera can accurately identify nearby target objects (such as elderly people) and obtain their posture information, such as walking posture and gestures. In this case, the laser point cloud data acquired by the lidar serves as an auxiliary reference, providing additional three-dimensional spatial information for the depth camera's identification, enhancing the accuracy and robustness of the identification. For distant obstacles, this embodiment can primarily rely on lidar for detection. Due to its long-range and high-precision characteristics, lidar can accurately scan and construct a three-dimensional map of the surrounding environment, including distant obstacles and terrain changes, thereby obtaining comprehensive obstacle information. In this case, the depth images acquired by the depth camera serve as an auxiliary reference, improving the comprehensiveness of environmental understanding. It is understood that visual image data in the multimodal data can serve as auxiliary information when acquiring obstacle and posture information, in order to improve the accuracy of obstacle and posture information.

[0060] In some embodiments, the obstacle information described above is a set of attributes, such as the type, relative position, size, and shape of objects in the scene where the target object (e.g., an elderly person) is located that potentially obstruct the movement of the target object. Obstacle information affects the risk parameters of the target object; conversely, it can also influence the robot's planning of a safe following path.

[0061] In some embodiments, during the process of identifying obstacle information, laser point cloud data is used as the primary source, supplemented by depth images and visual images, to address the issues of accuracy and efficiency in long-distance obstacle detection. Specifically, laser radar, with its long detection range (up to tens of meters) and high spatial resolution, can quickly scan a large area of ​​the environment and generate dense point clouds; depth images supplement fine depth information in local areas (such as the edges of steps) to help correct noise or missing information in the point cloud; and visual image data is used to provide semantic information (such as distinguishing between "walls" and "curtains"), reducing the probability of misidentifying harmless objects as obstacles.

[0062] In some embodiments, the environment can be scanned by LiDAR to generate point cloud data covering a certain range (e.g., 10-30 meters) in front of the robot; then, for low-density areas (e.g., narrow corners) in the LiDAR point cloud, depth images are used to supplement local details (e.g., the outlines of small obstacles); the visual image data is identified by a semantic segmentation model (e.g., Mask R-CNN) to obtain obstacle categories (e.g., "chair" or "threshold") and associated with the point cloud location.

[0063] Step S302: Identify the pose information of the target object based on the multimodal data.

[0064] In some embodiments, the above-mentioned posture information refers to the real-time state of various parts of the target object in three-dimensional space. Taking the target object as a human being as an example, the posture information may include the coordinates of skeletal key points (such as the head, shoulders, and hips), joint angles, and movement speeds.

[0065] In some embodiments, during the process of recognizing pose information, depth images are used as the primary source, supplemented by laser point cloud data and visual images, to address the accuracy and robustness issues of near-field pose estimation. Specifically, depth images provide high-resolution near-field human contour information, enabling accurate extraction of skeletal key points; laser point cloud data is used to assist in correcting occluded areas in the depth image and provides absolute scale calibration (avoiding proportional errors from the depth sensor); and visual image data provides color and texture information to aid in the acquisition of human contour information.

[0066] In some embodiments, a depth image of the target object can be captured by a depth camera. Generally, the target object is within the near field range of the depth camera (e.g., within 1-3 meters). For the acquired visual image data, the extracted color and texture information of the target object is combined with the captured depth image to determine the human body contour information of the target object. The above-mentioned skeletal key points are obtained based on the human body contour information. High-density point cloud sampling is performed on the area around the target object (1-3 meters). After registration with the depth image, the three-dimensional coordinates of the skeletal key points are corrected, and the coordinates of the skeletal key points can be obtained. The joint angles can be determined based on the coordinates of the skeletal key points, and the movement speed can be determined based on the relative positional differences between the coordinates of the skeletal key points at multiple times.

[0067] Step S303: Using the obstacle information and the posture information, generate the risk parameters of the target object.

[0068] The risk parameter of the target object is a comprehensive indicator of the likelihood of the target object falling due to its own postural imbalance or interference from environmental obstacles. As described in the above embodiments, the risk parameter can be expressed as a quantitative value. In some embodiments, the risk parameter can be expressed as a probability value (e.g., 0%-100%) or a risk level (e.g., low, medium, high).

[0069] In this application embodiment, to improve the accuracy of risk parameter assessment, a dynamic modeling method that combines the target object's own state with environmental threats can be used to predict the risk of falling in complex scenarios. For example, this application mainly considers the following exemplary scenario: even if the target object's posture is temporarily stable, if there is slippery ground or steps (obstacle information) in front, risk parameters with a higher risk still need to be generated.

[0070] In some embodiments, the balance state of the target object can be assessed first based on posture information (including the coordinates of key points of the skeleton, joint angles, and movement speed); based on obstacle information (including the type of object, its relative position to the target object, size, shape, etc.), the potential interference of obstacles in the environment on the target object's movements can be assessed; using data-driven models (such as neural networks) in related technologies, the interaction between the target object's posture change trend and the spatial distribution of obstacles can be analyzed to predict the probability of the target object falling in the next few seconds, thereby generating the risk parameter.

[0071] In this embodiment, by identifying obstacle information in the scene where the target object is located based on multimodal data, the complementary advantages of depth images, laser point cloud data, and visual image data can be fully utilized to accurately obtain key attributes such as the type, location, size, and shape of objects in the environment that may pose a potential obstacle to the movement of the target object, thereby improving the understanding of the surrounding environment and the accuracy of obstacle detection. At the same time, by identifying the posture information of the target object through multimodal data, the posture changes of the target object in three-dimensional space can be captured in real time, including the coordinates of key points of the skeleton, joint angles, and movement speed, providing an important basis for assessing the balance state of the target object. Based on this detailed obstacle and posture information, the risk parameters of the target object can be generated using a data-driven model, which can comprehensively consider the target object's own state and environmental threats, predict the risk of falling in complex scenarios, and provide strong support for taking timely preventive measures.

[0072] In some embodiments, the above-mentioned behavioral strategies include at least one of the following: the robot's position movement strategy and the robot's posture change strategy.

[0073] The position movement strategy is used to control the robot to move to the target location so that the robot can perform a support action on the target object at the target location; the posture change strategy is used to control the robot to switch to the target posture so that the robot can perform a support action on the target object in the target posture.

[0074] In some embodiments, a positional movement strategy is used to adjust the robot's position in physical space to reach the optimal support position (such as the side, rear, or near a support point of the target object), ensuring that physical intervention can be applied quickly and stably when the target object becomes unbalanced. This positional movement strategy can be generated by the robot through path planning and motion control algorithms.

[0075] In some embodiments, the target orientation may include the robot's target direction relative to the target object. This target direction is determined based on center of gravity trajectory prediction. Based on the target object's posture information, the robot can predict its center of gravity landing point within the next 1-2 seconds to determine the target direction in which it needs to move. For example, if the predicted center of gravity shifts to the left front, the robot needs to move to a position 45° to the left front. In an example scenario, when an elderly person's center of gravity shifts to the right front and their right foot is about to step onto the slippery ground, the robot determines through trajectory prediction that they may fall to the right front, then plans a path around the sofa on the left, moving to a position 1 meter to the right front to prepare to provide support from the right side.

[0076] In some embodiments, the posture change strategy is used to adjust the robot's own mechanical structure (such as the joint angle of the robotic arm, chassis height, etc.) to switch to a physical posture suitable for performing the assistance action, ensuring that the direction of force application matches the direction of imbalance of the target object, and maintaining its own stability.

[0077] In some embodiments, the optimal force application angle and contact point position can be calculated based on the target object's imbalance direction (e.g., leaning backward or tilting sideways) and force analysis. For example, a backward fall requires a robotic arm to lift the object from behind, with the force applied vertically upward; a side fall requires lateral support from under the armpits. It should be noted that, based on the robot's dynamics model, the projected center of gravity of the robot in the adjusted posture can be positioned within the supporting polygon (e.g., the wheelbase range of a four-wheeled robot) to prevent tipping over during force application. In an example scenario, when an elderly person leans to the left due to weakness in their left leg, the robot quickly lowers its chassis height (reducing the center of gravity height), extends its left robotic arm to under the armpit and maintains a 15° tilt angle (opposite to the direction of the fall), and slightly opens its right robotic arm to balance the reaction force.

[0078] In this embodiment, the positional movement strategy ensures that the robot can safely and accurately move to the target location in complex environments, preparing for the assistance action. Simultaneously, the posture change strategy ensures that the robot provides stable and comfortable support when performing the assistance task. Through these strategies, the robot can significantly reduce the risk of fall injury to the target object in complex dynamic scenarios.

[0079] Figure 4 This is a schematic diagram of the implementation process of a fall prevention method provided in an embodiment of this application. Figure 4 .based on Figure 1 When the behavior strategy includes a positional movement strategy, the target orientation includes the distance between the robot and the target object and the robot's orientation; Figure 1 Step S103 can be updated to steps S401 to S403, combining Figure 4 The steps shown are explained.

[0080] Step S401: Determine the distance between the robot and the target object based on the risk parameter; the distance is negatively correlated with the risk parameter.

[0081] The distance between the robot and the target object is a three-dimensional spatial interval between them, typically calculated using Euclidean distance (meters). In assistance scenarios, this distance needs to be dynamically adjusted to balance safety monitoring and emergency intervention requirements. In this embodiment, a higher risk parameter results in a smaller distance between the robot and the target object; conversely, a lower risk parameter results in a larger distance.

[0082] In some embodiments, to address the intervention delay or excessive interference caused by a fixed robot following distance, the distance can be adjusted in real time using risk parameters. In cases of high risk, the distance can be shortened to increase intervention speed, while in cases of low risk, a safe distance can be maintained to reduce the feeling of pressure. For example, the risk parameters can be converted into a distance setpoint using a linear or nonlinear mapping function (such as an exponential decay function).

[0083] For example, in low-risk situations (0%–30%), the default distance is maintained at 1.5 meters; in medium-risk situations (30%–70%), the distance is dynamically adjusted to 1.5–0.8 meters; and in high-risk situations (70%–100%), the minimum safe distance of 0.5 meters is locked. In one example scenario, when an elderly person walks on a wet ground, causing the risk parameter to rise to 80%, the robot gradually approaches from an initial distance of 1.2 meters to 0.6 meters, ready to provide support at any time.

[0084] Step S402: Determine the orientation between the robot and the target object based on the risk parameter; the angle difference between the orientation and the direction in which the robot points to the target object is negatively correlated with the risk parameter.

[0085] Wherein, the orientation between the robot and the target object refers to the spatial orientation of the robot chassis or robotic arm actuator relative to the target object; the angle difference is the angle between the robot's current orientation and the direction pointing towards the target object. In this embodiment, a higher risk parameter results in a smaller angle difference (the robot's orientation needs to be more precisely aligned with the target object); a lower risk parameter results in a larger angle difference.

[0086] In some embodiments, in order to solve the problem of failure to provide timely assistance due to robot orientation deviation, the angle difference can be adjusted by risk parameters. When the risk is high, the robot is forced to orient itself precisely in the direction of the target object's imbalance (e.g., leaning back corresponds to the robot's front facing the target's back).

[0087] For example, in the case of low risk (0% to 30%), the allowable angle difference is ≤30°; in the case of medium risk (30% to 70%), the allowable angle difference is ≤15°; and in the case of high risk (70% to 100%), the allowable angle difference is ≤5°.

[0088] Step S403: Generate the position movement strategy based on the distance between the robot and the target object and the robot's orientation.

[0089] The position movement strategy includes parameters such as the robot's target position, movement path, and speed curve, which are used to dynamically adjust distance and orientation.

[0090] In some embodiments, an optimal positional movement strategy is calculated by an algorithm based on the distance between the robot and the target object and the robot's orientation. This strategy aims to ensure that the robot maintains a safe and effective distance when approaching the target object and approaches with an appropriate orientation to perform subsequent operations or tasks. By comprehensively considering distance and orientation, the robot's movement path can be optimized, unnecessary movements can be reduced, and task execution efficiency can be improved.

[0091] In this embodiment, by dynamically adjusting the distance between the robot and the target object, the incidence of collisions and other potential risk events can be effectively reduced, improving the robot's safety in complex environments. At the same time, by optimizing the robot's orientation, this step makes the robot more efficient and accurate in interacting with the target object, reducing operational errors or risks caused by improper orientation. Based on these adjustments, the generated position movement strategy (including parameters such as target position, movement path, and speed curve) can achieve precise monitoring and efficient intervention of the robot in assistance scenarios.

[0092] Figure 5 This is a schematic diagram of the implementation process of a fall prevention method provided in an embodiment of this application. Figure 5 .based on Figure 1 The position movement strategy satisfies the following range; the method further includes steps S501 to S502, combining... Figure 5 The steps shown are explained.

[0093] Step S501: Obtain the data acquisition range of each sensing device corresponding to the multimodal data; the data acquisition range is determined based on the acquisition method of the sensing device for the target object and the setting position of the sensing device in the robot.

[0094] The data acquisition range refers to the spatial area in which the sensing device can effectively acquire data on the target object or environment, and is determined by the physical characteristics of the sensor (such as field of view and detection distance) and its installation location. For example, when the horizontal scanning angle of the lidar is 270° and the detection distance is 20 meters, its data acquisition range can be described as a fan-shaped three-dimensional space with a radius of 20 meters centered on the sensor.

[0095] In some embodiments, the effective data acquisition range of each sensor can be calculated through geometric modeling based on the physical parameters (field of view, resolution, detection distance) and installation location (e.g., top or front of the robot) of the sensor.

[0096] Step S502: Based on the data acquisition range corresponding to each of the sensing devices and the data acquisition weight of each of the sensing devices, generate a following range with the target object as the reference.

[0097] Wherein, at any position within the following range, there is at least one sensor whose acquisition range includes the target object.

[0098] The robot following range is the spatial area within which the robot can maintain effective monitoring and interaction while following a target object. This following range is determined based on the data acquisition range and / or data acquisition weight of each sensor device. The data acquisition weight of a sensor device represents the importance or reliability of the data collected by that device in the decision-making process, and is used to characterize the quality and reliability of the data collected by that device. The position movement strategy is used to control the robot's movement within the data acquisition range.

[0099] To address the challenge of determining a tracking range that effectively covers the target object while ensuring efficient monitoring when a robot is following it, step S502 constructs a tracking range based on the target object by comprehensively considering the data acquisition range and weights of each sensor. This tracking range allows the robot to effectively perceive and monitor the target object through at least one sensor, regardless of its position during the tracking process, thereby improving the accuracy and reliability of the tracking.

[0100] In some embodiments, a sub-following range centered on the target object can be created for each sensing device based on its data acquisition range. This sub-following range represents the area where the sensing device can effectively sense and monitor the target object. The following range is obtained by merging the sub-following ranges of all sensing devices.

[0101] In some embodiments, after obtaining the above-mentioned following range, the position movement strategy can be generated by combining the distance between the robot and the target object and the orientation of the robot. In this embodiment, at least one candidate position that satisfies the distance between the target object and the orientation of the robot can be determined within the above-mentioned following range. Then, based on the data acquisition weights of the sensing devices where each target position is located, a target position is determined from at least one candidate position. The position movement strategy is then generated based on the target position and a preset path planning strategy.

[0102] In this embodiment, by acquiring the data acquisition range of each sensor corresponding to multimodal data, the perception boundary of each sensor can be accurately defined based on the sensor's acquisition method of the target object and its placement position in the robot. Simultaneously, based on these data acquisition ranges and the data acquisition weight of each sensor, a following range with the target object as the baseline can be comprehensively evaluated and generated. This ensures that at any position of the robot within this following range, the target object can be effectively covered by at least one sensor, improving the accuracy and reliability of the following. Based on the embodiments provided in this application, the robot can achieve more intelligent and flexible following of the target object, optimizing the information perception and processing capabilities during the following process.

[0103] In some embodiments, the detection index value of the robot at any position within the following range meets a preset threshold; the detection index value is determined by the following method: obtaining the detection weight corresponding to each of the sensing devices; and weighting and summing the sub-index values ​​corresponding to each sensing device based on the detection weight corresponding to each sensing device to obtain the detection index value.

[0104] The detection index value is determined based on the sub-index value corresponding to each of the sensing devices. When the robot is within the data acquisition range of the sensing device, the sub-index value is positive; when the robot is outside the data acquisition range of the sensing device, the sub-index value is negative.

[0105] In some embodiments, the detection index value is a comprehensive quantitative indicator used to measure the detection effectiveness of the robot's position relative to the data acquisition range of each sensor device when following a target object. This detection index value is calculated by comprehensively considering the detection weights of each sensor device and their corresponding sub-index values, and is used to evaluate whether the robot's detection performance at any position within the following range meets preset requirements.

[0106] In some embodiments, the detection weight is a relative importance or reliability index assigned to each sensing device when constructing the detection index value. This weight is usually determined based on the detection performance parameters of the sensing device (such as detection distance, accuracy, stability, etc.) and the specific requirements of the application scenario, and is used to reflect the degree of contribution of different sensing devices to the overall detection performance in the weighted summation process.

[0107] The sub-index value is a basic quantitative value reflecting the robot's detection status within or outside the data acquisition range of a specific sensing device. When the robot is within the data acquisition range of the sensing device, the sub-index value is positive, indicating that the sensing device can effectively detect the target object; conversely, when the robot is outside the data acquisition range, the sub-index value is negative, indicating that the sensing device cannot effectively detect the target object. In some embodiments, the closer the robot is to the target object, the larger the sub-index value; conversely, the farther the robot is from the target object, the smaller the sub-index value.

[0108] To ensure that the robot's detection performance meets a preset threshold requirement at any position within the following range, this application embodiment calculates a comprehensive detection index value by comprehensively considering the detection weights of each sensor device and their corresponding sub-index values. This value is then compared with a preset threshold. This allows the robot to maintain a certain standard of detection performance regardless of its position during the following process, thereby improving the accuracy and reliability of the following.

[0109] In this embodiment, by obtaining the detection weight corresponding to each sensing device, the characteristics of each sensing device and its effectiveness in different scenarios can be fully utilized, providing an accurate weight allocation basis for subsequent weighted summation. Simultaneously, by weighted summing the sub-index values ​​corresponding to each sensing device based on these detection weights, a detection index value that comprehensively reflects the robot's detection performance at any position within the following range can be obtained. The positive or negative setting of the sub-index value can intuitively reflect whether the robot is within the effective data acquisition range of the sensing device, thereby enhancing the sensitivity and practicality of the detection index value. Furthermore, by setting a preset threshold and comparing it with the detection index value, the robot's following strategy can be flexibly adjusted to ensure that the robot maintains effective detection of the target object throughout the following process.

[0110] Figure 6 This is a schematic diagram of the implementation process of a fall prevention method provided in an embodiment of this application. Figure 6 .based on Figure 1 The method further includes steps S601 to S603, combining... Figure 6 The steps shown are explained.

[0111] Step S601: Execute the behavior strategy.

[0112] In some embodiments, the above-mentioned behavioral strategy includes at least one of the following: a position movement strategy of the robot and a posture change strategy of the robot. The position movement strategy controls the robot to move to a target location, so that the robot can perform a supporting action on the target object at the target location; the posture change strategy controls the robot to switch to a target posture, so that the robot can perform the supporting action on the target object in the target posture. The posture change strategy adjusts the robot's own mechanical structure (such as the joint angle of the robotic arm, chassis height, etc.) to switch to a physical posture suitable for performing the supporting action, ensuring that the direction of force application matches the direction of imbalance of the target object, and maintaining its own stability.

[0113] Understandably, the aforementioned behavioral strategies are intended to enable the robot to provide rapid and accurate assistance to the target object at critical moments.

[0114] For example, the robot continuously monitors the elderly person's movements and postures using cameras and sensors to assess their fall risk, obtain risk parameters, and initiate a behavioral strategy execution program based on these parameters. This includes: according to a location movement strategy, the robot plans and executes movement commands to quickly move from its current position to the vicinity of the elderly person, ensuring it can provide assistance from the optimal position. Simultaneously, according to a posture change strategy, the robot adjusts the joint angles of its robotic arm and the height of its chassis to switch to the most suitable posture for assisting the elderly person. Ultimately, the robot successfully prepares to assist the elderly person before they fall.

[0115] Step S602: In response to the risk parameters meeting the preset risk conditions, a support action is generated based on the current pose of the robot and the current pose of the target object.

[0116] The preset risk condition can be that the risk parameter exceeds a preset risk threshold, for example, the probability of falling is ≥70%. Pose is a comprehensive description of an object's position and orientation in space. In this embodiment, the robot's current pose and the target object's current pose are important references for generating the assisting action. The assisting action is a series of actions or posture adjustments taken by the robot to help the target object prevent a fall. In some embodiments, the assisting action may include moving to the vicinity of the target object, adjusting the robot's posture to match the target object's imbalance state, applying appropriate force to stabilize the target object, etc.

[0117] In some embodiments, while following an elderly person, the robot monitors risk parameters in real time to see if they meet preset risk conditions. Once the conditions are met, the robot immediately initiates a pose acquisition program, using integrated sensors (such as cameras, LiDAR, and inertial measurement units) to acquire real-time pose information of itself and the target object. Subsequently, the robot uses pose analysis algorithms, combined with the target object's imbalance state, its own mechanical structure, and mobility, to generate appropriate assistance actions.

[0118] Step S603: Perform the supporting action.

[0119] The assisting action includes at least one of the following: shortening the distance between the robot and the target object; changing the robot's actions to match the current posture of the target object.

[0120] In this embodiment, to prevent the target object from falling, the robot can reduce the spatial distance between itself and the target object by moving its position during the assistance action, so as to provide more direct and effective support and stability; and / or, adjust its own movement pattern according to the target object's current posture and movement to better adapt to and cooperate with the target object's needs. This includes adjusting the robot's movement trajectory, speed, force, and other parameters to ensure the coordination and effectiveness of the assistance action.

[0121] In this embodiment, by executing the behavioral strategy, the robot's action plan can be flexibly adjusted according to the state of the target object and environmental conditions, providing suitable prerequisites for subsequent assistance actions, thereby improving the effectiveness of robot assistance to a certain extent. Simultaneously, in response to the risk parameters meeting preset risk conditions, assistance actions are generated based on the robot's current pose and the target object's current pose. This allows for real-time risk assessment and the formulation of appropriate assistance strategies, helping the robot provide assistance more accurately and reducing adverse consequences caused by misjudgment or delayed response. When performing assistance actions, by shortening the distance between the robot and the target object and changing the robot's actions to match the target object's current posture, physical support can be provided more effectively to the target object, maintaining its stability while enhancing the coordination and comfort of the actions.

[0122] Figure 7 This is a schematic diagram of the implementation process of a fall prevention method provided in an embodiment of this application. Figure 7 .based on Figure 1 The method further includes steps S701 to S703, combining... Figure 7 The steps shown are explained.

[0123] Step S701: In response to the risk parameters meeting the preset risk conditions, store the collected multimodal data.

[0124] In some embodiments, while generating the assisting action based on the robot's current pose and the target object's current pose, the collected multimodal data can be stored. During real-time monitoring of risk parameters, once a risk parameter is detected to reach or exceed a preset risk condition, a data storage program is immediately initiated to save the currently collected multimodal data (such as video recordings, audio recordings, sensor data, etc.) to a designated database on the local machine or server.

[0125] Step S702: When the multimodal data characterizes the target object as having fallen, generate event record data corresponding to the fall event based on the stored multimodal data.

[0126] In some embodiments, when it is necessary to determine whether a target object has fallen, the stored multimodal data is first loaded, and then the analysis is performed based on the multimodal data. If it is confirmed that a fall has occurred, detailed event record data is generated based on the data timestamp, the target object's state information, and environmental conditions, and saved to the event log database.

[0127] Step S703: If the multimodal data indicates that the target object has not experienced a fall event, delete the stored multimodal data.

[0128] In some embodiments, multimodal data that does not characterize a fall event can be automatically cleaned up after a preset time period. For example, stored multimodal data can be periodically checked, and data records not associated with a fall event can be deleted to free up storage space.

[0129] In this embodiment, by responding to risk parameters and meeting preset risk conditions, the collected multimodal data is stored in a timely manner. This ensures that key information can be retained for subsequent analysis when the target object may be at risk of falling. At the same time, when the multimodal data clearly indicates that the target object has fallen, detailed event record data is generated based on this data, which can accurately record the specific circumstances of the fall event and provide strong support for subsequent emergency response and accident analysis. When the multimodal data does not show that the target object has fallen, timely deletion of this data can effectively free up storage space and avoid unnecessary resource occupation.

[0130] In some embodiments, when the multimodal data characterizes a fall event of the target object, the method further includes steps S704 and S705.

[0131] Step S704: Assess the injury level of the target object based on the event record data.

[0132] In some embodiments, the event recording data includes multimodal data corresponding to a fall event. This multimodal data includes not only the relevant data of the target object collected after the fall, but also the multimodal data of the target object before and during the fall. In this way, the entire fall event of the target object can be analyzed based on the multimodal data collected throughout the fall process to obtain a more accurate injury level.

[0133] The injury level is a classification based on the extent of injury suffered by the target in the fall. This injury level can be determined based on medical standards and experience to guide subsequent rescue measures.

[0134] In some embodiments, the injury level may include: minor injury, where the target falls but is able to stand up on their own or with only slight assistance, without obvious external injury or pain; moderate injury, where the target falls but is unable to stand up on their own and requires assistance from others, or is accompanied by minor external injury or pain; severe injury, where the target falls but is accompanied by severe external injury or pain, or suspected fractures, internal organ damage, etc., requiring immediate first aid and prompt transport to a hospital for professional treatment; and critical injury, where the target falls but exhibits severe abnormal vital signs such as loss of consciousness or respiratory arrest, falling under the category of emergency rescue, requiring immediate initiation of emergency procedures to save lives.

[0135] Step S705: Implement the corresponding level of rescue strategy based on the injury level.

[0136] The rescue strategy includes at least one of the following: generating rescue information and sending it to the rescued object at the corresponding rescue level; sending the event record data to the rescued object at the corresponding rescue level; and executing the lift-up strategy.

[0137] The rescue targets differ depending on the injury level. For example, rescue targets may include medical personnel, family members, etc. When the injury level is low, the corresponding rescue target may be family members; when the injury level is high, the corresponding rescue target may be medical personnel.

[0138] In some embodiments, when the injury level is minor, the rescue targets are family members or caregivers. The corresponding rescue strategy is to send an alarm message to the family member's mobile phone containing the time, location, and brief description of the fall; simultaneously, the robot performs a help-up strategy. If the injury is determined to be moderate, the rescue targets are community doctors and family members. The strategy includes sending event recording data such as video of the fall and impact force curves to the community doctor's terminal. After the robot assists in helping the person up, it monitors indicators such as heart rate and respiration in real time. For severe injuries, the rescue targets are emergency centers and hospitals. An emergency call is automatically triggered, and the patient's location, basic information, and multimodal data (such as thermal imaging temperature maps and joint torsion angle analysis) are sent. For critical injuries, the rescue targets include emergency centers and the nearest AED device administrator. The robot can guide nearby people to retrieve AED devices through indoor navigation, or perform standardized chest compressions through a robotic arm. Vital sign data is transmitted back to the emergency platform in real time for doctors to provide remote guidance.

[0139] In this embodiment, the injury level of the target object can be objectively and accurately assessed based on event recording data (such as fall videos, sensor data, etc.), thereby providing a strong reference for subsequent rescue. At the same time, the corresponding rescue strategy can be quickly matched and executed according to the assessed injury level. For example, for minor injuries, it may only be necessary to send simple rescue information to the family, while for severe or critical injuries, an emergency rescue process needs to be initiated immediately to ensure that rescue measures can be implemented in a targeted manner.

[0140] Based on the foregoing embodiments, this application provides an anti-fall device, which includes various units and modules included in each unit, and can be implemented by a processor in a computer device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0141] Figure 8 This is a schematic diagram of the composition of an anti-fall device provided in an embodiment of this application, as shown below. Figure 8 As shown, the fall prevention device 800 includes: an acquisition module 810, an evaluation module 820, and a generation module 830, wherein:

[0142] The acquisition module 810 is used to acquire multimodal data of the scene where the target object is located during the process of following the target object;

[0143] Evaluation module 820 is used to evaluate the risk parameters of the target object based on the multimodal data;

[0144] The generation module 830 is used to generate a behavior strategy for the robot based on the risk parameters; the behavior strategy is used to help the target object when it is about to fall.

[0145] In some embodiments, the evaluation module 820 is further configured to detect the center of gravity change data of the target object based on the multimodal data; and predict the risk parameters of the target object based on the center of gravity change data of the target object.

[0146] In some embodiments, the multimodal data includes depth data, and the evaluation module 820 is further configured to identify obstacle information in the scene where the target object is located based on the multimodal data; identify the posture information of the target object based on the multimodal data; and generate risk parameters of the target object using the obstacle information and the posture information.

[0147] In some embodiments, the behavioral strategy includes at least one of the following: the robot's position movement strategy and the robot's posture change strategy; wherein, the position movement strategy is used to control the robot to move to a target location so that the robot can perform a supporting action on the target object at the target location; the posture change strategy is used to control the robot to switch to a target posture so that the robot can perform a supporting action on the target object in the target posture.

[0148] In some embodiments, when the behavior strategy includes a positional movement strategy, the target orientation includes the distance between the robot and the target object and the robot's orientation; the generation module 830 is further configured to determine the distance between the robot and the target object based on the risk parameter; the distance is negatively correlated with the risk parameter; determine the orientation between the robot and the target object based on the risk parameter; the angle difference between the orientation and the direction the robot points towards the target object is negatively correlated with the risk parameter; and generate the positional movement strategy based on the distance between the robot and the target object and the robot's orientation.

[0149] In some embodiments, the position movement strategy satisfies a following range; the generation module 830 is further configured to obtain the data acquisition range of each sensing device corresponding to the multimodal data; the data acquisition range is determined based on the acquisition method of the sensing device for the target object and the setting position of the sensing device in the robot; based on the data acquisition range corresponding to each sensing device and the data acquisition weight of each sensing device, a following range with the target object as the reference is generated; wherein, at any position of the robot in the following range, at least one sensing device has an acquisition range that includes the target object.

[0150] In some embodiments, the detection index value of the robot at any position within the following range meets a preset threshold; the generation module 830 is further configured to obtain the detection weight corresponding to each of the sensing devices; and to perform a weighted summation of the sub-index values ​​corresponding to each of the sensing devices based on the detection weights corresponding to each of the sensing devices to obtain the detection index value; wherein, the detection index value is determined based on the sub-index values ​​corresponding to each of the sensing devices, and the sub-index value is positive when the robot is within the data acquisition range corresponding to the sensing device; and negative when the robot is outside the data acquisition range corresponding to the sensing device.

[0151] In some embodiments, the anti-fall device 800 includes an execution module, which is configured to execute the behavior strategy; in response to the risk parameter satisfying a preset risk condition, generate a support action based on the current pose of the robot and the current pose of the target object; and execute the support action; wherein the support action includes at least one of the following: shortening the distance between the robot and the target object; or changing the robot's action to match the current pose of the target object.

[0152] In some embodiments, the execution module is configured to, in response to the risk parameter meeting a preset risk condition, store the collected multimodal data; if the multimodal data indicates that the target object has fallen, generate event record data corresponding to the fall event based on the stored multimodal data; and if the multimodal data indicates that the target object has not fallen, delete the stored multimodal data.

[0153] In some embodiments, when the multimodal data characterizes a fall event of the target object, the execution module is configured to assess the injury level of the target object based on the event recording data; execute a corresponding level of rescue strategy based on the injury level; the rescue strategy includes at least one of the following: generating rescue information and sending it to the rescue object of the corresponding rescue level; sending the event recording data to the rescue object of the corresponding rescue level; and executing a help-up strategy.

[0154] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this application can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the apparatus embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0155] It should be noted that, in the embodiments of this application, if the above-mentioned anti-fall method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0156] This application provides a robot including a multimodal sensor, a memory, and a processor. The multimodal sensor is used to collect real-time multimodal data. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above method.

[0157] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium can be transient or non-transient.

[0158] This application provides a computer program including computer-readable code, wherein when the computer-readable code is executed in a computer device, a processor in the computer device performs some or all of the steps in the above-described method.

[0159] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0160] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0161] Figure 9 This is a schematic diagram of the hardware entity of a robot provided in an embodiment of this application, such as... Figure 9 As shown, the hardware entity of the robot 900 includes: a processor 901, a memory 902, and a multimodal sensor 903. The memory 902 stores a computer program that can run on the processor 901. When the processor 901 executes the program, it implements the steps in the method of any of the above embodiments. The multimodal sensor 903 is used to collect real-time multimodal data.

[0162] The memory 902 stores computer programs that can run on the processor. The memory 902 is configured to store instructions and applications that can be executed by the processor 901. It can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) of the processor 901 and various modules in the robot 900. It can be implemented by flash memory or random access memory (RAM).

[0163] When processor 901 executes the program, it implements the steps of any of the above-mentioned anti-fall methods. Processor 901 typically controls the overall operation of robot 900.

[0164] This application provides a computer storage medium storing one or more programs that can be executed by one or more processors to implement the steps of the fall prevention method as described in any of the above embodiments.

[0165] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0166] The aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this application does not specifically limit the specific implementation.

[0167] The aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0168] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0169] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0170] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0171] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0172] Furthermore, in the various embodiments of this application, all functional units can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0173] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) corresponding to a robot to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0174] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for preventing falls, characterized in that, Applied to robots, the method includes: During the process of following the target object, multimodal data for the scene in which the target object is located is acquired; The risk parameters of the target object are evaluated based on the multimodal data; the risk parameters are used to reflect the fall risk of the target object. The robot's behavior strategy is generated based on the risk parameters; the behavior strategy is used to help the target object when it is about to fall, and the behavior strategy includes a positional movement strategy that meets the following range. The method further includes: Obtain the data acquisition range of each sensing device corresponding to the multimodal data; the data acquisition range is determined based on the acquisition method of the sensing device for the target object and the setting position of the sensing device in the robot; Based on the data acquisition range corresponding to each of the aforementioned sensing devices and the data acquisition weight of each of the aforementioned sensing devices, a following range is generated with the target object as the reference. Wherein, at any position within the following range, at least one sensor device of the robot has a data acquisition range that includes the target object; the detection index value of the robot at any position within the following range satisfies a preset threshold; the detection index value is determined by the following method: Obtain the detection weight corresponding to each of the aforementioned sensing devices; The sub-index values ​​corresponding to each of the aforementioned sensing devices are weighted and summed based on the detection weights corresponding to each sensing device to obtain the detection index value. The detection index value is determined based on the sub-index value corresponding to each of the sensing devices. When the robot is within the data acquisition range of the sensing device, the sub-index value is positive; when the robot is outside the data acquisition range of the sensing device, the sub-index value is negative.

2. The method according to claim 1, characterized in that, The multimodal data includes depth data, and the risk parameters of the target object assessed based on the multimodal data include at least one of the following: Based on the multimodal data, detect the center of gravity change data of the target object, and predict the risk parameters of the target object based on the center of gravity change data of the target object; Based on the multimodal data, obstacle information in the scene where the target object is located is identified, and the posture information of the target object is identified based on the multimodal data. Using the obstacle information and the posture information, risk parameters of the target object are generated.

3. The method according to claim 1, characterized in that, The behavioral strategy also includes a posture change strategy; wherein, the position movement strategy is used to control the robot to move to the target location so that the robot can perform a supporting action for the target object at the target location; the posture change strategy is used to control the robot to switch to the target posture so that the robot can perform a supporting action for the target object in the target posture; When the behavior strategy includes a positional movement strategy, the target orientation includes the distance between the robot and the target object and the robot's orientation; generating the robot's behavior strategy based on the risk parameters includes: The distance between the robot and the target object is determined based on the risk parameter; the distance is negatively correlated with the risk parameter. The orientation between the robot and the target object is determined based on the risk parameters; the angle difference between the orientation and the direction in which the robot points towards the target object is negatively correlated with the risk parameters. The positional movement strategy is generated based on the distance between the robot and the target object and the robot's orientation.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Execute the aforementioned behavioral strategy; In response to the risk parameters meeting preset risk conditions, a support action is generated based on the current pose of the robot and the current pose of the target object; Perform the aforementioned assistance action; The assisting action includes at least one of the following: shortening the distance between the robot and the target object; changing the robot's actions to match the current posture of the target object.

5. The method according to any one of claims 1 to 3, characterized in that, The method further includes: In response to the risk parameters meeting preset risk conditions, the collected multimodal data is stored; When the multimodal data characterizes a fall event of the target object, event record data corresponding to the fall event is generated based on the stored multimodal data; If the multimodal data indicates that the target object has not experienced a fall event, the stored multimodal data is deleted.

6. The method according to claim 5, characterized in that, When the multimodal data characterizes a fall event of the target object, the method further includes: The injury level of the target object is assessed based on the event record data; Execute the corresponding level of rescue strategy based on the level of injury; The rescue strategy includes at least one of the following: generating rescue information and sending it to the rescued object at the corresponding rescue level; sending the event record data to the rescued object at the corresponding rescue level; and executing the lift-up strategy.

7. A fall prevention device, characterized in that, The device, applied to robots, includes: The acquisition module is used to acquire multimodal data of the scene in which the target object is located during the process of following the target object; The evaluation module is used to evaluate the risk parameters of the target object based on the multimodal data; A generation module is used to generate a behavior strategy for the robot based on the risk parameters; the behavior strategy is used to help the target object when it is about to fall, and the behavior strategy includes a positional movement strategy that meets the following range. The generation module is further configured to obtain the data acquisition range of each sensing device corresponding to the multimodal data; the data acquisition range is determined based on the acquisition method of the sensing device for the target object and the setting position of the sensing device in the robot; based on the data acquisition range corresponding to each sensing device and the data acquisition weight of each sensing device, a following range based on the target object is generated. Wherein, at any position within the following range, at least one sensor device of the robot has a data acquisition range that includes the target object; the detection index value of the robot at any position within the following range satisfies a preset threshold; the detection index value is determined by the following method: obtaining the detection weight corresponding to each sensor device; weighted summing of the sub-index values ​​corresponding to each sensor device based on the detection weight of each sensor device to obtain the detection index value; wherein, the detection index value is determined based on the sub-index value corresponding to each sensor device, and the sub-index value is positive when the robot is within the data acquisition range corresponding to the sensor device; the sub-index value is negative when the robot is outside the data acquisition range corresponding to the sensor device.

8. A robot, characterized in that, Includes multimodal sensors, memory, and processors, among which, The multimodal sensor is used to collect real-time multimodal data; The memory stores computer programs that can run on a processor; When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 6.

Images