Sensor-based accompanying robot intelligent prediction method, equipment and medium

Abstract

The invention discloses a sensor-based accompanying robot intelligent prediction method and device, and a medium, and belongs to the technical field of accompanying robots. The method comprises the following steps: collecting user behavior, vision and environment data; filtering and standardizing the data based on a Kalman filtering algorithm to generate standard data; processing the standard data based on an action recognition algorithm, and determining a user behavior; constructing a digital model based on the standard environment data, processing user behaviors and the digital model by combining a preset target and a reward function, and selecting a navigation path; predicting the position and the motion direction of the user at the next moment based on historical environment data and a parallel interactive multi-model Kalman filtering algorithm; calculating a motion instruction accompanying the robot according to the navigation strategy and the user prediction information; according to the method, various sensor data can be effectively fused, the accuracy of user behavior recognition is improved, and a flexible and efficient navigation strategy is formulated.

Description

Technical Field

[0001] The present application relates to the technical field of companion robots, and in particular to a sensor-based companion robot intelligent prediction method, device and medium. Background Art

[0002] In the context of the rapid development of modern science and technology, companion robots, as an important member of the intelligent service field, are gradually integrated into people's daily lives. Such robots are designed to follow and serve specific users in various environments, such as providing navigation, assisted walking, and carrying items. However, it is not easy to achieve efficient and safe companion services. It requires robots to accurately understand the user's behavioral intentions, perceive changes in the surrounding environment in real time, and make reasonable navigation decisions based on this.

[0003] Traditional companion robot systems often rely on single sensor data to identify user behavior and perceive the environment, which leads to limitations in understanding complex scenes. For example, relying solely on visual sensors may not accurately determine the user's motion state, especially in light changes or occlusions; and using only inertial sensors may have difficulty distinguishing subtle differences in user movements. In addition, traditional navigation strategies are often based on fixed rules or simple algorithms, lacking adaptability and flexibility to dynamic changes in the environment, and are prone to collisions or getting lost in environments with dense crowds or many obstacles.

[0004] Therefore, how to effectively integrate multiple sensor data, improve the accuracy of user behavior recognition, and formulate flexible and efficient navigation strategies has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present application provide a sensor-based companion robot intelligent prediction method, device and medium to solve the following technical problems: how to effectively integrate multiple sensor data, improve the accuracy of user behavior recognition, and formulate flexible and efficient navigation strategies.

[0006] In a first aspect, an embodiment of the present application provides a sensor-based companion robot intelligent prediction method, which is applied to a companion robot and a wearable device. The companion robot is provided with a camera unit, a laser radar and a depth sensor, and the wearable device is provided with an inertial measurement unit. The method comprises: collecting the user's behavior data based on the wearable device, collecting the user's visual data by the camera unit, and determining the environmental data around the companion robot by the laser radar and the depth sensor; wherein the behavior data includes acceleration and angular velocity, and the environmental data includes at least pedestrian position, pedestrian speed, pedestrian distance and obstacles; filtering the behavior data, image data and environmental data based on a preset Kalman filtering algorithm, and performing standard processing to generate standard data; wherein the standard data includes standard behavior data, standard visual data and standard environmental data; processing the standard data based on a preset action recognition algorithm to determine the user's user behavior; wherein, The user behavior includes at least one of the following data: standing, walking, turning, and running; a digital model is constructed based on standard environmental data; the user behavior and the digital model are processed based on preset goals and reward functions to determine a navigation path; wherein the goals include safely following the user, avoiding obstacles and crowded areas, and the reward function includes high scores for successfully avoiding collisions and rewards for efficiently reaching the destination; the behavior data and the navigation path are processed based on standard environmental data and historical environmental data of the standard environmental data to determine the navigation strategy; the user's behavior data is processed based on a preset parallel interactive multi-model Kalman filter algorithm and motion pattern to predict the user's position and direction of motion at the next unit moment; wherein the unit moment is the prediction interval of the companion robot; the motion instructions of the companion robot are calculated based on the navigation strategy and the user's position and direction of motion at the next unit moment; wherein the motion instructions include: machine speed, machine acceleration, and machine steering angle.

[0007] In one implementation of the present application, behavioral data, image data, and environmental data are filtered and standardized to generate standard data, specifically including: processing behavioral data based on a preset adaptive Kalman filter algorithm to adjust the filtering parameters of the adaptive Kalman filter algorithm according to the dynamic change characteristics of the behavioral data to generate preliminary standard behavioral data; extracting feature vectors of image data based on a preset feature extraction algorithm to generate preliminary standard visual data; processing environmental data based on an adaptive Kalman filter to generate preliminary standard environmental data; normalizing preliminary standard behavioral data, preliminary standard visual data, and preliminary standard environmental data to the same data range to generate standard data.

[0008] In one implementation of the present application, standard data is processed based on a preset action recognition algorithm to determine the user's user behavior, specifically including: constructing a preliminary action recognition model; wherein the preliminary action recognition model is a neural network model obtained by training based on user behavior data; training the preliminary action recognition model based on a preset transfer learning algorithm and historical data of the behavior data to obtain an action recognition model; wherein an action recognition algorithm is provided in the action recognition model; and inputting the standard data into the action recognition model to determine the user's user behavior.

[0009] In one implementation of the present application, a digital model is constructed based on standard environmental data, specifically including: mapping the standard environmental data to a preset three-dimensional space coordinate system to construct an initial environmental model; wherein the three-dimensional space coordinate system is divided into a plurality of voxels; processing the initial environmental model based on the standard environmental data to fill the environmental attributes of the plurality of voxels to construct a digital model; wherein the environmental attributes include walkability, dense population, and obstacles.

[0010] In one implementation of the present application, user behavior and digital models are processed based on preset goals and reward functions to determine a navigation path, specifically including: taking a companion robot as a starting point, and determining a target point based on the user and a preset distance; determining a goal, and constructing a reward function based on the goal; wherein the goals include safely following the user, avoiding obstacles, and avoiding crowded areas, and the reward function is set with different weights; processing the starting point, target point, and digital model based on a preset path planning algorithm to determine multiple preliminary navigation paths; and evaluating multiple preliminary navigation paths based on the weights of the reward function to determine the navigation path.

[0011] In one implementation of the present application, behavior data and navigation paths are processed based on standard environment data and historical environment data of the standard environment data to determine a navigation strategy, specifically including: retrieving a preset historical environment database based on the standard environment data to determine whether the companion robot has historical environment data of the standard environment data; if not, adopting a navigation strategy based on the navigation path; if so, obtaining the current time of searching the companion robot, and retrieving the historical database based on the current time to determine whether there is historical environment data at the same time; if not, adopting a navigation strategy based on the navigation path; if so, evaluating the navigation path and behavior data based on the historical environment data to generate an evaluation result; wherein the evaluation result includes potential obstacles; and modifying the navigation path based on the evaluation result to determine the navigation strategy.

[0012] In one implementation of the present application, the user's behavior data is processed based on a preset parallel interactive multi-model Kalman filter algorithm and motion pattern to predict the user's position and movement direction at the next unit moment, specifically including: constructing multiple motion models; wherein each motion model corresponds to a user behavior, and the motion models include but are not limited to uniform straight-line walking, accelerated running, deceleration and stopping, and sudden turning; processing multiple motion models based on the Kalman filter algorithm, and updating the predicted states of multiple motion models in real time with the user's historical behavior data; wherein the predicted state includes position, speed and acceleration; processing the user's behavior data based on a preset motion pattern recognition algorithm to evaluate the matching degree of multiple motion models; processing the behavior data based on the motion model with the highest matching degree to determine a preliminary prediction; and calculating the user's position and movement direction at the next unit moment based on the preliminary prediction and the unit moment.

[0013] In one implementation of the present application, the motion instructions of the companion robot are calculated based on the navigation strategy and the position and movement direction of the user at the next unit time, specifically including: calculating the target position and direction that the companion robot needs to reach based on the path points determined by the navigation strategy and the predicted position of the user at the next unit time; determining the constraints of the companion robot; wherein the constraints include maximum speed, maximum acceleration, steering angle range, and motion smoothness requirements; processing the current state of the companion robot based on the target position and direction to generate preliminary motion instructions; processing the preliminary motion instructions based on the constraints to determine the motion instructions.

[0014] In the second aspect, the embodiment of the present application also provides a sensor-based companion robot intelligent prediction device, which is applied to the companion robot and the wearable device. The companion robot is provided with a camera unit, a laser radar and a depth sensor, and the wearable device is provided with an inertial measurement unit. The device includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can: collect the user's behavior data based on the wearable device, collect the user's visual data based on the camera unit, and determine the environmental data around the companion robot through the laser radar and the depth sensor; wherein the behavior data includes acceleration and angular velocity, and the environmental data includes at least pedestrian position, pedestrian speed and pedestrian distance; based on a preset Kalman filtering algorithm, the behavior data, image data and environmental data are filtered and standardized to generate standard data; wherein the standard data includes standard behavior data, standard visual data and standard environment data. data; processing standard data based on a preset action recognition algorithm to determine the user's user behavior; wherein the user behavior includes at least one of the following data: standing, walking, turning, running; building a digital model based on standard environmental data; processing user behavior and digital models based on preset goals and reward functions to determine the selection of navigation paths; wherein the goals include safely following the user, avoiding obstacles and crowded areas, and the reward function includes high scores for successfully avoiding collisions and rewards for efficiently reaching the destination; processing behavior data and navigation paths based on standard environmental data and historical environmental data of standard environmental data to determine navigation strategies; processing user behavior data based on a preset parallel interactive multi-model Kalman filtering algorithm and motion pattern to predict the user's position and direction of motion at the next unit moment; wherein the unit moment is the prediction interval of the companion robot; calculating motion instructions for the companion robot based on the navigation strategy and the user's position and direction of motion at the next unit moment; wherein the motion instructions include: machine speed, machine acceleration, and machine steering angle.

[0015] In the third aspect, the embodiment of the present application also provides a non-volatile computer storage medium for intelligent prediction of a companion robot based on sensors, which is applied to a companion robot and a wearable device. The companion robot is provided with a camera unit, a laser radar and a depth sensor, and the wearable device is provided with an inertial measurement unit, and computer executable instructions are stored. The computer executable instructions are set to: based on the wearable device, the user's behavior data is collected, the camera unit collects the user's visual data, and the laser radar and the depth sensor determine the environmental data around the companion robot; wherein the behavior data includes acceleration and angular velocity, and the environmental data includes at least the pedestrian position, pedestrian speed and pedestrian distance; based on a preset Kalman filtering algorithm, the behavior data, image data and environmental data are filtered and standardized to generate standard data; wherein the standard data includes standard behavior data, standard visual data and standard environmental data; the standard data is processed based on a preset action recognition algorithm to determine User behavior of the user; wherein the user behavior includes at least one of the following data: standing, walking, turning, and running; constructing a digital model based on standard environmental data; processing the user behavior and the digital model based on preset goals and reward functions to determine the selection of a navigation path; wherein the goals include safely following the user, avoiding obstacles and crowded areas, and the reward function includes high scores for successfully avoiding collisions and rewards for efficiently reaching the destination; processing the behavior data and the navigation path based on standard environmental data and historical environmental data of the standard environmental data to determine the navigation strategy; processing the user's behavior data based on a preset parallel interactive multi-model Kalman filter algorithm and motion pattern to predict the user's position and direction of motion at the next unit moment; wherein the unit moment is the prediction interval of the companion robot; calculating the motion instructions of the companion robot based on the navigation strategy and the user's position and direction of motion at the next unit moment; wherein the motion instructions include: machine speed, machine acceleration, and machine steering angle.

[0016] The sensor-based companion robot intelligent prediction method, device and medium provided in the embodiments of the present application have at least the following technical effects:

[0017] By integrating various sensor data, the accurate collection of user behavior and environmental data is achieved to a certain extent, thereby improving the reliability and diversity of the data. The data is filtered and standardized through the adaptive Kalman filtering algorithm and the feature extraction algorithm to generate standard data; the action recognition model is optimized through the transfer learning algorithm to accurately identify user behavior, enhancing the robot's understanding ability of user behavior. The construction of the digital model and the application of the path planning algorithm enable the robot to intelligently plan the navigation path, effectively avoiding obstacles and crowded areas to a certain extent. Through the parallel interactive multi-model Kalman filtering algorithm, the prediction of the user's position and movement direction at the next unit time is realized; through the navigation strategy and the user prediction information, the motion instructions that meet the constraint conditions are calculated, ensuring the safe and efficient following of the robot to a certain extent. In summary, this application can effectively fuse various sensor data, improve the accuracy of user behavior recognition, and formulate flexible and efficient navigation strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0019] Figure 1 It is a flowchart of an intelligent prediction method for an accompanying robot based on sensors provided by an embodiment of the present application;

[0020] Figure 2 It is a schematic internal structure diagram of an intelligent prediction device for an accompanying robot based on sensors provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0022] The embodiments of the present application provide an intelligent prediction method, device, and medium for an accompanying robot based on sensors to solve the following technical problems: how to effectively fuse various sensor data, improve the accuracy of user behavior recognition, and formulate flexible and efficient navigation strategies.

[0023] The technical solutions proposed in the embodiments of the present application will be described in detail below with reference to the drawings.

[0024] Figure 1A sensor-based companion robot intelligent prediction flow chart provided in the embodiment of the present application. Figure 1 As shown, a sensor-based companion robot intelligent prediction method provided in an embodiment of the present application specifically includes the following steps:

[0025] The embodiments of the present application are applied to a companion robot and a wearable device. The companion robot is provided with a camera unit, a laser radar and a depth sensor, and the wearable device is provided with an inertial measurement unit.

[0026] In the embodiments of the present application, the companion robot is used to follow the user to achieve the effects of accompanying the elderly, helping to place goods, etc., and the wearable device is used to collect the user's motion information, that is, to recognize the user's motion.

[0027] The companion robot is equipped with a camera unit, a laser radar and a depth sensor.

[0028] The camera unit is arranged on the head of the companion robot to capture the user's facial features, body posture and image information of the surrounding environment in real time.

[0029] The laser radar is installed on the side of the companion robot to scan the surrounding environment and generate a three-dimensional point cloud map.

[0030] Depth sensors work in conjunction with lidar to supplement lidar's lack of detection at short distances.

[0031] The inertial measurement unit is built into a wearable device (such as a smart watch, a smart bracelet, and in the embodiment of the present application, the wearable device is an anklet or a legging), and includes an acceleration sensor and a gyroscope.

[0032] The accelerometer is used to measure the user's linear acceleration in three-dimensional space, reflecting the user's walking speed and running rhythm.

[0033] The gyroscope is used to measure the rotational movement of the user's body or limbs, that is, the angular velocity, and to identify the user's turning intention and changes in body posture.

[0034] Step 1: collect the user's behavior data based on the wearable device, collect the user's visual data by the camera unit, and determine the environmental data around the robot using the lidar and depth sensor; wherein the behavior data includes acceleration and angular velocity, and the environmental data includes at least the pedestrian's position, pedestrian's speed, and pedestrian's distance.

[0035] In an embodiment of the present application related to step 1: a companion robot is used to follow an elderly person to help the elderly person load items and accompany the elderly person.

[0036] The elderly wear an anklet (wearable device), which is connected to the companion robot via Bluetooth. The wearable device collects the elderly's behavioral data through the built-in inertial measurement unit. The companion robot follows the elderly 2 meters behind and recognizes the elderly's body posture (visual data) through the camera (camera unit). The companion robot uses lidar and depth sensors to identify the position, speed, distance and obstacle information of people around the companion robot.

[0037] Step 2: Filter and standardize the behavior data, image data, and environmental data to generate standard data; wherein the standard data includes standard behavior data, standard visual data, and standard environmental data.

[0038] Step 21: Process the behavior data based on a preset adaptive Kalman filter algorithm to adjust the filter parameters of the adaptive Kalman filter algorithm according to the dynamic change characteristics of the behavior data to generate preliminary standard behavior data.

[0039] The adaptive Kalman filter algorithm is a Kalman filter algorithm that can automatically adjust the filter parameters according to the dynamic changing characteristics of the data.

[0040] First, initialize the parameters of the adaptive Kalman filter algorithm.

[0041] Furthermore, the acceleration and angular velocity output by the inertial measurement unit in the wearable device are collected in real time.

[0042] Furthermore, the collected acceleration and angular velocity are input as measurement values ​​into the adaptive Kalman filter algorithm for state estimation and error update.

[0043] Furthermore, according to the dynamic change characteristics of the behavior data, the filtering parameters of the adaptive Kalman filter algorithm are adjusted in real time.

[0044] Further, preliminary standard behavior data is output.

[0045] Step 22: extract feature vectors of the image data based on a preset feature extraction algorithm to generate preliminary standard visual data.

[0046] First, image data output by the camera unit on the companion robot is collected in real time.

[0047] Furthermore, the collected image data is preprocessed to remove noise and enhance contrast.

[0048] Furthermore, a scale-invariant feature transformation algorithm is applied to extract features from the preprocessed image data.

[0049] Furthermore, preliminary standard visual data is output.

[0050] Step 23: Process the environmental data based on the adaptive Kalman filter to generate preliminary standard environmental data.

[0051] First, the environmental data output by the laser radar and depth sensor on the companion robot is collected in real time, and further, the collected environmental data is input into the adaptive Kalman filter algorithm as a measurement value to output preliminary standard environmental data. The same as step 21, no further description is given here.

[0052] Step 24: Normalize the preliminary standard behavior data, preliminary standard visual data, and preliminary standard environment data to the same data range to generate standard data.

[0053] First, a data range analysis is performed on the preliminary standard behavioral data, preliminary standard visual data, and preliminary standard environmental data to determine the data range.

[0054] Further, a data range is selected as a target range, which can accommodate the variation range of all preliminary standard data.

[0055] Furthermore, linear normalization is applied to normalize the preliminary standard behavior data, preliminary standard visual data, and preliminary standard environment data to within the target range, and then the standard data is output.

[0056] Step 3: Process the standard data based on a preset action recognition algorithm to determine the user's user behavior; wherein the user behavior includes at least one of the following data: standing, walking, turning, and running.

[0057] Step 31: construct a preliminary action recognition model; wherein the preliminary action recognition model is a neural network model obtained by training based on user behavior data.

[0058] First, user behavior data is collected as training samples, including standing, walking, turning, and running. The training samples are used to train a preliminary action recognition model with universal applicability. For example, images of standing and data of human bodies in standing state are obtained through the Internet.

[0059] Furthermore, the collected user behavior data is preprocessed, such as denoising and normalization, to improve the accuracy and efficiency of model training.

[0060] Furthermore, the model is designed according to the characteristics of user behavior data.

[0061] Furthermore, the preprocessed user behavior data is used to train the neural network, and by adjusting the network parameters and optimizing the algorithm, the model can accurately identify different user behaviors.

[0062] Further, after the training is completed, a preliminary action recognition model is obtained, which can preliminarily recognize behaviors of the user such as standing, walking, turning, running, etc.

[0063] It can be understood that the preliminary action recognition model is a model with universality.

[0064] Step 32: Train the preliminary action recognition model based on the historical data of the preset transfer learning algorithm and behavior data to obtain an action recognition model; wherein, an action recognition algorithm is set in the action recognition model.

[0065] First, collect the historical data of the user behavior data.

[0066] Further, apply the transfer learning algorithm to transfer the knowledge in the preliminary action recognition model to the historical data, and further train the model by combining the current training data and the historical data.

[0067] Further, during the training process, adjust the parameters and optimization strategies of the transfer learning algorithm to make full use of the information in the historical data and improve the generalization ability and recognition accuracy of the model.

[0068] Further, after the training is completed, an action recognition model is obtained.

[0069] Step 33: Input the standard data into the action recognition model to determine the user behavior of the user.

[0070] First, collect the standard data of the user in real time.

[0071] Further, input the collected standard data into the action recognition model as the input data of the action recognition model.

[0072] Further, the action recognition model processes the input standard data, and determines the current behavior of the user through the forward propagation of the neural network and the calculation of the action recognition algorithm.

[0073] Further, output the user behavior of the user.

[0074] Step 4: Construct a digital model based on the standard environment data.

[0075] Step 41: Map the standard environment data to a preset three-dimensional space coordinate system to construct an initial environment model; wherein, the three-dimensional space coordinate system is divided into multiple voxels.

[0076] First, determine the standard environment data, which has been generated by Step 2.

[0077] Further, set a suitable three-dimensional space coordinate system, and determine the origin, axis directions and unit length of the coordinate system, and this process is set manually.

[0078] Furthermore, each object or area in the standard environment data is mapped to a corresponding position in the three-dimensional space coordinate system.

[0079] Furthermore, according to a preset voxel size, the three-dimensional space is divided into a plurality of voxels, and each voxel represents a small area in the space.

[0080] Furthermore, based on the mapped data and the voxel segmentation results, an initial environment model is constructed, which only contains the spatial position and basic geometric shape of the object, but has not yet been given environmental attributes.

[0081] Step 42: Process the initial environment model based on the standard environment data to fill the environment attributes for multiple voxels to construct a digital model; wherein the environment attributes include walkability, dense population, and obstacles.

[0082] First, according to actual needs, define the types of environmental attributes that need to be filled, such as walkable, crowded, and obstacles, and set the corresponding value range or standard for each attribute.

[0083] Furthermore, the standard environment data is analyzed to extract the environment attribute information related to each voxel. For example, terrain data is analyzed to determine whether a voxel is walkable; personnel flow data is used to determine the density of personnel in a voxel; and obstacle data is used to identify whether there is an obstacle in a voxel.

[0084] Furthermore, the environmental attribute information obtained through analysis is filled into the corresponding voxels in the initial environmental model, and a specific attribute label is assigned to each voxel.

[0085] Furthermore, after attribute filling, the final digital model is obtained.

[0086] Step 5: Process the user behavior and digital model based on preset goals and reward functions to determine the selected navigation path; wherein the goals include safely following the user, avoiding obstacles and crowded areas, and the reward function includes high scores for successfully avoiding collisions and rewards for efficiently reaching the destination.

[0087] Step 51: Take the companion robot as the starting point and determine the target point based on the user and the preset distance.

[0088] First, before navigation begins, the current position of the companion robot is used as the starting point of navigation.

[0089] Furthermore, the user's location information is acquired in real time through sensors (such as camera units, infrared sensors).

[0090] Furthermore, a safety distance threshold is set, and the distance between the user position and the current position of the companion robot is calculated.

[0091] If the distance exceeds the safety distance threshold, a point a certain distance (such as 1 meter or 2 meters, which can be adjusted according to actual needs) in front of the user's current position will be used as the target point; if the distance is within the safety distance range, the current following state will be maintained, without setting a specific target point or setting a fine-tuning target point to maintain an appropriate distance.

[0092] Step 52: determine the goal and construct a reward function based on the goal; wherein the goal includes safely following the user, avoiding obstacles, and avoiding crowded areas, and the reward function is set with different weights.

[0093] First, clarify the goals that need to be achieved during navigation, namely, safely following the user and avoiding obstacles and crowded areas.

[0094] Further, construct the reward function:

[0095] Set reward items related to the user's distance, such as positive rewards for maintaining an appropriate distance and negative rewards for being too close or too far; negative rewards for collision and positive rewards for successfully avoiding; negative rewards for entering crowded areas and positive rewards for avoiding them.

[0096] Furthermore, different weights can be set for each item in the reward function according to actual needs. For example, the weight of safely following the user is higher because it is a basic requirement for navigation; while the weight of avoiding crowded areas may be adjusted according to different scenarios. For example, in a shopping mall, the weight of avoiding crowded areas is lower.

[0097] Step 53: Process the starting point, the target point and the digital model based on a preset path planning algorithm to determine a plurality of preliminary navigation paths.

[0098] Select a suitable path planning algorithm according to actual needs. The path planning algorithm used in the embodiment of the present application is the A* algorithm.

[0099] First, the starting point and the target point determined in step 51 are used as inputs to the path planning algorithm.

[0100] Furthermore, the digital model constructed in step 4 is used as the environmental information input of the path planning algorithm.

[0101] Furthermore, a path planning algorithm (such as an A* algorithm) is run to calculate multiple preliminary navigation paths from the starting point to the target point.

[0102] Step 54: Evaluate a plurality of preliminary navigation paths based on the weight of the reward function to determine a navigation path.

[0103] First, for each preliminary navigation path, its reward value is calculated according to the reward function. The higher the reward value, the more the path meets the target requirements.

[0104] Furthermore, the reward values ​​of each preliminary navigation path are compared to find the path with the highest reward value.

[0105] Furthermore, the path with the highest reward value is taken as the final navigation path for the companion robot to execute.

[0106] In the embodiment of the present application, the companion robot is started at the entrance of the shopping mall, and its starting position is determined by the positioning system. The location information of the customer (user) is obtained in real time through the camera. According to the customer's position and the current position of the companion robot, the distance between the two is calculated, and a safety distance threshold of 1.5 meters is set. If the distance exceeds the threshold, the point 1.5 meters in front of the customer is used as the target point; if the distance is within the safe range, the current following state is maintained.

[0107] Determine navigation goals: follow customers safely, avoid obstacles (such as shelves, display cabinets), and avoid crowded areas (such as promotion areas and rest areas).

[0108] Constructing the reward function:

[0109] Following customers safely: +10 points / seconds for maintaining an appropriate distance, -5 points / seconds for being too close or too far.

[0110] Avoid obstacles: -20 points for a collision, +5 points for a successful avoidance.

[0111] Avoid crowded areas: -10 points / seconds for entering crowded areas, +2 points / seconds for avoiding them.

[0112] Set weights: The weight of safely following customers is 1.0, the weight of avoiding obstacles is 0.8, and the weight of avoiding crowded areas is 0.6 (considering that there may be many crowded areas in shopping malls, the weights are appropriately reduced to avoid excessive avoidance. The weights can be set according to the actual situation).

[0113] The starting point, target point and digital model (including mall layout, obstacle location and crowd density) are used as inputs to the A* algorithm.

[0114] Run the A* algorithm to calculate multiple preliminary navigation paths from the starting point to the target point.

[0115] For each preliminary navigation path, its reward value is calculated according to the reward function.

[0116] Compare the reward values ​​of each path and find the path with the highest reward value.

[0117] The path with the highest reward value is taken as the final navigation path for the companion robot to execute.

[0118] Step 6: Process the behavior data and the navigation path based on the standard environment data and the historical environment data of the standard environment data to determine the navigation strategy.

[0119] Step 61: Retrieve a preset historical environment database based on the standard environment data to determine whether the companion robot has historical environment data of the standard environment data.

[0120] First, obtain the standard environment data of the current environment according to step 2.

[0121] Furthermore, the acquired standard environment data is used as a query condition to search a preset historical environment database to find out whether there is historical environment data similar to the current environment.

[0122] Further, if similar historical environmental data is found, the process proceeds to step 63 , and if not found, the process proceeds to step 62 .

[0123] Step 62: If not, use the navigation strategy based on the navigation path.

[0124] First, in the absence of finding similar historical environment data, the companion robot will rely on the current navigation path as a navigation strategy.

[0125] Furthermore, the companion robot moves according to the navigation path while updating the environmental data in real time.

[0126] Step 63: If yes, obtain the current time of the search companion robot, and search the historical database based on the current time to determine whether there is historical environmental data at the same time.

[0127] First, the current time is obtained through the robot's internal clock or an external time synchronization system.

[0128] Furthermore, the current time is used as a query condition to further search the historical environment database to find out whether there is historical environment data that is the same as or close to the current time.

[0129] Furthermore, if the historical environmental data at the same time is found, the process proceeds to step 65 ; if not found, the process proceeds to step 64 .

[0130] Step 64: If not, use the navigation strategy based on the navigation path.

[0131] Same as step 62, which will not be described in detail here.

[0132] Step 65: If yes, evaluate the navigation path and behavior data based on the historical environment data to generate an evaluation result; wherein the evaluation result includes potential obstacles.

[0133] First, historical environment data that is the same or close to the current time is extracted from the historical environment database.

[0134] Furthermore, the historical environment data, the current navigation path and the behavior data are compared to consider the impact of possible changing factors in the historical environment on the navigation path. The specific consideration process is described in the embodiments of the present application.

[0135] Furthermore, based on the results of the comprehensive analysis, an assessment report is generated, which clearly points out potential obstacles and risk points.

[0136] Step 66: Modify the navigation path based on the evaluation result to determine the navigation strategy.

[0137] First, carefully review the assessment report to understand the specific location and impact of potential obstacles and risk points.

[0138] Furthermore, according to the evaluation results, necessary adjustments are made to the original navigation path, such as changing the moving route, adjusting the moving speed, adding obstacle avoidance actions, etc.

[0139] Furthermore, the adjusted navigation path is used as a new navigation strategy to guide the companion robot to navigate.

[0140] In the embodiments of the present application:

[0141] When the robot is started in the industrial park, it obtains standard environmental data of the current environment, including obstacle locations, road width, and crowd density.

[0142] The acquired standard environmental data is used as the query condition to search the preset historical environmental database to find out whether there is similar historical environmental data.

[0143] (If no similar historical environmental data is found)

[0144] If similar historical environment data is not found in the historical environment database, the companion robot will rely on the current navigation path for navigation.

[0145] The navigation path is calculated by the path planning algorithm (A* algorithm), which plans an optimal path from the starting point to the target point.

[0146] If similar historical environment data is found in the historical environment database, the companion robot will obtain the current time and further search the historical database based on the current time to find out whether there is historical environment data that is the same or close to the current time, so as to consider the impact of time factors on the navigation path.

[0147] (If historical environmental data at the same time is not found) If the historical environmental data at the same time is not found in the historical database, the companion robot will still rely on the current navigation path for navigation.

[0148] If the historical environmental data at the same time is found in the historical database, the companion robot extracts this data and conducts a comprehensive analysis with the current navigation path and behavior data.

[0149] Considering the possible changes in the historical environment (such as temporary construction, road closures, etc.), the navigation path is evaluated and an evaluation result is generated. The evaluation result points out potential obstacles and risk points, such as construction areas, crowded areas, etc.

[0150] The original navigation path is adjusted and optimized based on the evaluation results, such as avoiding construction areas and areas where people gather. The adjusted navigation path is used as a new navigation strategy to guide the companion robot to navigate.

[0151] Step 7: Process the user's behavior data based on the preset parallel interactive multi-model Kalman filter algorithm and motion pattern to predict the user's position and motion direction at the next unit time; where the unit time is the prediction interval of the accompanying robot.

[0152] Step 71: construct multiple motion models; wherein each motion model corresponds to a user behavior, and the motion models include but are not limited to uniform straight-line walking, accelerated running, decelerated stopping, and sudden turning.

[0153] First, by analyzing the user's historical behavior data and actual application scenarios, we determine the main types of behaviors that users may take, such as walking in a straight line at a constant speed, running at an accelerated speed, slowing down and stopping, and turning suddenly.

[0154] Furthermore, for each user behavior, a corresponding motion model is constructed. For example, the uniform straight-line walking model can describe the state of a user moving in a straight line at a constant speed; the accelerated running model can describe the process of a user gradually accelerating while running; the deceleration stop model can describe the state of a user gradually decelerating until stopping; and the sudden turn model can describe the behavior of a user suddenly changing the direction of movement.

[0155] Furthermore, parameters are set for each motion model, and the parameters include initial position, velocity, and acceleration.

[0156] Step 72: Process multiple motion models based on the Kalman filter algorithm, and update the predicted states of the multiple motion models in real time with the user's historical behavior data; wherein the predicted states include position, speed, and acceleration.

[0157] First, the initial state is set for each motion model based on the user's historical behavior data.

[0158] Furthermore, as the user's behavior continues, new observation data (such as the user's location, speed, etc.) is continuously acquired. The Kalman filter algorithm is used to combine the observation data and the dynamic equation of the motion model to update the predicted state of each motion model in real time.

[0159] Step 73: Process the user's behavior data based on a preset motion pattern recognition algorithm to evaluate the matching degree of multiple motion models.

[0160] First, extract key features from the user's behavior data, such as position change rate, speed change rate, acceleration change rate, etc.

[0161] Furthermore, the extracted features are input into a preset motion pattern recognition algorithm, and the algorithm determines which behavior the user is currently taking based on these features.

[0162] Furthermore, for each motion model, the matching degree between the motion model and the user's current behavior is calculated. The matching degree can be obtained by comparing the difference between the predicted state of the motion model and the actual behavior data of the user.

[0163] Step 74: Process the behavior data based on the motion model with the highest matching degree to determine a preliminary prediction.

[0164] First, according to the evaluation results of the matching degree, the motion model with the highest matching degree is selected as the best model.

[0165] Furthermore, the user's behavior data is processed using the best model to obtain a preliminary prediction result of the user's next unit time, including position and movement direction.

[0166] Step 75: Calculate the user's position and movement direction at the next unit time based on the preliminary prediction and the unit time.

[0167] First, determine the appropriate unit time length based on actual application requirements and the performance of the accompanying robot.

[0168] Furthermore, based on the preliminary prediction results and the unit time, the specific position and movement direction of the user at the next unit time are calculated.

[0169] Furthermore, the calculated future state is output as a prediction result for the companion robot to make subsequent decisions and actions.

[0170] Step 8: Based on the navigation strategy and the user's position and movement direction at the next unit time, calculate the movement instructions of the accompanying robot; wherein the movement instructions include: machine speed, machine acceleration and machine steering angle.

[0171] Step 81: Based on the waypoints determined by the navigation strategy and the predicted position of the user at the next unit time, calculate the target position and direction that the companion robot needs to reach.

[0172] First, a series of waypoints are extracted from the preset navigation strategy, which represent the desired motion trajectory of the robot.

[0173] Furthermore, the user behavior prediction algorithm is used to calculate the predicted position of the user at the next unit time.

[0174] Furthermore, the target position that the robot needs to reach is calculated based on the user's predicted position and navigation path points. This is usually achieved by selecting the path point closest to the user's predicted position or interpolating based on the distribution of the path points and the user's movement trend.

[0175] Furthermore, based on the target position and the current position of the robot, the target direction that the robot needs to face is calculated. This can be obtained by calculating the direction vector between two points or using a more complex path planning algorithm.

[0176] Step 82: Determine the constraints of the companion robot; wherein the constraints include maximum speed, maximum acceleration, steering angle range, and motion smoothness requirements.

[0177] First, determine the robot's maximum speed, maximum acceleration, and steering angle range based on its hardware specifications and motion capabilities.

[0178] Furthermore, in order to ensure the comfort and safety of the user, the robot's motion smoothness requirements are set, such as limiting the rate of change of acceleration and avoiding sharp turns.

[0179] Furthermore, the constraints of the robot are comprehensively determined based on the robot's performance and user experience requirements.

[0180] Step 83: Process the current state of the companion robot based on the target position and orientation to generate preliminary motion instructions.

[0181] First, the robot's current position, velocity, acceleration, and direction of movement are acquired through its sensor system.

[0182] Further, according to the target position and direction and the current state of the robot, the preliminary motion parameters such as the speed, acceleration and steering angle that the robot needs to adjust are calculated. This can be achieved through a proportional-integral-derivative (PID) control algorithm or other motion control algorithms.

[0183] Furthermore, the calculated preliminary motion parameters are converted into motion instruction formats that can be understood by the robot, such as motor control signals, steering gear rotation angles, etc.

[0184] Step 84: Process the preliminary motion instructions based on the constraint conditions to determine the motion instructions.

[0185] First, the preliminary motion command is compared with the robot's constraints to check whether it exceeds the maximum velocity, maximum acceleration, steering angle range, or violates the motion smoothness requirement.

[0186] Furthermore, if the preliminary motion command violates the constraint conditions, it is adjusted. For example, if the preliminary velocity exceeds the maximum velocity limit, it is reduced to the maximum velocity; if the preliminary acceleration changes too fast, it is smoothed.

[0187] Furthermore, the adjusted motion instructions are used as motion control instructions actually executed by the robot and are sent to the control system of the robot to execute corresponding movements.

[0188] The above is an embodiment of the method proposed in this application. Based on the same inventive concept, the embodiment of this application also provides a sensor-based companion robot intelligent prediction device, whose structure is as follows: Figure 2 shown.

[0189] Figure 2 A schematic diagram of the internal structure of a sensor-based companion robot intelligent prediction device provided in an embodiment of the present application. Figure 2 As shown, the device includes:

[0190] at least one processor 201;

[0191] and, a memory 202 communicatively connected to the at least one processor;

[0192] The memory 202 stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor 201 to enable at least one processor 201 to:

[0193] Based on the wearable device, the user's behavior data is collected, the camera unit collects the user's visual data, and the laser radar and depth sensor determine the environmental data around the robot; wherein the behavior data includes acceleration and angular velocity, and the environmental data at least includes the pedestrian's position, pedestrian speed and pedestrian distance; based on the preset Kalman filter algorithm, the behavior data, image data and environmental data are filtered and standardized to generate standard data; wherein the standard data includes standard behavior data, standard visual data and standard environmental data; based on the preset action recognition algorithm, the standard data is processed to determine the user's user behavior; wherein the user behavior includes at least one of the following data: standing, walking, turning, running; constructing a digital model based on the standard environmental data; based on the preset target and The reward function processes user behavior and digital models to determine the selection of a navigation path; wherein the goals include safely following the user, avoiding obstacles and crowded areas, and the reward function includes high scores for successfully avoiding collisions and rewards for efficiently reaching the destination; the behavior data and navigation path are processed based on standard environmental data and historical environmental data of the standard environmental data to determine the navigation strategy; the user's behavior data is processed based on a preset parallel interactive multi-model Kalman filter algorithm and motion pattern to predict the user's position and direction of motion at the next unit moment; wherein the unit moment is the prediction interval of the companion robot; the motion instructions of the companion robot are calculated based on the navigation strategy and the user's position and direction of motion at the next unit moment; wherein the motion instructions include: machine speed, machine acceleration and machine steering angle.

[0194] Some embodiments of the present application provide corresponding Figure 1 A non-volatile computer storage medium for intelligent prediction of a companion robot based on a sensor, storing computer executable instructions, wherein the computer executable instructions are set as follows:

[0195] Based on the wearable device, the user's behavior data is collected, the camera unit collects the user's visual data, and the laser radar and depth sensor determine the environmental data around the robot; wherein the behavior data includes acceleration and angular velocity, and the environmental data at least includes the pedestrian's position, pedestrian speed and pedestrian distance; based on the preset Kalman filter algorithm, the behavior data, image data and environmental data are filtered and standardized to generate standard data; wherein the standard data includes standard behavior data, standard visual data and standard environmental data; based on the preset action recognition algorithm, the standard data is processed to determine the user's user behavior; wherein the user behavior includes at least one of the following data: standing, walking, turning, running; constructing a digital model based on the standard environmental data; based on the preset target and The reward function processes user behavior and digital models to determine the selection of a navigation path; wherein the goals include safely following the user, avoiding obstacles and crowded areas, and the reward function includes high scores for successfully avoiding collisions and rewards for efficiently reaching the destination; the behavior data and navigation path are processed based on standard environmental data and historical environmental data of the standard environmental data to determine the navigation strategy; the user's behavior data is processed based on a preset parallel interactive multi-model Kalman filter algorithm and motion pattern to predict the user's position and direction of motion at the next unit moment; wherein the unit moment is the prediction interval of the companion robot; the motion instructions of the companion robot are calculated based on the navigation strategy and the user's position and direction of motion at the next unit moment; wherein the motion instructions include: machine speed, machine acceleration and machine steering angle.

[0196] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the IoT device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0197] The system and medium provided in the embodiments of the present application correspond one-to-one to the method. Therefore, the system and medium also have similar beneficial technical effects to the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the system and medium will not be repeated here.

[0198] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0199] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0200] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0201] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0202] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0203] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0204] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0205] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0206] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A sensor-based companion robot intelligent prediction method, applied to a companion robot and a wearable device, wherein the companion robot is provided with a camera unit, a laser radar and a depth sensor, and the wearable device is provided with an inertial measurement unit, characterized in that: The method comprises: Based on the wearable device collecting the user's behavior data, the camera unit collecting the user's visual data, the laser radar and the depth sensor determining the environmental data around the companion robot; wherein the behavior data includes acceleration and angular velocity, and the environmental data includes at least the pedestrian position, pedestrian speed, pedestrian distance and obstacles; Filtering and standardizing the behavior data, the image data, and the environment data to generate standard data; wherein the standard data includes standard behavior data, standard visual data, and standard environment data; Processing the standard data based on a preset action recognition algorithm to determine the user behavior of the user; wherein the user behavior includes at least one of the following data: standing, walking, turning, and running; Building a digital model based on the standard environmental data; Processing the user behavior and the digital model based on preset goals and reward functions to determine a navigation path; wherein the goals include safely following the user, avoiding obstacles and crowded areas, and the reward function includes obtaining high scores for successfully avoiding collisions and obtaining rewards for efficiently reaching the destination; Processing the behavior data and the navigation path based on the standard environment data and the historical environment data of the standard environment data to determine a navigation strategy; Processing the user's behavior data based on a preset parallel interactive multi-model Kalman filter algorithm and motion pattern to predict the user's position and motion direction at the next unit time; wherein the unit time is the prediction interval of the companion robot; Based on the navigation strategy and the position and movement direction of the user at the next unit time, the movement instructions of the companion robot are calculated; wherein the movement instructions include: machine speed, machine acceleration and machine steering angle.

2. A sensor-based companion robot intelligent prediction method according to claim 1, characterized in that: The behavior data, the image data, and the environment data are filtered and standardized to generate standard data, specifically including: Processing the behavior data based on a preset adaptive Kalman filter algorithm to adjust the filter parameters of the adaptive Kalman filter algorithm according to the dynamic change characteristics of the behavior data to generate preliminary standard behavior data; Extracting feature vectors of the image data based on a preset feature extraction algorithm to generate preliminary standard visual data; Processing the environmental data based on the adaptive Kalman filter to generate preliminary standard environmental data; The preliminary standard behavior data, preliminary standard visual data, and preliminary standard environment data are normalized to the same data range to generate the standard data.

3. The sensor-based companion robot intelligent prediction method according to claim 1, characterized in that: Processing the standard data based on a preset action recognition algorithm to determine the user behavior of the user specifically includes: Constructing a preliminary action recognition model; wherein the preliminary action recognition model is a neural network model obtained by training based on user behavior data; Training the preliminary action recognition model based on a preset transfer learning algorithm and the historical data of the behavior data to obtain the action recognition model; wherein the action recognition algorithm is provided in the action recognition model; The standard data is input into the action recognition model to determine the user behavior of the user.

4. The sensor-based companion robot intelligent prediction method according to claim 1, characterized in that: Building a digital model based on the standard environmental data specifically includes: Mapping the standard environment data to a preset three-dimensional space coordinate system to construct an initial environment model; wherein the three-dimensional space coordinate system is divided into a plurality of voxels; The initial environment model is processed based on the standard environment data to fill the environment attributes for the plurality of voxels to construct a digital model; wherein the environment attributes include walkability, dense population, and obstacles.

5. The sensor-based companion robot intelligent prediction method according to claim 1, characterized in that: Processing the user behavior and the digital model based on a preset goal and a reward function to determine a navigation path specifically includes: Taking the companion robot as a starting point, and determining a target point based on the user and a preset distance; Determine a goal and construct a reward function based on the goal; wherein the goal includes safely following the user, avoiding obstacles, and avoiding crowded areas, and the reward function is set with different weights; Processing the starting point, the target point and the digital model based on a preset path planning algorithm to determine a plurality of preliminary navigation paths; A plurality of the preliminary navigation paths are evaluated based on a weight of the reward function to determine a navigation path.

6. The sensor-based companion robot intelligent prediction method according to claim 1, characterized in that: The method further comprises: processing the behavior data and the navigation path based on the standard environment data and the historical environment data of the standard environment data to determine a navigation strategy, specifically comprising: Retrieving a preset historical environment database based on the standard environment data to determine whether the companion robot has historical environment data of the standard environment data; If not, adopt the navigation strategy based on the navigation path; If so, obtaining the current time of the search companion robot, and searching the historical database based on the current time to determine whether there is historical environmental data at the same time; If not, adopt the navigation strategy based on the navigation path; If so, evaluating the navigation path and behavior data based on the historical environment data to generate an evaluation result; wherein the evaluation result includes potential obstacles; The navigation path is modified based on the evaluation result to determine the navigation strategy.

7. The sensor-based companion robot intelligent prediction method according to claim 1, characterized in that: Processing the user's behavior data based on a preset parallel interactive multi-model Kalman filter algorithm and motion pattern to predict the user's position and motion direction at the next unit time, specifically including: Construct multiple motion models; each motion model corresponds to a user behavior, and the motion models include but are not limited to uniform straight-line walking, accelerated running, decelerated stopping, and sudden turning; Processing the plurality of motion models based on a Kalman filter algorithm, and updating the predicted states of the plurality of motion models in real time with the historical behavior data of the user; wherein the predicted states include position, speed and acceleration; Processing the user's behavior data based on a preset motion pattern recognition algorithm to evaluate the matching degree of the plurality of motion models; processing the behavioral data based on the motion model with the highest matching degree to determine a preliminary prediction; The position and movement direction of the user at the next unit time are calculated based on the preliminary prediction and the unit time.

8. The sensor-based companion robot intelligent prediction method according to claim 1, characterized in that: Based on the navigation strategy and the position and movement direction of the user at the next unit time, the movement instruction of the companion robot is calculated, specifically including: Based on the waypoints determined by the navigation strategy and the predicted position of the user at the next unit time, the target position and direction that the companion robot needs to reach are calculated; Determining the constraint conditions of the companion robot; wherein the constraint conditions include maximum speed, maximum acceleration, steering angle range, and motion smoothness requirements; processing the current state of the companion robot based on the target position and orientation to generate preliminary motion instructions; The preliminary motion command is processed based on the constraints to determine the motion command.

9. A sensor-based companion robot intelligent prediction device, applied to a companion robot and a wearable device, wherein the companion robot is provided with a camera unit, a laser radar and a depth sensor, and the wearable device is provided with an inertial measurement unit, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: Based on the wearable device collecting the user's behavior data, the camera unit collecting the user's visual data, the laser radar and the depth sensor determining the environmental data around the companion robot; wherein the behavior data includes acceleration and angular velocity, and the environmental data includes at least the pedestrian position, pedestrian speed, pedestrian distance and obstacles; The behavior data, the image data and the environment data are filtered based on a preset Kalman filter algorithm, and are standardized to generate standard data; wherein the standard data includes standard behavior data, standard visual data and standard environment data; Processing the standard data based on a preset action recognition algorithm to determine the user behavior of the user; wherein the user behavior includes at least one of the following data: standing, walking, turning, and running; Building a digital model based on the standard environmental data; Processing the user behavior and the digital model based on preset goals and reward functions to determine a navigation path; wherein the goals include safely following the user, avoiding obstacles and crowded areas, and the reward function includes obtaining high scores for successfully avoiding collisions and obtaining rewards for efficiently reaching the destination; Processing the behavior data and the navigation path based on the standard environment data and the historical environment data of the standard environment data to determine a navigation strategy; Processing the user's behavior data based on a preset parallel interactive multi-model Kalman filter algorithm and motion pattern to predict the user's position and motion direction at the next unit time; wherein the unit time is the prediction interval of the companion robot; Based on the navigation strategy and the position and movement direction of the user at the next unit time, the movement instructions of the companion robot are calculated; wherein the movement instructions include: machine speed, machine acceleration and machine steering angle.

10. A non-volatile computer storage medium for intelligent prediction of a companion robot based on a sensor, storing computer executable instructions, applied to a companion robot and a wearable device, wherein the companion robot is provided with a camera unit, a laser radar and a depth sensor, and the wearable device is provided with an inertial measurement unit, characterized in that: The computer executable instructions are configured to: Based on the wearable device collecting the user's behavior data, the camera unit collecting the user's visual data, the laser radar and the depth sensor determining the environmental data around the companion robot; wherein the behavior data includes acceleration and angular velocity, and the environmental data includes at least the pedestrian position, pedestrian speed, pedestrian distance and obstacles; The behavior data, the image data and the environment data are filtered based on a preset Kalman filter algorithm, and are standardized to generate standard data; wherein the standard data includes standard behavior data, standard visual data and standard environment data; Processing the standard data based on a preset action recognition algorithm to determine the user behavior of the user; wherein the user behavior includes at least one of the following data: standing, walking, turning, and running; Building a digital model based on the standard environmental data; Processing the user behavior and the digital model based on preset goals and reward functions to determine a navigation path; wherein the goals include safely following the user, avoiding obstacles and crowded areas, and the reward function includes obtaining high scores for successfully avoiding collisions and obtaining rewards for efficiently reaching the destination; Processing the behavior data and the navigation path based on the standard environment data and the historical environment data of the standard environment data to determine a navigation strategy; Processing the user's behavior data based on a preset parallel interactive multi-model Kalman filter algorithm and motion pattern to predict the user's position and motion direction at the next unit time; wherein the unit time is the prediction interval of the companion robot; Based on the navigation strategy and the position and movement direction of the user at the next unit time, the movement instructions of the companion robot are calculated; wherein the movement instructions include: machine speed, machine acceleration and machine steering angle.

Images