Human body recognition and tracking robot based on deep learning

Abstract

The application belongs to the technical field of robots, and particularly relates to a human body recognition and tracking robot based on deep learning, wherein the appearance structure of the human body recognition and tracking robot includes a case designed by 3D printing, the case adopts a double-layer structure, a robot chassis selected is installed on the bottom layer, joint nodes are determined, different interframe target persons are recognized and target direction positions are determined through a Cascade R-CNN model, in order to improve the accuracy of robot positioning of target persons, various sensors are arranged on the robot chassis to assist the robot in completing positioning of the target persons, and meanwhile, a robot positioning method based on UWB and ODOM fusion is matched to autonomously judge the distance between the target person and the robot, reasonably plan a following path, and complete a tracking task.

Description

Technical Field

[0001] This invention belongs to the field of robotics technology, specifically a human body recognition and tracking robot based on deep learning. Background Technology

[0002] In modern society, with the development of technology, the interaction between humans and machines is playing an increasingly important role in daily life. The main purpose of human-computer interaction is to improve the way humans interact with machines, so that machines can better understand user needs and improve user experience.

[0003] Meanwhile, the application of robots is becoming increasingly widespread. Whether in factory assembly lines, catering service work, or in special dangerous applications such as bomb disposal, search and rescue, there are all kinds of robots being used. Especially in complex terrain and dangerous environments, traditional human-computer interaction products mostly need to be controlled through hardware and cannot be directly operated. At present, human-computer interaction products need to be able to automatically recognize action information and thus autonomously track target people.

[0004] Current robot algorithms mostly employ human recognition algorithms based on convolutional neural networks. However, these algorithms are easily affected by factors such as object occlusion and lighting, leading to target loss. Using 3D skeletal features in human recognition can effectively improve the accuracy of human identification, thereby enabling the tracking of target individuals. Therefore, this paper proposes a human recognition and tracking robot based on deep learning to address the above problems. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies and solve at least one of the technical problems mentioned in the background art, this invention proposes a human body recognition and tracking robot based on deep learning.

[0006] The purpose of this invention is to study a human body recognition and tracking robot based on deep learning. Specifically, it proposes a method for predicting the movement direction and speed of a target person by recognizing the person and predicting human motion based on 3D skeleton. The Cascade R-CNN algorithm is used to recognize the target person, and the 3D skeleton recognition algorithm based on the AlphaPose model is used to predict the movement direction and speed of the target person. Combined with a robot localization method based on the fusion of UWB and ODOM, the robot autonomously judges the distance between itself and the target person, rationally plans the following path, and combines PID control and visual servo strategies. At the same time, a host computer is designed to issue commands to the robot, and finally completes the human body recognition and tracking task.

[0007] The human body recognition and tracking robot's appearance structure includes a chassis designed using 3D printing. The chassis has a double-layer structure, with the selected robot chassis installed on the bottom layer and the upper layer serving as a storage platform for placing items according to individual needs. The chassis is made of galvanized steel sheet. A monocular camera, a power supply system, and various sensors are mounted on the robot chassis.

[0008] The human recognition and tracking robot described above can store and retrieve data collected by various sensors. This includes permanent storage media for data collected by algorithms, preset programs, optical image acquisition devices such as cameras, distance sensors, infrared sensors, ultrasonic sensors, and radar, as well as memory media for exchanging the collected data. Memory is used to temporarily store computational data in the CPU and data exchanged with external storage devices such as hard drives. Memory includes random access memory (RAM), read-only memory (ROM), and cache. Currently, commonly used memory standards are SDRAM, DDR2, RDRAM, DDR3, DDR4, and DDR5, with DDR5 being the latest memory standard, supporting very fast data exchange rates, large memory capacity, and parallel expansion. The permanent storage medium is the hard drive, which can store data that needs to be retained for a long time. Current hard drives include mechanical hard drives with tracks and heads for data storage, as well as solid-state drives (SSDs) made with solid-state electronic storage chip arrays. In this method, the robot should have storage chips capable of storing data, realizing the basic functions of a hard drive.

[0009] The selection of the target person is achieved through a host computer interface. There are two selection methods: one is to input three clear, high-resolution digital photos containing the target person; the other is to use real-time images acquired by the robot's optical acquisition device and select the target person from the images via touch screen, mouse click, or other methods. After the target person is selected, the robot's algorithm begins initialization, including the following calculation process:

[0010] The robot uses a pre-stored character learning algorithm in its storage medium. Through a neural network, it performs convolution, pooling, and normalization operations on the target character's morphological image to extract the target character's features. Using three images for feature learning increases robustness and enables accurate character recognition. If a real-time image selection method is used, three discontinuous images between frames are selected for feature learning. At the same time, the angle is slightly adjusted during image acquisition to obtain different images of the character from different angles, achieving accurate feature recognition.

[0011] Once the target to be tracked is determined, a rectangle will be used to select the target in the software interface that controls the robot, including the APP, computer client, web page, etc., and the distance will be marked. At that time, the motion judgment algorithm stored in the robot, including the 3D human skeleton recognition algorithm, will determine the motion state of the person to be tracked. When the algorithm recognizes that the person to be tracked has started to move, the target person tracking motion control method combining PID control and visual servo strategy will be used to drive the robot to move forward and track.

[0012] The robot's motion state recognition of the target person includes a pre-installed 3D human skeleton recognition algorithm to identify the skeleton of the target person. Based on experimental results, a displacement threshold δ and an angle change threshold θ are set. When the absolute value of the displacement change of the lower skeleton is greater than δ and the angle between the original and current lower skeleton positions is greater than θ, the target person is determined to be in motion. When δ is positive, the target person moves to the right; conversely, when δ is negative, the target person moves to the left. The robot's 3D skeleton recognition position is determined based on real-time images captured by its front-facing monocular camera. The algorithm establishes a relationship between the displacement and angle differences of the skeleton recognition position and the threshold values ​​to identify the movement state of the person being tracked. In this algorithm, the magnitude of the displacement threshold δ and the angle change threshold θ determines the quadruped robot's sensitivity to the target being tracked. If the threshold is set too low, the quadruped robot's motion module will be activated when the person moves slightly, which will increase the robot's power consumption. If the threshold is set too high, the quadruped robot's response will become sluggish, and the target will be lost when the person's movement amplitude is small. Therefore, the threshold is generally set based on empirical conclusions, typically with a linear displacement of 10cm for the tail apex of the skeleton recognition and an angle exceeding 15°.

[0013] The target person tracking motion control method combining PID control and visual servoing strategy includes heading angle adjustment based on PID control and visual servoing function. In the described visual servoing part, the robot identifies the target person on the screen based on the image captured by the front monocular camera, and uses the PID control strategy to enable the robot to track the target person in its field of vision. When the robot cannot accurately find the target person due to occlusion during the tracking process, the visual servoing system automatically adjusts the robot speed to stabilize the target person. Through the synergistic effect of these two functions, the adverse effects of the target person's position deviation on the tracking effect can be reduced, thereby significantly enhancing the stability and accuracy of the tracking function.

[0014] Once the robot begins to move, the tracking of the target person employs a combination of two algorithms. First, a convolutional neural network-based person recognition algorithm is used. For the target person, the back, especially the lower body and legs, are typically presented to the quadruped robot. Therefore, the person recognition algorithm is optimized for the recognition of the back and legs. This method uses the Cascade R-CNN algorithm and is trained on training sets of different people to obtain the algorithm, which can accurately identify the back and legs of the person. The extracted feature interpretations mainly cover the width, length, and fullness of the back, and the shape, length, and fullness of the legs. Through the recognition and learning of human features, accurate tracking of the target person can be achieved.

[0015] The host computer is developed based on PyQt5 software. Its main functions include target person recognition and recognition of commands issued by the target person. After initialization, operators can issue commands to the robot through various means such as mobile APP, web browser, and computer client. The host computer realizes remote control of human body recognition and tracking robot, improving the convenience of the robot. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0017] Figure 1 This is a schematic diagram of the human body recognition and tracking robot in this invention;

[0018] Figure 2 This is a flowchart illustrating the overall process of target person identification and tracking in this invention.

[0019] Figure 3 The flowchart of the target person recognition algorithm of this invention is shown below;

[0020] Figure 4 This is a flowchart of a human motion prediction and tracking algorithm based on 3D skeleton in this invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Specific implementation examples are given below.

[0023] Please see Figures 1-4 This invention provides a human body recognition and tracking robot based on deep learning. The robot's external structure includes a 3D-printed chassis with a double-layer structure. The bottom layer houses the selected robot chassis, with casters for support. The upper layer serves as a storage platform for placing items according to individual needs. The chassis is made of galvanized steel. A single-disc camera, a power supply system, and various sensors are mounted on the robot chassis.

[0024] The aforementioned algorithm for identifying target individuals decomposes images captured by a robot's front-facing monocular camera into multiple frames for processing. Based on a pre-trained AlphaPose model, it identifies the 3D skeletal pose of the target individual in the image, determines joint nodes, and uses a Cascade R-CNN model to identify the target individual across different frames and determine the tracking target's location. To improve the robot's accuracy in locating the target individual, this invention incorporates various sensors on the robot's chassis to assist in locating the target individual. Simultaneously, it utilizes a robot localization method based on UWB and ODOM fusion to autonomously determine the distance to the target individual, rationally plan the following path, and complete the tracking task.

[0025] The overall flowchart of the robot's target person recognition and tracking is as follows: Figure 2 As shown:

[0026] S1: Input the target information to be tracked: Extract the features of the target to be tracked through the front monocular camera. The feature extraction method is to collect a clear video of a person under appropriate brightness, including basic images of the person's front, side and lower limbs. The number of image frames collected should not be less than 180 frames, which are used for target feature acquisition and posture skeleton simulation correction of the person.

[0027] S2: Feature extraction: After selecting the target person and enabling tracking on the host computer terminal, the feature information of the target person is extracted again.

[0028] S3: Identify the target person: During the tracking process, different moving targets are identified by recognizing the moving targets ahead, and the target person among them is tracked.

[0029] S4: Predict the direction and speed of the target person's movement: During the tracking process, a 3D human posture skeleton recognition algorithm based on the AlphaPose model is used to determine the direction of the target person's movement. When the target person is affected by obstacles or other factors, the robot collects the characteristics of the target person. The direction and speed of the target person's movement can be calculated by simulating the posture of the 3D skeleton of the target person during the image retention stage.

[0030] S5: Repeat the above steps until the robot chassis receives a pause follow command.

[0031] The flowchart of the target person recognition algorithm of the present invention is as follows:

[0032] S1: Comparison and scoring: The image captured by the front monocular camera is compared with the image pre-entered by the algorithm to obtain a recognition score;

[0033] S2: Set thresholds: Set two thresholds, namely the accurate tracking threshold and the auxiliary tracking threshold. The accurate tracking threshold is generally set around 0.95, and the auxiliary tracking threshold is set around 0.45.

[0034] S3: Identify the target person: The accurate tracking threshold is used for feature recognition of the image. When the recognition score exceeds the accurate tracking threshold, the target object is set as the target person to be tracked. If the score is less than the accurate tracking threshold but greater than the auxiliary tracking threshold, the motion prediction result obtained from the 3D pose skeleton analysis of the target person will be compared. If the data obtained by the orientation and distance sensors and the orientation and distance of the target to be checked meet the error allowable conditions, then the target is identified as the target person.

[0035] The aforementioned 3D skeleton-based human motion prediction algorithm includes:

[0036] S1: Image preprocessing: The STN network is used to map and transform the target person image to achieve the position correction of the object, including translation, rotation and scaling;

[0037] S2: Eliminate redundant poses: Determine pose distance by measuring the distance between joint nodes in different poses, and use spatial distance to perceive the difference between different feature candidate boxes;

[0038] S3: Select target person: Combining pose distance and spatial distance, multiple candidate human bounding boxes generated during the 3D human skeleton recognition process are filtered out, and only one target person is retained.

[0039] The robot localization method based on UWB and ODOM fusion uses UWB for absolute robot localization, while ODOM provides motion information. An extended Kalman filter (EKF) is used to fuse the UWB and ODOM data. The EKF filter employed in this invention is an extended Kalman filter, primarily used for state estimation and filtering of nonlinear systems to improve robot accuracy and robustness. The basic principle of EKF filtering is to linearize the nonlinear system, transforming it into a linear system, and then using Kalman filtering for state estimation and filtering. Specifically, EKF filtering approximates the nonlinear system's state and observation equations into a set of linear equations through Taylor expansion, and then uses Kalman filtering for state estimation and filtering. For each sampling time, the EKF filter estimates and filters the system state through two steps: prediction and update.

[0040] The described target person tracking motion control method combines PID control and visual servoing strategy. In the visual servoing part, the robot identifies the target person on the screen based on the image captured by the front-end monocular camera, and uses the PID control strategy to enable the robot to track the target person in its field of vision. When the robot cannot accurately locate the target person due to occlusion during tracking, the visual servoing system automatically adjusts the robot speed to stabilize the target person. Through the synergistic effect of these two functions, the adverse impact of target person position deviation on the tracking effect can be reduced, thereby significantly enhancing the stability and accuracy of the tracking function.

[0041] Once the robot begins to move, the tracking of the target person employs a combination of two algorithms. First, based on a convolutional neural network-based person recognition algorithm, the target person is typically shown their back, especially the lower body and legs. Therefore, the algorithm used is optimized for recognizing the back and legs. This invention uses the Cascade R-CNN algorithm, trained on a training set of different individuals, enabling accurate recognition of the back and legs. The extracted feature interpretations primarily cover the width, length, and fullness of the back, as well as the shape, length, and fullness of the legs. Through the recognition and learning of human body features, accurate tracking of the person to be tracked can be achieved.

[0042] The host computer is developed based on PyQt5 software, and its main functions include target person recognition and recognition of commands issued by the target person. After initialization, operators can issue commands to the robot through various means such as mobile APP, web browser, and computer client. This host computer realizes remote control of the robot for human recognition and tracking, improving the robot's convenience.

[0043] The present invention may also include:

[0044] 1. Detailed information about the Cascade R-CNN model

[0045] In robotics, the Cascade R-CNN algorithm is used to identify target individuals. This type of algorithm's identification process for the target individual to be tracked includes the following characteristics:

[0046] The color image of the target person to be tracked, captured by the front monocular camera, is fed into the Cascade R-CNN model to calculate the feature map. The feature map is then fed into the Feature Selection Network (RPN) to select and filter candidate boxes.

[0047] For candidate box recognition in RPN networks, the features include:

[0048] S1: In the Feature Map, candidate bounding boxes are generated centered on each pixel. The sizes of these candidate bounding boxes are 128, 256, and 512 pixels, with aspect ratios of 1:1, 1:2, and 2:1. This will generate 9 candidate anchor boxes for each pixel.

[0049] S2: Subsequently, based on the different anchor boxes and the labeled object fact location boxes (hereinafter referred to as GT boxes, full name Ground Truth), the bounding box classifier obtains the scores of the two. Then, based on this score, the candidate boxes are labeled as foreground and background. Generally, the candidate boxes labeled as foreground have the following characteristics: (1) They have the highest intersection-union ratio (IOU) with a certain GT (IOU refers to the proportion of the overlap between the candidate box and the labeled box to the labeled box). (2) Candidate boxes with an IOU higher than 0.7 with the GT. For the selection of background candidate boxes, anchor boxes with an IOU less than 0.3 are classified as background. For the RPN network, non-foreground and non-ground candidate boxes are meaningless, so anchor candidate boxes with an IOU between 0.3 and 0.7 are generally discarded. Each anchor candidate box receives two scores after being fed into the boundary classifier: one for considering the candidate box as foreground and one for considering it as background. The offset between the anchor candidate box and the labeled box is calculated through the boundary box adjuster layer, which typically has four parameters: the coordinates of the center, as well as the width and length.

[0050] S3: Due to the large number of candidate boxes, some positive label candidate boxes, i.e., foreground and some background candidate boxes, are usually selected for model training.

[0051] S4: In the Cascade R-CNN network, the image is typically fed into the RPN network. Based on the feature extraction of the RPN network, a portion of the feature map is obtained. This feature-extracted image is then fed into a pooling layer, followed by a detection head. The first detection head has a lower Cross-Union Ratio (CUI) threshold, which is used to obtain the position and class of a candidate box. The feature map image containing this candidate box is then fed into the next detection head, which has a higher CUI threshold than the previous detection head. Regression and other operations are then performed to obtain the coordinates and classification of a smaller candidate box. The feature map containing this candidate box is then fed into the last detection head, which has the highest CUI. Through this detection head, the coordinates and classification of the candidate box are obtained. This result is considered the final result. This cascaded operation ensures the accuracy of the recognition of tracked persons.

[0052] S5: The results obtained from the model algorithm, including position coordinate information and classification information, are transmitted to the image on the operator's or observer's terminal through the robot system, while the 3D skeleton recognition algorithm is called to perform calculations and predictions simultaneously.

[0053] 2. AlphaPose model

[0054] The 3D skeleton recognition algorithm for quadruped robots uses the AlphaPose model to identify the joints of the human skeleton. This model's recognition of the human skeleton includes the following features:

[0055] The STN (Spatial Transformer Networks) is used to map and transform the object to be detected, achieving position correction of the object, including translation, rotation, and scaling. This network can transform the original image and generates six parameters representing the degree of image transformation. The following formulas represent the mathematical expressions for translation, rotation, and scaling:

[0056] The following expression represents a translation transformation of the image, the position of which is determined by six parameters.

[0057]

[0058] For image rotation transformation, the mathematical expression is characterized by setting the parameters b and c of the above formula to the scaling factor required. If the image is rotated, the parameters are greater than 1, otherwise they are between 0 and 1. At the same time, the values ​​of the two parameters a and d are set to 0.

[0059] For image rotation, the parameters are set to trigonometric functions, with parameters b and c set to sine and cosine functions respectively, to calculate the coordinate information of the rotated image.

[0060] The STN network overcomes the limitation of neural networks in lacking translation invariance. Furthermore, the STN network is independent and can be inserted into any position within a neural network to perform its function. In practical use, we found that the STN network has a strong ability to correct images after use, achieving stable positive image output. This lays a solid foundation for subsequent 3D skeletal pose recognition.

[0061] In 3D human skeleton recognition, multiple candidate bounding boxes are usually generated. To address this, redundant candidate boxes need to be removed until only one remains. For eliminating multiple poses, a distance function is used.

[0062] d(P i P j |Λ)=K Sim (P i P j |σ1)+λH sim (P i P j |σ2) Among them, Λ={σ1, σ2, λ}

[0063] In the formula, the former represents the pose distance, and the latter represents the spatial distance. The pose distance is determined by the distance between joint nodes in different poses, thus eliminating redundant poses. Each pose contains multiple joint nodes, and each node contains a score. A candidate box of 1 / 10 of the image size is selected for the bounding box of a joint node in one pose. When a joint node in another pose is within the candidate box, the pose distance between the two is calculated. The formula for calculating the pose distance is:

[0064]

[0065] Spatial distance is used to calculate the similarity between two candidate boxes. Since each candidate box is bounded by one joint node and is 1 / 10 of the image size, spatial distance is used to perceive the difference between different feature candidate boxes. The two are then aggregated, and the harmonic parameter λ is set according to different situations. The results are calculated, and those less than a threshold are considered to be the same pose feature, with the pose joint with the highest score retained. In this method, the harmonic parameter λ is set to 0.2, and the threshold is set to 0.3.

[0066] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A human body recognition and tracking robot based on deep learning, characterized by: The robot includes an external structure for object placement and human body recognition and tracking. The external structure of the object placement and human body recognition and tracking robot includes a 3D-printed chassis. The chassis has a double-layer structure. The bottom layer is where the selected robot chassis is installed, and casters are installed below the chassis for support. The upper layer serves as an object placement platform. A monocular camera and a power supply system are installed on the robot chassis. It also includes an algorithm for identifying target people. The algorithm decomposes the image captured by the robot's front monocular camera into multiple frames for processing. Based on the pre-trained AlphaPose model, it identifies the 3D skeletal pose of the target person in the image, determines the joint nodes, and uses the CascadeRCNN model to identify the target person between different frames and determine the tracking target's orientation. Simultaneously, it utilizes a robot localization method based on the fusion of UWB and ODOM to autonomously determine the distance to the target person, rationally plan the following path, and complete the tracking task; The overall process by which the robot identifies and tracks the target person is as follows: S1: Input the target information to be tracked: Extract the features of the target to be tracked through the front monocular camera. The feature extraction method is to collect a clear video of a person under appropriate brightness, with no less than 180 frames of images, for the feature acquisition of the target and the posture skeleton simulation correction of the person. S2: Feature extraction: After selecting the target person and enabling tracking on the host computer terminal, the feature information of the target person is extracted again. S3: Identify the target person: During the tracking process, different moving targets are identified by recognizing the moving targets ahead, and the target person among them is tracked. S4: Predict the target person's direction and speed of movement: This is achieved using AlphaPose-based tracking. The model uses a 3D human posture skeleton recognition algorithm to determine the direction of movement of the target person. When the target person is affected by obstacles and the robot collects the characteristics of the target person, the direction and speed of movement of the target person are calculated by simulating the posture of the 3D skeleton of the target person during the image retention stage. S5: Repeat the above steps until the robot chassis receives a pause following command; The algorithm flow for target person recognition is as follows: S1: Comparison and scoring: The image captured by the front monocular camera is compared with the image pre-entered by the algorithm to obtain a recognition score; S2: Set thresholds: Set two thresholds, namely the accurate tracking threshold and the auxiliary tracking threshold. The accurate tracking threshold is set to 0.95, and the auxiliary tracking threshold is set to 0.

45. S3: Identify the target person: The accurate tracking threshold is used for image feature recognition. When the score of the identified target object exceeds the accurate tracking threshold, the target object is set as the target person to be tracked. If the score is less than the accurate tracking threshold but greater than the auxiliary tracking threshold, the motion prediction result obtained from the 3D pose skeleton analysis of the target person will be compared. If the data obtained by the orientation and distance sensors and the orientation and distance of the target to be checked meet the error allowable conditions, then the target is identified as the target person.

2. The human body recognition and tracking robot based on deep learning according to claim 1, characterized in that: The aforementioned algorithm for identifying target persons uses a Cascade R-CNN model to identify target persons across different frames and determine the location of the target to be tracked. The Cascade R-CNN algorithm obtains feature maps of the image through filters. On these feature maps, an RPN (Random Object Detector based on a sliding window) is used to divide the feature map into several partitions. Multiple candidate boxes of different sizes are generated, using the center pixel of each partition as the center. These candidate boxes are then evaluated to determine if a target is detected. If so, the position of the selected box is adjusted based on the information in the feature map, and then the result is fed into the system. The computation is performed in a neural network. The Cascade R-CNN used in this method has a 59-layer neural network structure, including three residual units and a total of nine convolutional layers. The input image is limited to 112*112 pixels and is then fed into the convolutional layer, pooling layer, and normalization layer for computation. Finally, the computation result is fed into a classifier to determine the type of the target object and whether the target object is the target to be tracked. The classification structure adopts a score interval classification. When the calculated result is within the classification interval of a certain classifier, the target is classified into the category of that classifier. At the same time, if it is close to the median threshold of the classifier, the accuracy of identifying the target is higher.

3. The human body recognition and tracking robot based on deep learning according to claim 1, characterized in that: The AlphaPose model contains modules for recognizing and calculating the translation, rotation, and scaling of objects. These modules can perceive the translation, scaling, and rotation of images through different calculation methods. For translation, the parameters trained by this module are numerically calculated. For scaling, a diagonal matrix is ​​used, with the diagonal elements set as scaling factors. By introducing sine and cosine functions into the calculated matrix, image rotation recognition is achieved. Subsequently, the corresponding coordinate transformation can be realized through these matrices, thus providing a theoretical basis for human skeleton recognition.

4. The human body recognition and tracking robot based on deep learning according to claim 1, characterized in that: The robot localization method based on the fusion of UWB and ODOM uses UWB for absolute robot localization, ODOM to provide motion information, an extended Kalman filter to fuse UWB and ODOM data, and an EKF filter for filtering. The EKF filter is an extended Kalman filter used for state estimation and filtering of nonlinear systems to improve the robot's accuracy and robustness.

5. The human body recognition and tracking robot based on deep learning according to claim 4, characterized in that: The EKF filter transforms the nonlinear system into a linear one by linearizing it. The system is then used for state estimation and filtering using Kalman filtering. Specifically, EKF filtering approximates the nonlinear system's state equations and observation equations into a set of linear equations by Taylor expansion, and then uses Kalman filtering for state estimation and filtering. For each sampling time, the EKF filter estimates and filters the system state through two steps: prediction and update.

6. A human body recognition and tracking robot based on deep learning according to claim 1, characterized in that: It also includes a motion control method for tracking people by combining PID control with visual servoing strategy. Through the synergistic effect of these two functions, the adverse effects of target position deviation on the tracking effect are reduced.

Images