A human target tracking method, device and robot

Abstract

The present invention provides a method, device, and robot for tracking a human target. The method comprises: obtaining a depth image and a normal image of the human target at the current time; processing the depth image to obtain corresponding depth image tracking information; processing the normal image to obtain corresponding normal image tracking information; comprehensively determining current tracking information based on the depth image tracking information and the normal image tracking information, and updating the human target tracking information based on the valid current tracking information; tracking the human target based on the human target tracking information; wherein, if the depth image tracking information detects a human body, but the normal image tracking information does not detect a human body, the human body detected by the depth camera is deemed to be false, and the current tracking information is invalid. The present invention improves the accuracy of human recognition, reduces the occurrence of tracking errors during the tracking process, optimizes motion processing, and avoids lag or deadlock during motion.

Description

Technical Field

[0001] The present invention relates to the field of target tracking, and in particular to a method and device for tracking a human target, and a robot. Background Art

[0002] Currently, robots tracking human targets often rely on depth cameras on the robot to detect the human skeleton. After identifying the human target, the depth camera can provide point cloud data (i.e., three-dimensional position information) of each important joint node of the human skeleton. Based on the three-dimensional position information of a certain joint, the distance and deflection angle between the joint and the depth camera can be calculated. The robot then sends instructions to the motion control module to perform movement and / or rotation movements, thereby controlling the robot's walking. However, when facing objects with smooth surfaces, such as mirrored doors and smooth aluminum alloy door frames, the depth camera often misidentifies bones due to the reflection of light, resulting in tracking anomalies such as the robot colliding with the door frame. Summary of the Invention

[0003] The purpose of the present invention is to provide a method and device for tracking human targets, and a robot, which are used to solve the problem that depth cameras are prone to recognition errors when facing objects with smooth surfaces, thereby causing abnormal robot tracking.

[0004] The technical solutions provided by the present invention are as follows:

[0005] A method for tracking a human target, comprising: obtaining a depth image and a normal image of the human target at a current time; processing the depth image to obtain corresponding depth image tracking information;

[0006] Processing the ordinary image to obtain corresponding ordinary image tracking information;

[0007] Comprehensively determine current tracking information based on the depth image tracking information and the common image tracking information, and update the human target tracking information based on the valid current tracking information;

[0008] Tracking the human target according to the human target tracking information;

[0009] The step of comprehensively determining the current tracking information based on the depth image tracking information and the ordinary image tracking information includes:

[0010] If the depth image tracking information detects a human body, but the ordinary image tracking information does not detect a human body, it is considered that the human body detected by the depth camera is false, the current tracking does not detect a human body, and the current tracking information is invalid.

[0011] In some embodiments, the depth image tracking information includes first distance and first angle information of the human target relative to the robot; the general image tracking information includes second angle information of all detected human bodies relative to the robot;

[0012] The step of comprehensively determining the current tracking information based on the depth image tracking information and the ordinary image tracking information further includes:

[0013] If a human body is detected in both the depth image tracking information and the ordinary image tracking information, comparing the first angle information with all the second angle information;

[0014] If there is a deviation between the second angle information and the first angle information within the preset deviation range, it is considered that the human body detected by the depth camera in this tracking is present, the current tracking information is valid, and the current tracking information is obtained according to the depth image tracking information.

[0015] In some embodiments, the depth image is processed to obtain corresponding depth image tracking information, including: if there are multiple human bodies in the depth image and the human target to be tracked has been determined, then the latest position information of the human target is obtained from the stored human target tracking information, and a human body within a preset position range from the latest position information of the human target is selected from the depth image as an analysis object, and corresponding depth image tracking information is obtained based on the analysis result of the analysis object.

[0016] In some embodiments, selecting a human body whose latest position information from the human target is within a preset position range from the depth image as an analysis object includes:

[0017] Acquire the three-dimensional position information of the tracking node of the human target from the human target tracking information, and use it as reference position information;

[0018] Obtaining three-dimensional position information of a tracking node of each human body in the depth image;

[0019] The three-dimensional position information of the tracking node of each human body in the depth image is traversed, and compared with the reference position information according to a preset rule, and a human body within a preset position range from the reference position information is selected as an analysis object.

[0020] In some embodiments, the preset rule is:

[0021] If the vertical change of the tracking node of the human body relative to the reference position information is within a preset position range, and the front-to-back change of the tracking node of the human body relative to the reference position information is within a preset position range, the human body is taken as the analysis object.

[0022] In some embodiments, processing the depth image to obtain corresponding depth image tracking information further includes:

[0023] If there are multiple human bodies in the depth image, but the human target to be tracked has not been determined, the human body performing a predetermined gesture or the human body closest to the robot is selected from the depth image as the human target, and the corresponding depth image tracking information is obtained based on the analysis results of the human target.

[0024] In some embodiments, the ordinary image is processed to obtain corresponding ordinary image tracking information, including: if there are multiple human bodies in the ordinary image, obtaining the second angle information of each human body relative to the robot, and obtaining corresponding ordinary image tracking information based on all the second angle information.

[0025] In some embodiments, the human target tracking information includes a first distance and a first angle information of the human target relative to the robot;

[0026] Tracking the human target according to the human target tracking information includes:

[0027] Comparing the first angle information of the current human target tracking information with the first angle information of the previous human target tracking information;

[0028] If the difference between the two first angle information exceeds the deviation threshold, the robot performs a rotation operation;

[0029] Comparing the first distance of the current human target tracking information with the first distance of the previous human target tracking information;

[0030] If the difference between the two first distances does not exceed the distance threshold, no new moving process is performed and the original moving process is maintained.

[0031] The present invention also provides a tracking device for a human target, comprising:

[0032] An image acquisition module is used to acquire a depth image and a normal image of a human target at the current time;

[0033] A depth image processing module, configured to process the depth image to obtain corresponding depth image tracking information;

[0034] A common image processing module, used for processing the common image to obtain corresponding common image tracking information;

[0035] a comprehensive determination module, configured to comprehensively determine current tracking information based on the depth image tracking information and the ordinary image tracking information, and update the human target tracking information based on the valid current tracking information;

[0036] A motion processing module, configured to track the human target according to the human target tracking information;

[0037] The comprehensive determination module is further configured to: if the depth image tracking information detects a human body, but the ordinary image tracking information does not detect a human body, deem the human body detected by the depth camera to be false, and the current tracking information is invalid because no human body is detected.

[0038] The present invention also provides a robot comprising a depth camera for acquiring depth images, a common camera for acquiring common images, and a tracking device; the tracking device is used to execute the aforementioned method for tracking a human target.

[0039] Compared with the prior art, the human target tracking method, device, and robot provided by the present invention can at least bring the following beneficial effects:

[0040] 1. The present invention adds an ordinary camera to the robot, uses the ordinary camera to identify the human body and estimate the deviation angle between the human body and the robot, and assists the tracking results of the depth camera, thereby improving the accuracy of human body recognition and effectively improving the human body recognition errors caused by infrared reflection of the depth camera, which leads to strange phenomena such as "banging into the door".

[0041] 2. The present invention reduces the occurrence of wrong tracking of a specific human body by judging the continuity of data based on the limited position change of the human body target in a short period of time.

[0042] 3. The present invention optimizes motion processing and compares the human target tracking information twice. If the position change of the human target is small, no new rotation or movement operation is performed, thus avoiding jamming or deadlock during motion. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The following will explain the preferred implementation scheme in a clear and easy-to-understand manner with reference to the accompanying drawings, and further illustrate the above-mentioned characteristics, technical features, advantages and implementation methods of a human target tracking method and device, and a robot.

[0044] Figure 1 It is a flow chart of an embodiment of a method for tracking a human target of the present invention;

[0045] Figure 2 1 is a schematic structural diagram of an embodiment of a human target tracking device of the present invention;

[0046] Figure 3 is a schematic structural diagram of an embodiment of a robot of the present invention;

[0047] Figure 4 This is a schematic diagram of the joints of the human skeleton. DETAILED DESCRIPTION

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific embodiments of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings and other embodiments can be obtained based on these drawings without inventive work.

[0049] To simplify the drawings, only the parts relevant to the present invention are schematically shown in each figure. They do not represent the actual structure of the product. Furthermore, to simplify the drawings and facilitate understanding, in some figures, only one of the components with the same structure or function is schematically depicted or labeled. As used herein, "one" not only means "only one" but also "more than one."

[0050] In one embodiment of the present invention, Figure 1 As shown, a method for tracking a human target includes:

[0051] Step S100 obtains a depth image and a normal image of a human target at a current time;

[0052] Step S200 processes the depth image to obtain corresponding depth image tracking information;

[0053] Step S300 processes the normal image to obtain corresponding normal image tracking information;

[0054] Step S400 comprehensively determines current tracking information based on the depth image tracking information and the common image tracking information, and updates the human target tracking information based on the valid current tracking information;

[0055] Wherein, step S400 includes:

[0056] In step S410, if a human body is detected according to the depth image tracking information, but not detected according to the ordinary image tracking information, it is considered that the human body detected by the depth camera is false, no human body is detected in this tracking, and the tracking information is invalid;

[0057] Step S500 tracks the human target according to the human target tracking information.

[0058] The tracking method provided in this embodiment is applied to a robot, which is provided with a depth camera and a normal camera.

[0059] As mentioned in the background, depth cameras and ordinary cameras each have their own advantages and disadvantages. Depth cameras can obtain 3D information of objects and detect human skeletons, but they may mistakenly identify skeletons for objects with smooth surfaces. Ordinary cameras, such as RGB cameras, can only obtain 2D planar information of objects, but they will not mistakenly identify skeletons for objects with smooth surfaces, such as a smooth mirrored door.

[0060] This embodiment leverages the above features by installing at least two cameras on the robot: a depth camera and a standard camera. Both cameras simultaneously capture the same scene containing a human target. The resulting depth image and standard image are analyzed separately, and then a comprehensive assessment is made of the depth image tracking information and the standard image tracking information. This effectively identifies instances where the depth camera mistakenly detects a human. If the depth camera makes a false detection during tracking, but the standard camera doesn't detect a human, the tracking information is invalidated, eliminating the need to update the human target tracking information based on the invalid information. This prevents tracking anomalies caused by erroneous data.

[0061] It is preferred that the depth camera and the ordinary camera be installed together, so that the fields of view of the two cameras are basically the same.

[0062] In one embodiment, step S200 includes:

[0063] In step S210, if there are multiple human bodies in the depth image but the human target to be tracked has not yet been determined, the human body performing the predetermined gesture or the human body closest to the robot is selected from the depth image as the human target, and the corresponding depth image tracking information is obtained based on the analysis results of the human target.

[0064] Specifically, if this is the first tracking, the tracking target has not yet been determined. If there are multiple human bodies in the depth image, the human body closest to the robot can be selected as the tracking target. Specifically,

[0065] like Figure 4 As shown, there are multiple skeletal joint nodes in the human body. One of the nodes is selected as the tracking node for all human bodies. For example, the center node of the human body (SPINE_MID, the node between the two chests) is selected. Tracking this node is equivalent to tracking the human body.

[0066] Using the SDK (Software Development Kit) provided by the depth camera manufacturer, we can obtain a list of the bones present in each frame, as well as the position data of the important joints (i.e., nodes) corresponding to each bone. This includes the horizontal x-coordinate value, vertical y-coordinate value, and forward z-coordinate value of the node in the robot's body coordinate system. The robot's body coordinate system is a rectangular coordinate system with its origin at the robot itself. For example, the origin is the robot's center node. The x-coordinate value reflects whether the node is to the left or right of the robot, and the z-coordinate value reflects the distance between the node and the robot.

[0067] The three-dimensional position information (i.e. (x, y, z) coordinate values) of the tracking node of each human body in the depth image is obtained, and the distance between the corresponding human body and the robot is obtained according to the z coordinate value of the three-dimensional position information. The human body corresponding to the smallest distance value is selected as the tracking target.

[0068] Alternatively, existing technologies may be used to detect a human body performing a predetermined gesture from a depth image, and the human body performing the predetermined gesture may be used as a tracking target.

[0069] In one embodiment, step S200 includes: step S220: if there are multiple human bodies in the depth image and the human target to be tracked has been determined, the latest position information of the human target is obtained from the human target tracking information, and a human body within a preset position range from the latest position information of the human target is selected from the depth image as an analysis object, and corresponding depth image tracking information is obtained according to the analysis result of the analysis object.

[0070] If the tracking target has been determined, but there are multiple human bodies in the depth image, it is necessary to select the tracking target from the multiple human bodies, otherwise the wrong person will be tracked.

[0071] To this end, the latest position information of the tracked human target is obtained from the human target tracking information, and the human body within a preset position range from the latest position information of the human target is selected from the depth image as the analysis object. Specifically, it may include:

[0072] Acquire the three-dimensional position information of the tracking node of the human target from the human target tracking information and use it as reference position information;

[0073] Obtain the three-dimensional position information of each tracking node of the human body in the depth image;

[0074] The three-dimensional position information of each tracking node of the human body in the depth image is traversed, and it is compared with the reference position information according to preset rules, and the human body within the preset position range from the reference position information is selected as the analysis object.

[0075] Based on the data of two adjacent frames, the vertical change of the tracking target and the change of the tracking target in the front-back direction relative to the robot will not fluctuate too much. That is, the y and z coordinate values ​​of the tracking node of the same human body will not fluctuate too much. Therefore, the preset rules can be set as:

[0076] If the vertical change (i.e., the change in the y-coordinate) of the tracking node relative to the reference position information is within the preset position range, and the front-to-back change (i.e., the change in the z-coordinate) of the tracking node relative to the reference position information is also within the preset position range, then the person is considered an analysis target. If multiple people meet these conditions, the person closest to the reference position information is selected as the analysis target.

[0077] The distance between each human body and the previous human target can also be calculated based on the three-dimensional position information of the tracking node of each human body in the depth image and the reference position information. If the distance is within the preset position range, the corresponding human body is used as the analysis object, that is, the human body is the same person as the previous human target.

[0078] Therefore, the depth image tracking information only records the analysis results of the human target (ie, the tracking object).

[0079] When tracking a specific human body, this embodiment reduces the possibility of following the wrong target or losing the target during the tracking process by judging the continuity of data.

[0080] In one embodiment, the depth image tracking information includes a first distance and a first angle information of the human target relative to the robot. The first distance and the first angle information are obtained by:

[0081] Obtaining the three-dimensional position information of the tracking node of the human target in the depth image;

[0082] According to the x-coordinate value and the z-coordinate value of the tracking node, the first angle information of the human target relative to the robot is obtained; according to the z-coordinate value of the tracking node, the first distance of the human target relative to the robot is obtained.

[0083] Specifically, assume that the deflection angle of the human target relative to the robot is defined as: in the robot body coordinate system, a perpendicular line is drawn through the tracking node G to the xoz plane (the plane formed by the x-axis and the z-axis), obtaining the foot of the perpendicular g, and the angle between the straight line from the origin o to the foot of the perpendicular g and the z-axis. The x-coordinate value of the tracking node is divided by the z-coordinate value to obtain the tangent value of the deflection angle; the deflection angle is obtained based on the tangent value, and the deflection angle is recorded as the first angle information. The deflection angle can also be defined in other ways, such as the angle between the straight line from the origin to the foot of the perpendicular g and the x-axis, and the calculation method can be changed accordingly.

[0084] Angle information includes direction and angle value. For example, 6 means the position on the left side of the robot with an angle of 6 degrees; -6 means the position on the right side of the robot with an angle of 6 degrees.

[0085] Sometimes the human skeleton detected by the depth camera may not be accurate. First, you need to determine whether the preset joints exist. If so, the human body is detected and the 3D position information of the tracking node of the human target is obtained. Preset joints include but are not limited to the NECK joint, SPINE_MID joint, KNEE_LEFT joint, and KNEE_RIGHT joint.

[0086] In one embodiment, step S300 includes:

[0087] In step S310 , the ordinary image is input into the human body detection model for processing, the second angle information of each human body relative to the robot in the ordinary image is obtained, and corresponding ordinary image tracking information is obtained according to all the second angle information.

[0088] The human body detection model is trained using deep learning algorithms, such as YOLOV3 target detection, which can detect the human body quickly and frame the position of the human body in each frame.

[0089] For each human body, the coordinate position of the human body's center in the image (i.e., the horizontal x-coordinate value and the vertical y-coordinate value) is determined. Based on the horizontal coordinates of this position, a second angle information of the human body relative to the robot is estimated. This second angle information includes both orientation and angle value. To facilitate comparison of the second angle information with the first angle information, the definition of the deflection angle corresponding to the second angle information should be the same as that of the first angle information.

[0090] For example, the deviation angle of the human body relative to the robot is defined as the angle between the straight line from the origin to the tracking node and the z-axis. Let WIDTH be the horizontal resolution of the screen, HEIGHT be the vertical resolution of the screen; angle is the empirical angle value, and X is the horizontal coordinate of the center point of the human body.

[0091] Calculate the camera's focal length: FocusDistance = (WIDTH / 2.0) / tan(toRadians(angle)).

[0092] Calculate the rotation parameter RotateParam = FocusDistance / WIDTH;

[0093] Calculate distance = X / WIDTH-0.5;

[0094] Determine the orientation of the human body relative to the robot. Orientation = distance < 0? -1:1; -1: indicates that the human body is on the left side of the robot; 1: indicates that the human body is on the right side of the robot.

[0095] Calculate the radian value of the deflection angle of the human body relative to the robot: grad = atan(abs(distance / RotateParam)); calculate the corresponding angle value: degree = toDegrees(grad);

[0096] Among them, degree is the angle value of the second angle information, orientation is the direction of the second angle information; toRadians() is a function that converts angles to radians, toDegrees() is a function that converts radians to angles, abs() is a function for finding absolute values, atan() is an inverse tangent function, and tan() is a tangent function.

[0097] When there are multiple human bodies in the common image, the common image tracking information includes second angle information of all detected human bodies relative to the robot.

[0098] The human body detection model is sometimes time-consuming to process. Therefore, for a normal image and a depth image captured at the same time, the output of the normal image processing results may be much slower than that of the depth image. Therefore, a timestamp can be added to the normal image tracking information to indicate the time of the normal image analysis result. This way, when making a comprehensive judgment based on the depth image tracking information and the normal image tracking information, the timestamp of the obtained normal image tracking information is first consulted. Only normal image tracking information with a timestamp within the allowed range is of reference value; otherwise, it is discarded.

[0099] In one embodiment, step S400 further includes:

[0100] The depth image tracking information includes the first angle information of the human target relative to the robot; the general image tracking information includes the second angle information of all detected human bodies relative to the robot;

[0101] Step S420: If a human body is detected according to both the depth image tracking information and the ordinary image tracking information, the first angle information is compared with all the second angle information;

[0102] In step S430, if there is a deviation between the second angle information and the first angle information within the preset deviation range, it is considered that the human body detected by the current tracking depth camera exists, the current tracking information is valid, and the depth image tracking information is used as the current tracking information; otherwise, the current tracking information is considered invalid.

[0103] Specifically, depth image tracking information only contains the first-angle information of a single human body, namely the human target, while normal image tracking information contains the second-angle information of all human bodies appearing in the normal image. If the deviation between the first-angle information and one of the second-angle information is within a preset error range, the depth camera detects the presence of a human body and the tracking is valid. If the current tracking is valid, the current depth image tracking information is used as the current tracking information, and the human target tracking information is updated based on this current tracking information.

[0104] By integrating depth image tracking information and ordinary image tracking information, the accuracy of tracking information can be improved.

[0105] In one embodiment, step S500 includes:

[0106] The human target tracking information includes a first distance and a first angle information of the human target relative to the robot;

[0107] Step S510 compares the first angle information of the current human target tracking information with the first angle information of the previous human target tracking information;

[0108] In step S520, if the difference between the two first angle information exceeds the deviation threshold, the robot performs a rotation operation;

[0109] Step S530 compares the first distance of the current human target tracking information with the first distance of the previous human target tracking information;

[0110] In step S540, if the difference between the two first distances does not exceed the distance threshold, no new moving process is performed and the original moving process is maintained; otherwise, a new moving process is performed.

[0111] Specifically, when the robot is moving according to human target tracking information, it needs to stop the last movement and then perform a new movement. If the interval between the two movements is too close, the robot's chassis will be frequently operated and movement will be stuck; if the rotation is performed for a too small deflection angle, it will cause the robot to enter a deadlock state where it is always rotating.

[0112] The first distance of the human target tracking information reflects the distance of the human target relative to the robot, and the first angle information reflects the deviation angle of the human target relative to the robot.

[0113] In view of the above problems, if the distance change between the current human target tracking information and the previous data is less than the distance threshold, no new movement processing will be performed to avoid repeated operation of the chassis, which may cause the robot to experience movement lag.

[0114] For angle data that does not exceed the deflection threshold, the rotation operation is not performed to avoid the deadlock state of the robot rotating all the time. The rotation operation is only performed for angle data that exceeds the deflection threshold.

[0115] In this embodiment, an ordinary camera is added to the robot, and the ordinary camera is used to identify the human body and estimate the deviation angle between the human body and the robot, thereby assisting the tracking result of the depth camera, thereby improving the accuracy of human body recognition and effectively improving the human body recognition error caused by infrared reflection of the depth camera; a set of judgment methods for identifying specific human bodies is designed, and according to the limited position change of the human target in a short time, the continuity of the judgment data is used to reduce the situation of following the wrong person in the process of tracking the specific human body; the motion processing is optimized, and the human target tracking information before and after is compared. If the position change of the human target is small, no new rotation or movement operation is performed, thereby avoiding the occurrence of jamming or deadlock in the robot movement.

[0116] Based on the same inventive concept, an embodiment of the present invention further provides a tracking device, which can be located on a robot and used to execute the tracking method for a human target of any of the aforementioned embodiments.

[0117] In one embodiment of the present invention, Figure 2 As shown, a tracking device for a human target includes:

[0118] The image acquisition module 100 is used to acquire a depth image and a normal image of a human target at a current time;

[0119] The depth image processing module 200 is used to process the depth image to obtain corresponding depth image tracking information;

[0120] The ordinary image processing module 300 is used to process ordinary images to obtain corresponding ordinary image tracking information;

[0121] A comprehensive determination module 400 is configured to comprehensively determine current tracking information based on the depth image tracking information and the common image tracking information, and update the human target tracking information based on the valid current tracking information;

[0122] A motion processing module 500 is used to track a human target according to the human target tracking information;

[0123] The comprehensive determination module 400 is also used to determine that if a human body is detected based on the depth image tracking information, but not detected based on the ordinary image tracking information, the human body detected by the depth camera is considered to be false, and no human body is detected in this tracking, and the tracking information is invalid.

[0124] In one embodiment, the depth image processing module 200 is also used to select a human body performing a predetermined gesture or a human body closest to the robot as a human target from the depth image if there are multiple human bodies in the depth image but the human target to be tracked has not yet been determined, and obtain corresponding depth image tracking information based on the analysis results of the human target.

[0125] In one embodiment, the depth image processing module 200 is also used to obtain the latest position information of the human target from the human target tracking information if there are multiple human bodies in the depth image and the human target to be tracked has been determined, and select a human body within a preset position range from the depth image to the latest position information of the human target as an analysis object, and obtain corresponding depth image tracking information based on the analysis result of the analysis object.

[0126] In one embodiment, the depth image processing module 200 is also used to obtain the three-dimensional position information of the tracking node of the human target from the human target tracking information, and use it as reference position information; obtain the three-dimensional position information of the tracking node of each human body in the depth image; traverse the three-dimensional position information of the tracking node of each human body in the depth image, compare it with the reference position information according to preset rules, and select the human body within the preset position range from the reference position information as the analysis object.

[0127] In one embodiment, the ordinary image processing module is also used to input the ordinary image into the human body detection model for processing. The human body detection model is obtained through training using a deep learning algorithm. If there are multiple human bodies in the ordinary image, the second angle information of each human body relative to the robot is obtained, and the corresponding ordinary image tracking information is obtained based on all the second angle information.

[0128] In one embodiment, the depth image tracking information includes the first distance and first angle information of the human target relative to the robot; the ordinary image tracking information includes the second angle information of all detected human bodies relative to the robot; the comprehensive determination module is also used to compare the first angle information with all the second angle information if it is determined that a human body is detected based on both the depth image tracking information and the ordinary image tracking information; if there is a second angle information that deviates from the first angle information within a preset deviation range, it is considered that the human body detected by the depth camera of this tracking exists, the current tracking information is valid, and the depth image tracking information is used as the current tracking information; otherwise, the current tracking information is considered invalid.

[0129] In one embodiment, the human target tracking information includes a first distance and a first angle information of the human target relative to the robot;

[0130] The motion processing module is also used to compare the first angle information of the current human target tracking information with the first angle information of the previous human target tracking information; if the difference between the two first angle information exceeds the deviation threshold, the robot performs a rotation operation; compare the first distance of the current human target tracking information with the first distance of the previous human target tracking information; if the difference between the two first distances does not exceed the distance threshold, no new movement processing is performed and the original movement processing is maintained; otherwise, a new movement processing is performed.

[0131] Since the embodiment of the human target tracking device and the embodiment of the human target tracking method provided above are based on the same inventive concept and can achieve the same technical effect, the other specific contents of the embodiment of the human target tracking device can refer to the description of the contents of the embodiment of the human target tracking method provided above.

[0132] In one embodiment of the present invention, Figure 3 As shown, a robot comprises:

[0133] Depth camera 10, a normal camera 20 and tracking device 30;

[0134] The tracking device 30 is used to execute the human target tracking method described in any of the aforementioned embodiments.

[0135] It should be noted that the above embodiments can be freely combined as needed. The above description is only a preferred embodiment of the present invention. It should be pointed out that those skilled in the art can make several improvements and modifications without departing from the principles of the present invention, and such improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for tracking a human target, characterized in that: For a robot equipped with a depth camera capable of acquiring depth images and a normal camera capable of acquiring normal images, the tracking method includes: Obtain the depth image and normal image of the human target at the current time; Processing the depth image to obtain corresponding depth image tracking information; the depth image tracking information includes a deflection angle of the human target relative to the robot, recorded as first angle information; Processing the ordinary image to obtain corresponding ordinary image tracking information; the ordinary image tracking information includes all detected deflection angles of the human body relative to the robot, recorded as second angle information; Comprehensively determine current tracking information based on the depth image tracking information and the ordinary image tracking information, and update human target tracking information based on the valid current tracking information; the human target tracking information includes first angle information of the human target relative to the robot; Tracking the human target according to the human target tracking information; The step of comprehensively determining the current tracking information based on the depth image tracking information and the ordinary image tracking information includes: If the depth image tracking information detects a human body, but the ordinary image tracking information does not detect a human body, it is considered that the human body detected by the depth camera is false, the current tracking does not detect a human body, and the current tracking information is invalid; If a human body is detected in both the depth image tracking information and the ordinary image tracking information, comparing the first angle information with all the second angle information; If there is a deviation between the second angle information and the first angle information within the preset deviation range, it is considered that the human body detected by the depth camera in this tracking is present, the current tracking information is valid, and the current tracking information is obtained according to the depth image tracking information.

2. The method for tracking a human target according to claim 1, wherein: Processing the depth image to obtain corresponding depth image tracking information includes: If there are multiple human bodies in the depth image and the human target to be tracked has been determined, the latest position information of the human target is obtained from the stored human target tracking information, and a human body within a preset position range from the latest position information of the human target is selected from the depth image as an analysis object, and the corresponding depth image tracking information is obtained according to the analysis result of the analysis object.

3. The method for tracking a human target according to claim 2, wherein: The step of selecting a human body whose latest position information from the human target is within a preset position range from the depth image as an analysis object includes: Acquire the three-dimensional position information of the tracking node of the human target from the human target tracking information, and use it as reference position information; Obtaining three-dimensional position information of a tracking node of each human body in the depth image; The three-dimensional position information of the tracking node of each human body in the depth image is traversed, and compared with the reference position information according to a preset rule, and a human body within a preset position range from the reference position information is selected as an analysis object.

4. The method for tracking a human target according to claim 3, wherein: The preset rules are: If the vertical change of the tracking node of the human body relative to the reference position information is within a preset position range, and the front-to-back change of the tracking node of the human body relative to the reference position information is within a preset position range, the human body is taken as the analysis object.

5. The method for tracking a human target according to claim 1, wherein: Processing the depth image to obtain corresponding depth image tracking information also includes: If there are multiple human bodies in the depth image, but the human target to be tracked has not been determined, the human body performing a predetermined gesture or the human body closest to the robot is selected from the depth image as the human target, and the corresponding depth image tracking information is obtained based on the analysis results of the human target.

6. The method for tracking a human target according to claim 1, wherein: Processing the ordinary image to obtain corresponding ordinary image tracking information includes: If there are multiple human bodies in the ordinary image, the second angle information of each human body relative to the robot is obtained, and the corresponding ordinary image tracking information is obtained according to all the second angle information.

7. The method for tracking a human target according to claim 1, wherein: The human target tracking information further includes a first distance between the human target and the robot; Tracking the human target according to the human target tracking information includes: Comparing the first angle information of the current human target tracking information with the first angle information of the previous human target tracking information; If the difference between the two first angle information exceeds the deviation threshold, the robot performs a rotation operation; Comparing the first distance of the current human target tracking information with the first distance of the previous human target tracking information; If the difference between the two first distances does not exceed the distance threshold, no new moving process is performed and the original moving process is maintained.

8. A tracking device for a human target, characterized in that: For robots, including: An image acquisition module is used to acquire a depth image and a normal image of a human target at the current time; a depth image processing module, configured to process the depth image to obtain corresponding depth image tracking information; the depth image tracking information includes a deflection angle of the human target relative to the robot, recorded as first angle information; a common image processing module, configured to process the common image to obtain corresponding common image tracking information; the common image tracking information includes the deflection angles of all detected human bodies relative to the robot, recorded as second angle information; a comprehensive determination module, configured to comprehensively determine current tracking information based on the depth image tracking information and the ordinary image tracking information, and update human target tracking information based on the valid current tracking information; the human target tracking information includes first angle information of the human target relative to the robot; A motion processing module, configured to track the human target according to the human target tracking information; The comprehensive determination module is also used to, if the depth image tracking information detects a human body but the ordinary image tracking information does not detect a human body, then it is considered that the human body detected by the depth camera is false, no human body is detected in this tracking, and the current tracking information is invalid; if both the depth image tracking information and the ordinary image tracking information detect a human body, then compare the first angle information with all the second angle information; if there is a second angle information whose deviation from the first angle information is within a preset deviation range, then it is considered that the human body detected by the depth camera in this tracking exists, the current tracking information is valid, and the current tracking information is obtained based on the depth image tracking information.

9. A robot, characterized in that: include: A depth camera for acquiring depth images, a normal camera for acquiring normal images, and a tracking device; The tracking device is used to execute the human target tracking method described in any one of claims 1-7.

Images