Method and device for cooperatively tracking target object

Abstract

The invention discloses a method and device for cooperatively tracking a target object, and the method comprises the steps: obtaining a target image which is shot by an aerial robot and comprises the target object, and obtaining the corresponding pixel coordinates of the target object in the target image; world coordinates of the target object are obtained based on the pixel coordinates, and motion state information of the target object is obtained based on the change condition of the world coordinates, so that the aerial robot tracks the target object in real time based on the world coordinates and the motion state information; and in response to the situation that the current environment does not meet the tracking condition of the aerial robot, controlling the ground robot to track the target object. According to the method, the target object can be accurately and efficiently tracked in a long-time and long-distance tracking scene with complex terrain and severe environment through the aerial robot and the ground robot.

Description

Technical Field

[0001] The present application relates to the field of target tracking technology, and in particular to a method for collaboratively tracking a target object, an apparatus for collaboratively tracking a target object, a collaborative control device, and a computer-readable storage medium. Background Art

[0002] Aerial robots (e.g., drones) and ground robots, as autonomous mobile carriers, are widely used in vision-based target detection scenarios. For example, aerial robots and ground robots can be used in target detection, target tracking, target monitoring and other detection scenarios in the fields of rescue, military operations, traffic management and environmental monitoring.

[0003] Aerial robots and ground robots each have their own advantages in target tracking scenarios. For example, aerial robots have fast maneuverability and a wide field of view, can locate and detect targets in a large area, and can avoid the limitations of ground obstacles, allowing them to operate in complex terrain and environments with dense obstacles; while ground robots have strong endurance and carrying capacity, and as ground operation equipment, they can provide more detailed and stable tracking, searching and execution capabilities on the ground. However, due to their weak endurance, aerial robots are greatly affected by weather and environment, and their performance in long-term, long-distance detection tasks and harsh environments may be affected; ground robots are easily restricted by terrain and move slowly, and the sensors mounted on ground robots are limited in height, which makes the field of view and monitoring range of ground robots relatively limited.

[0004] Therefore, how to accurately and efficiently track the target object in complex terrain, harsh environment, long time and long distance tracking scenarios is a problem that needs to be solved. Summary of the invention

[0005] The present invention provides a method, an apparatus, a collaborative control device and a computer-readable storage medium for collaborative tracking of a target object, so as to solve the problem that the existing aerial robots or ground robots alone cannot accurately and efficiently track the target object in tracking scenarios with complex terrain, harsh environment, long time and long distance.

[0006] In order to solve or partially solve the above technical problems, according to one aspect of the present invention, a method for collaborative tracking of a target object is provided, the method comprising:

[0007] Acquire a target image containing a target object captured by the aerial robot, and obtain pixel coordinates corresponding to the target object in the target image;

[0008] Obtaining the world coordinates of the target object based on the pixel coordinates, and obtaining the motion state information of the target object based on the change of the world coordinates, so that the aerial robot can track the target object in real time based on the world coordinates and the motion state information;

[0009] In response to the current environment not satisfying the tracking condition of the aerial robot, the ground robot is controlled to track the target object.

[0010] In one embodiment, controlling the ground robot to track the target object includes:

[0011] Based on the world coordinates and motion state information corresponding to the target object at the current time point, estimate the target world coordinates corresponding to the target object within a predetermined time period, or obtain the target world coordinates of the target object from the aerial robot in real time;

[0012] Controlling multiple ground robots to perform encirclement detection on the target world coordinates to track the target object.

[0013] In one embodiment, controlling a plurality of ground robots to perform encirclement detection on the target world coordinates includes:

[0014] Based on the orientation and distance of each ground robot relative to the target world coordinates, the expected encirclement speed of each ground robot with respect to the target world coordinates, and a preset encirclement movement strategy, each ground robot is controlled to perform encirclement detection on the target world coordinates.

[0015] In one embodiment, controlling the ground robot to track the target object includes:

[0016] Based on the world coordinates and the motion state information corresponding to the target object at the current time point, estimating a target area corresponding to the target object within a predetermined time period;

[0017] Controlling multiple ground robots to perform encirclement detection on the target area to track the target object.

[0018] In one embodiment, controlling a plurality of ground robots to perform encirclement detection on the target area includes:

[0019] Based on the orientation and distance of each ground robot relative to the center point of the target area, the expected encirclement speed of each ground robot with respect to the center point of the target area, and the preset encirclement movement strategy, control each ground robot to perform encirclement detection on the center point of the target area;

[0020] During the process of each of the ground robots performing encirclement detection on the center point of the target area, in response to any of the ground robots detecting the target object, the target position of the target object is obtained, and each of the ground robots is controlled to adjust the target of the encirclement detection to the target object.

[0021] In one embodiment, the motion state information includes at least one of the following:

[0022] The direction of movement of the target object;

[0023] The speed of the target object;

[0024] The acceleration of the target object.

[0025] In one implementation, obtaining the world coordinates of the target object based on the pixel coordinates includes: converting the pixel coordinates into camera coordinates based on affine transformation and perspective projection principles; and converting the camera system coordinates into world coordinates based on rigid body transformation and similarity transformation principles.

[0026] According to another aspect of the present invention, there is provided an apparatus for collaboratively tracking a target object, the apparatus comprising:

[0027] A pixel coordinate obtaining unit, used to obtain a target image captured by the aerial robot and containing a target object, and obtain pixel coordinates corresponding to the target object in the target image;

[0028] A coordinate and motion state determination unit, used to obtain the world coordinates of the target object based on the pixel coordinates, and obtain the motion state information of the target object based on the change of the world coordinates, so that the aerial robot can track the target object in real time based on the world coordinates and the motion state information;

[0029] The control tracking unit is used to control the ground robot to track the target object when the current environment does not meet the tracking conditions of the aerial robot.

[0030] According to another aspect of the present invention, a collaborative control device is provided, which includes a processor and a memory; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method as described above.

[0031] According to another aspect of the present invention, a computer-readable storage medium is provided, on which one or more computer instructions are stored, and the instructions are executed by a processor to implement the above-mentioned method for collaborative tracking of a target object.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] The method for collaborative tracking of a target object provided by the present invention includes: obtaining a target image containing a target object captured by an aerial robot, and obtaining the pixel coordinates corresponding to the target object in the target image; obtaining the world coordinates of the target object based on the pixel coordinates, and obtaining the motion state information of the target object based on the change of the world coordinates, so that the aerial robot can track the target object in real time based on the world coordinates and the motion state information; in response to the current environment not meeting the tracking conditions of the aerial robot, controlling the ground robot to track the target object. The method can track the target object in real time based on the advantages of the aerial robot's rapid maneuverability and wide field of view when the current environment meets the tracking conditions of the aerial robot, and can track the target object based on the endurance and more detailed and stable tracking, searching and execution capabilities of the ground robot when the current environment does not meet the tracking conditions of the aerial robot, thereby achieving accurate and efficient tracking of the target object in a tracking scene with complex terrain, harsh environment, long time and long distance through the aerial robot and the ground robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a flow chart of a method for collaborative tracking of a target object provided by an embodiment of the present application;

[0035] Figure 2 A unit block diagram of an apparatus for collaborative tracking of a target object provided in an embodiment of the present application;

[0036] Figure 3 It is a schematic diagram of the logical structure of the collaborative control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0038] For target tracking scenarios in complex environments, in order to achieve accurate and efficient tracking of target objects in tracking scenarios with complex terrain, harsh environments, long periods of time, and long distances, the present application provides a method for collaborative tracking of target objects, and a device, collaborative control device, and computer-readable storage medium corresponding to the method for collaborative tracking of target objects. The following provides embodiments to describe the above method, device, collaborative control device, and computer-readable storage medium in detail.

[0039] An embodiment of the present application provides a method for collaboratively tracking a target object, and the application subject of the method may be a computing device application for performing target tracking. Figure 1 A flowchart of a method for collaboratively tracking a target object provided in an embodiment of the present application is provided below in conjunction with Figure 1 The method provided in this embodiment is described in detail. The embodiments described below are used to explain the principle of the method and are not intended to be limiting for actual use.

[0040] like Figure 1 As shown, the method for collaborative tracking of a target object provided in this embodiment includes the following steps:

[0041] S101, obtaining a target image including a target object captured by the aerial robot, and obtaining pixel coordinates corresponding to the target object in the target image.

[0042] This step is used to obtain a target image containing a target object captured by the aerial robot, and obtain the pixel coordinates corresponding to the target object in the target image. Specifically, when tracking a target through the aerial robot, multiple frames of continuous overhead images captured by the aerial robot's overhead camera for the area where the target object is located are obtained at preset time intervals, and target detection is performed on the multiple frames of continuous overhead images to determine whether the overhead image contains the object to be tracked. If the overhead image contains the object to be tracked, the overhead image is determined as the target image, the above-mentioned object to be tracked is determined as the target object, and the pixel coordinates corresponding to the target object in the target image are simultaneously obtained.

[0043] It should be noted that the above-mentioned acquisition of the target image containing the target object captured by the aerial robot and the acquisition of the pixel coordinates corresponding to the target object in the target image can be performed by the aerial robot (for example, after the central control unit of the aerial robot receives the above-mentioned overhead image from the overhead camera, it performs target detection and tracking through the target detection network and target tracking network deployed on itself), or it can be a control terminal (for example, a ground control station). For example, after the overhead camera of the aerial robot captures the overhead image, it sends the captured overhead image to the central control unit of the ground control station through the communication module of the aerial robot, and the central control unit performs target detection and tracking on the overhead image.

[0044] S102, obtaining the world coordinates of the target object based on the above pixel coordinates, and obtaining the motion state information of the target object based on the change of the world coordinates, so that the aerial robot can track the target object in real time based on the world coordinates and the motion state information.

[0045] After obtaining the pixel coordinates corresponding to the target object in the target image in the above step, this step is used to obtain the world coordinates of the target object based on the above pixel coordinates, and obtain the motion state information of the target object based on the change of the world coordinates. The aerial robot can track the target object in real time based on the world coordinates and motion state information.

[0046] The above-mentioned obtaining the world coordinates of the target object based on the pixel coordinates may specifically refer to: converting the pixel coordinates to the camera coordinates based on the principles of affine transformation and perspective projection; converting the camera coordinates to the world coordinates based on the principles of rigid body transformation and similarity transformation. For example, the pixel coordinates corresponding to the target object in the target image are (u, v), and the camera intrinsic parameter matrix obtained by the camera calibration method is M1. Based on the pinhole imaging principle of the camera, the pixel coordinates can be transformed to the normalized camera coordinates (X c ,Y c ,1):

[0047] (X c ,Y c ,1) T =M1 -1 (u,v,1) T

[0048] Then, based on the principle of rigid body transformation of the coordinate system, the posture information (pitch, yaw, roll) of the aerial robot is used to obtain the camera's external parameter matrix M2 through the following formula:

[0049] M2=R z (roll)R y (yaw)R x (pitch)

[0050]

[0051]

[0052] Then, based on the normalized camera coordinates and the external parameter matrix M2, the camera coordinates (X c ,Y c ,1)Transform to world coordinates (X w ,Y w ,Z w ):

[0053]

[0054] Among them, h is the height of the aerial robot from the ground, X uav With Y uav is the world coordinate system position coordinate of the aerial robot. The above parameters can be obtained through the sensors of the aerial robot.

[0055] After obtaining the world coordinates of the target object, the motion state information of the target object can be obtained based on the changes in the world coordinates counted within a predetermined time period. In this embodiment, the motion state information of the target object can be one or more of the following information: the moving direction of the target object, the moving speed of the target object, and the motion acceleration of the target object.

[0056] It should be noted that after obtaining the world coordinates of the target object as described above, the world coordinates of the target object obtained as described above may also be filtered to filter out significant mispositioning caused by recognition errors and other factors, thereby obtaining accurate world coordinates of the target object.

[0057] S103, in response to the current environment not satisfying the tracking condition of the aerial robot, controlling the ground robot to track the target object.

[0058] In the process of the above-mentioned aerial robot tracking the target object in real time based on the world coordinates and motion state information, a series of situations that do not meet the tracking conditions of the aerial robot may occur. For example, the tracking time is too long, resulting in insufficient endurance of the aerial robot, the target object enters an unobservable area, etc., resulting in the aerial robot being unable to track. Therefore, this step is used to control the ground robot to track the target object when it is detected that the current environment does not meet the tracking conditions of the aerial robot.

[0059] In this embodiment, the ground robot can be controlled to track the target object in the following manner:

[0060] First, based on the world coordinates and motion state information corresponding to the target object at the current time point, the target world coordinates corresponding to the target object within a predetermined time period are estimated (for example, the motion state of the target object is to travel at a constant speed along a known path or in a preset direction, therefore, the moving distance and direction of the target object can be calculated within the predetermined time period, thereby combining the world coordinates of the target object at the current time point to estimate the target world coordinates of the target object), or, the target world coordinates of the target object from the aerial robot are acquired in real time.

[0061] Secondly, multiple ground robots are controlled to perform encirclement detection on the target world coordinates to track the target object.

[0062] In this embodiment, the above-mentioned control of multiple ground robots to perform encirclement detection on the target world coordinates may specifically refer to: based on the orientation and distance of each ground robot relative to the target world coordinates, the expected encirclement speed of each ground robot for the target world coordinates, and the preset encirclement movement strategy, controlling each ground robot to perform encirclement detection on the target world coordinates. The specific process is as follows:

[0063] First, the coordinates of the i-th ground robot and the target object being tracked and the distance between them are established in the coordinate system, as shown in the following formula (1):

[0064] z i (t) = p i (t)-p e (t) (1)

[0065] In the formula, p i (t) is the position coordinate vector (x i ,y i ) T ,p e (t) is the position coordinate vector of the target object being tracked (x e ,y e ) T , z i (t) represents the position distance vector (zx) between the i-th ground robot and the tracked target object i ,zy i ) T Therefore, the distance relationship between the i-th ground robot and the tracked target object can be expressed as

[0066] After the above distance relationship between the i-th ground robot and the target object being tracked is established, the angle relationship between them can be established in the following manner, as shown in the following equations (2) and (3):

[0067]

[0068] In formula (2), β i (t) represents the angle between the line connecting the i-th ground robot and the target object and the polar coordinate axis in the world coordinate system (also the vector z i (t) is the angle in the polar coordinate system), that is, β i (t) = arctan2(zy i ,zx i ), represents the difference in polar coordinates between the vectors of the target object being tracked and the i+1th ground robot and the ith ground robot, i.e., vector z i+1 (t) and vector z i (t) is the angle difference in the polar coordinate system. In formula (3), the term ∈ makes The term remains in the interval [0,2π].

[0069] After determining the position, distance, and angle relationship between the ground robot and the target object being tracked, the overall control law can be formulated as follows, that is, by using the velocity component vi1 (t) Control distance, velocity component v i2 (t) Control angle, as shown in formula (4):

[0070] v i (t) = v i1 (t)+v i2 (t) (4)

[0071] Formula (4) represents the overall control rate of multiple ground robots, where v i (t) represents the expected speed of the i-th ground robot, v i1 (t) and v i2 (t) are used as two velocity components to control the distance and angle between the ground robot and the target object being tracked. i1 (t) and v i2 (t) are uncorrelated, this embodiment introduces the following orthogonal pair b i (t) = [cosβ i (t), sinβ i (t)] T and c i (t) = [-sinβ i (t),cosβ i (t)] T The elements in this orthogonal pair are mainly composed of β i Composition, due to β i is changed in real time, so that vector b i The direction of (t) can always be pointed by the tracked target object to the i-th ground robot, c i (t) and b i (t) is vertical, therefore, a circle is constructed with the target object being tracked as the center and the i-th ground robot as a point on the circle, and a tangent line on the circle is constructed at the point where the i-th ground robot is located.

[0072] Since the vector b i The direction of (t) is always from the tracking target to the i-th ground robot. The following formula (5) is used to calculate the current distance D between the i-th ground robot and the tracking target. i (t) and expected distance Calculate the velocity component v i1 (t), which can adjust the distance relationship between the i-th ground robot and the tracked target object:

[0073]

[0074] In formula (5), v i1(t) Maintain the distance between the ground robot and the target object, κ1 is a preset adjustable constant, D i (t) represents the Euclidean distance between the i-th ground robot and the target object, that is, Represents the desired distance between the ground robot and the target object.

[0075] Because the vector c i The direction of (t) is always the tangent direction of the line between the target object being tracked and the i-th ground robot. Therefore, according to the current distance D between the i-th ground robot and the tracking target, the following formula (6) is used: i (t) and the temporary constructed reference variable z i (t), in order to adjust the angle relationship between the i-th ground robot and the tracked target object:

[0076] v i2 (t) = κ2D i (t)z i (t)c i (t) (6)

[0077] In formula (6), v i2 (t) Maintain the angle between the ground robot and the target object being tracked, κ2 is a preset adjustable constant, z i (t) is used to control the angle between different ground robots and the tracking target. When different ground robots surround the target object, z i (t) is as shown in the following formula (7):

[0078]

[0079] z i (t) Variables mainly use and To obtain, when the ground robot group has not completed the equal angle interval encirclement, and The two values ​​are not the same, z i (t) Control the ground robots so that the ground robot group gradually tends to surround the target over time. When the ground robot group has surrounded the target at an equal distance, the above two values ​​are the same, so at this time

[0080] In summary, the desired speed v is achieved. i (t) medium v i1 (t) and v i2 (t) The calculation of the two velocity components completes the acquisition of the desired velocity. After the ground robot group has completed the encirclement of the target object, the angle difference is calculated using the above formula (7) Accumulation pass Determine the current angle encirclement situation. If the situation is good, reduce the value in formula (5). Thereby further shortening the encirclement distance for encirclement.

[0081] In formula (7), represents the angular velocity of the ground robot group when it surrounds the target object. When , multiple ground robots only surround the target object and follow its movement. When the ground robot group surrounds the target object, it rotates and follows its movement.

[0082]

[0083] In formula (8), represents the sum of the angle differences between each ground robot and the previous ground robot. When , the angle intervals of the ground robot group surrounding the target object are equal.

[0084] when When it is less than a certain value, it means that multiple ground robots have a good angle condition to encircle the target object, and then reduce the angle in formula (5). That is, the expected distance between the ground robot and the target object to be tracked is reduced, the distance to encircle the target object is gradually shortened, the target object is controlled within a certain range, and the tracking and encirclement of the target object by multiple ground robots is completed.

[0085] It should be noted that the ground robot can also be controlled to track the target object in the following ways:

[0086] Based on the world coordinates and motion state information of the target object at the current time point, estimate the target area corresponding to the target object within a predetermined time period. The target area is the area that the aerial robot cannot detect at close range due to endurance problems or other problems.

[0087] Control multiple ground robots to encircle and detect the target area to track the target object.

[0088] The above-mentioned control of multiple ground robots to perform encirclement detection on the target area may specifically refer to: based on the orientation and distance of each ground robot relative to the center point of the target area, the expected encirclement speed of each ground robot with respect to the center point of the target area, and the preset encirclement movement strategy, control each ground robot to perform encirclement detection on the center point of the target area; in the process of each ground robot performing encirclement detection on the center point of the target area, in response to any ground robot detecting the target object, obtain the target position of the target object, and control each ground robot to adjust the target of the encirclement detection to the target object. For the encirclement detection process after each ground robot performs encirclement detection on the center point of the target area and subsequently adjusts the target of the encirclement detection to the target object, please refer to the process of the above-mentioned multiple ground robots performing encirclement detection on the target world coordinates, which will not be repeated here.

[0089] The method provided in this embodiment first obtains a target image containing a target object captured by an aerial robot, and obtains the pixel coordinates corresponding to the target object in the target image, then obtains the world coordinates of the target object based on the pixel coordinates, and obtains the motion state information of the target object based on the change of the world coordinates, so that the aerial robot can track the target object in real time based on the world coordinates and the motion state information. If the current environment does not meet the tracking conditions of the aerial robot, the ground robot is controlled to track the target object. This method can track the target object in real time based on the advantages of the aerial robot's rapid maneuverability and wide field of view when the current environment meets the tracking conditions of the aerial robot. When the current environment does not meet the tracking conditions of the aerial robot, the target object can be tracked based on the endurance of the ground robot and the more detailed and stable tracking, searching and execution capabilities. In this way, the aerial robot and the ground robot can jointly achieve accurate and efficient tracking of the target object in a tracking scene with complex terrain, harsh environment, long time and long distance.

[0090] The above embodiment provides a method for collaborative tracking of a target object. Correspondingly, another embodiment of the present application also provides a device for collaborative tracking of a target object. The device can be applied to a drone, an unmanned vehicle or a control terminal (such as a ground control station). Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For details of the relevant technical features, please refer to the corresponding description of the method embodiment provided above. The following description of the device embodiment is merely illustrative.

[0091] Please refer to Figure 2 To understand this embodiment, Figure 2 A unit block diagram of an apparatus for collaboratively tracking a target object provided in this embodiment, such as Figure 2 As shown, the device for collaborative tracking of a target object provided by this embodiment includes:

[0092] The pixel coordinate obtaining unit 201 is used to obtain a target image including a target object captured by the aerial robot, and obtain pixel coordinates corresponding to the target object in the target image;

[0093] A coordinate and motion state determination unit 202 is used to obtain the world coordinates of the target object based on the pixel coordinates, and obtain the motion state information of the target object based on the change of the world coordinates, so that the aerial robot can track the target object in real time based on the world coordinates and the motion state information;

[0094] The control tracking unit 203 is used to control the ground robot to track the target object when the current environment does not meet the tracking conditions of the aerial robot.

[0095] In some embodiments, controlling the ground robot to track the target object includes:

[0096] Based on the world coordinates and motion state information corresponding to the target object at the current time point, estimate the target world coordinates corresponding to the target object within a predetermined time period, or obtain the target world coordinates of the target object from the aerial robot in real time;

[0097] Control multiple ground robots to perform encirclement detection on the target world coordinates to track the target object.

[0098] In some embodiments, controlling multiple ground robots to perform encirclement detection on the target world coordinates includes:

[0099] Based on the orientation and distance of each ground robot relative to the target world coordinates, the expected encirclement speed of each ground robot for the target world coordinates, and the preset encirclement movement strategy, each ground robot is controlled to perform encirclement detection on the target world coordinates.

[0100] In some embodiments, controlling a ground robot to track a target object includes:

[0101] Based on the world coordinates and motion state information corresponding to the target object at the current time point, estimate the target area corresponding to the target object within a predetermined time period;

[0102] Control multiple ground robots to encircle and detect the target area to track the target object.

[0103] In some embodiments, controlling multiple ground robots to perform encirclement detection on a target area includes:

[0104] Based on the orientation and distance of each ground robot relative to the center point of the target area, the expected encirclement speed of each ground robot with respect to the center point of the target area, and the preset encirclement movement strategy, control each ground robot to perform encirclement detection on the center point of the target area;

[0105] In the process of each ground robot performing encirclement detection on the center point of the target area, in response to any ground robot detecting the target object, the target position of the target object is obtained, and each ground robot is controlled to adjust the encirclement detection target to the target object.

[0106] In some embodiments, the motion state information includes at least one of the following:

[0107] The direction of movement of the target object;

[0108] The speed of the target object;

[0109] The acceleration of the target object.

[0110] In some implementations, obtaining the world coordinates of the target object based on the pixel coordinates includes:

[0111] The pixel coordinates are converted into camera coordinates based on the principles of affine transformation and perspective projection; the camera coordinates are converted into world coordinates based on the principles of rigid body transformation and similarity transformation.

[0112] In the above-mentioned embodiments, a method for collaboratively tracking a target object and an apparatus for collaboratively tracking a target object are provided. In addition, another embodiment of the present application further provides a collaborative control device, which can be provided in an aerial robot or a ground robot, or can be provided independently. Since the collaborative control device embodiment is basically similar to the method embodiment, the description is relatively simple. For details of the relevant technical features, please refer to the corresponding description of the method embodiment provided above. The following description of the collaborative control device embodiment is merely illustrative. The collaborative control device embodiment is as follows:

[0113] Please refer to Figure 3 To understand this embodiment, Figure 3 A schematic diagram of the logical structure of the collaborative control device provided for this embodiment.

[0114] like Figure 3 As shown, the collaborative control device provided in this embodiment includes: a processor 301 and a memory 302;

[0115] The memory 302 is used to store computer instructions for data processing. When the computer instructions are read and executed by the processor 301, the following operations are performed:

[0116] Acquire a target image containing a target object captured by the aerial robot, and obtain pixel coordinates corresponding to the target object in the target image;

[0117] The world coordinates of the target object are obtained based on the pixel coordinates, and the motion state information of the target object is obtained based on the change of the world coordinates, so that the aerial robot can track the target object in real time based on the world coordinates and the motion state information;

[0118] When the current environment does not meet the tracking conditions of the aerial robot, the ground robot is controlled to track the target object.

[0119] In some embodiments, controlling the ground robot to track the target object includes:

[0120] Based on the world coordinates and motion state information corresponding to the target object at the current time point, estimate the target world coordinates corresponding to the target object within a predetermined time period, or obtain the target world coordinates of the target object from the aerial robot in real time;

[0121] Control multiple ground robots to perform encirclement detection on the target world coordinates to track the target object.

[0122] In some embodiments, controlling multiple ground robots to perform encirclement detection on the target world coordinates includes:

[0123] Based on the orientation and distance of each ground robot relative to the target world coordinates, the expected encirclement speed of each ground robot for the target world coordinates, and the preset encirclement movement strategy, each ground robot is controlled to perform encirclement detection on the target world coordinates.

[0124] In some embodiments, controlling a ground robot to track a target object includes:

[0125] Based on the world coordinates and motion state information corresponding to the target object at the current time point, estimate the target area corresponding to the target object within a predetermined time period;

[0126] Control multiple ground robots to encircle and detect the target area to track the target object.

[0127] In some embodiments, controlling multiple ground robots to perform encirclement detection on a target area includes:

[0128] Based on the orientation and distance of each ground robot relative to the center point of the target area, the expected encirclement speed of each ground robot with respect to the center point of the target area, and the preset encirclement movement strategy, control each ground robot to perform encirclement detection on the center point of the target area;

[0129] In the process of each ground robot performing encirclement detection on the center point of the target area, in response to any ground robot detecting the target object, the target position of the target object is obtained, and each ground robot is controlled to adjust the encirclement detection target to the target object.

[0130] In some embodiments, the motion state information includes at least one of the following:

[0131] The direction of movement of the target object;

[0132] The speed of the target object;

[0133] The acceleration of the target object.

[0134] In some implementations, obtaining the world coordinates of the target object based on the pixel coordinates includes:

[0135] The pixel coordinates are converted into camera coordinates based on the principles of affine transformation and perspective projection; the camera coordinates are converted into world coordinates based on the principles of rigid body transformation and similarity transformation.

[0136] In the above-mentioned embodiments, a method for collaboratively tracking a target object, an apparatus for collaboratively tracking a target object, and a collaborative control device are provided. In addition, another embodiment of the present application further provides a computer-readable storage medium for implementing the above-mentioned method. The computer-readable storage medium embodiment provided in the present application is described relatively simply. For the relevant parts, please refer to the corresponding description of the above-mentioned method embodiment. The embodiments described below are merely illustrative.

[0137] The computer-readable storage medium provided in this embodiment stores computer instructions, and when the instructions are executed by a processor, the steps of the above-mentioned method embodiment for collaborative tracking of a target object are implemented.

[0138] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0139] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0140] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0141] 2. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0142] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. A method for collaborative tracking of a target object, characterized in that: The method comprises: Acquire a target image containing a target object captured by the aerial robot, and obtain pixel coordinates corresponding to the target object in the target image; Obtaining the world coordinates of the target object based on the pixel coordinates, and obtaining the motion state information of the target object based on the change of the world coordinates, so that the aerial robot can track the target object in real time based on the world coordinates and the motion state information; In response to the current environment not satisfying the tracking condition of the aerial robot, the ground robot is controlled to track the target object.

2. The method according to claim 1, characterized in that The controlling the ground robot to track the target object includes: Based on the world coordinates and motion state information corresponding to the target object at the current time point, estimate the target world coordinates corresponding to the target object within a predetermined time period, or obtain the target world coordinates of the target object from the aerial robot in real time; Controlling multiple ground robots to perform encirclement detection on the target world coordinates to track the target object.

3. The method according to claim 2, characterized in that The controlling of the plurality of ground robots to perform encirclement detection on the target world coordinates includes: Based on the orientation and distance of each ground robot relative to the target world coordinates, the expected encirclement speed of each ground robot with respect to the target world coordinates, and a preset encirclement movement strategy, each ground robot is controlled to perform encirclement detection on the target world coordinates.

4. The method according to claim 1, characterized in that The controlling the ground robot to track the target object includes: Based on the world coordinates and the motion state information corresponding to the target object at the current time point, estimating a target area corresponding to the target object within a predetermined time period; Controlling multiple ground robots to perform encirclement detection on the target area to track the target object.

5. The method according to claim 4, characterized in that The controlling of the plurality of ground robots to perform encirclement detection on the target area comprises: Based on the orientation and distance of each ground robot relative to the center point of the target area, the expected encirclement speed of each ground robot with respect to the center point of the target area, and the preset encirclement movement strategy, control each ground robot to perform encirclement detection on the center point of the target area; During the process of each of the ground robots performing encirclement detection on the center point of the target area, in response to any of the ground robots detecting the target object, the target position of the target object is obtained, and each of the ground robots is controlled to adjust the target of the encirclement detection to the target object.

6. The method according to any one of claims 1, 2 and 4, characterized in that: The motion state information includes at least one of the following: The direction of movement of the target object; The speed of the target object; The acceleration of the target object.

7. The method according to claim 1, characterized in that Obtaining the world coordinates of the target object based on the pixel coordinates includes: The pixel coordinates are converted into camera coordinates based on the principles of affine transformation and perspective projection; and the camera system coordinates are converted into world coordinates based on the principles of rigid body transformation and similarity transformation.

8. A device for collaborative tracking of a target object, characterized in that: The device comprises: A pixel coordinate obtaining unit, used to obtain a target image captured by the aerial robot and containing a target object, and obtain pixel coordinates corresponding to the target object in the target image; A coordinate and motion state determination unit, used to obtain the world coordinates of the target object based on the pixel coordinates, and obtain the motion state information of the target object based on the change of the world coordinates, so that the aerial robot can track the target object in real time based on the world coordinates and the motion state information; The control tracking unit is used to control the ground robot to track the target object when the current environment does not meet the tracking conditions of the aerial robot.

9. A collaborative control device, characterized in that: comprising a processor and a memory; wherein, The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having one or more computer instructions stored thereon, characterized in that: The instructions are executed by a processor to implement the method according to any one of claims 1 to 7.

Images