Target positioning and tracking method, device and computer-readable storage medium

Abstract

The present application discloses a target positioning and tracking method, apparatus, and computer-readable storage medium. The method includes: obtaining a current image; determining whether the current image contains a target; if the current image contains a target, determining a current position relationship between the target and a robot for accompanying the target based on the current image; where the current image is captured by a monitoring device, the field of view of the monitoring device is the monitoring area where the robot and the target are located, and the current position relationship includes the current distance between the robot and the target and the current orientation of the robot relative to the target; or the current image is captured by the robot, and the current position relationship includes the size of a target area corresponding to the target in the current image and the distance by which the center of the target area deviates from the center of the current image. By the above method, the efficiency of target positioning and tracking can be improved.

Description

Technical Field

[0001] This application relates to the technical field of target tracking, and in particular, to a target positioning and tracking method, apparatus, and computer-readable storage medium. Background Art

[0002] With the development of technology, in order to meet the companionship needs of a target, robots for accompanying the target have emerged as the times require. The robot can accompany the target when the target is unattended. For example, when the owner goes out, the robot accompanies the pet at home to achieve the needs of watching the pet and feeding it regularly. Another example is that when the parent goes out, the robot accompanies the child at home to avoid dangerous situations.

[0003] During the process of the robot accompanying the target, it is necessary to position and track the target to determine the relative position relationship between the robot and the target. However, the current target positioning and tracking methods are not efficient to implement. Summary of the Invention

[0004] This application provides a target positioning and tracking method, apparatus, and computer-readable storage medium, which can solve the problem that the current target positioning and tracking methods are not efficient to implement.

[0005] To solve the above technical problems, one technical solution adopted by this application is: to provide a target positioning and tracking method. The method includes: acquiring a current image; determining whether the current image contains a target; if the current image contains a target, determining the current position relationship between the target and the robot for accompanying the target based on the current image; where the current image is captured by a monitoring device, the field of view of the monitoring device is the monitoring area where the robot and the target are located, the current position relationship includes the current distance between the robot and the target, and the current orientation of the robot relative to the target; or, the current image is captured by the robot, and the current position relationship includes the size of the target area corresponding to the target in the current image, and the distance between the center of the target area and the center of the current image deviating.

[0006] To solve the above technical problems, another technical solution adopted by this application is: to provide a target positioning and tracking device, the target positioning and tracking device includes a processor and a memory connected to the processor, where the memory stores program instructions; the processor is configured to execute the program instructions stored in the memory to implement the above method.

[0007] To solve the above technical problems, yet another technical solution adopted by this application is: to provide a computer-readable storage medium, storing program instructions, which can implement the above method when the program instructions are executed.

[0008] In the above manner, the present application obtains the current image captured by the monitoring device or the robot. When the current image contains the target, the current position relationship between the target and the robot is determined based on the current image (visual positioning and tracking). On the one hand, compared with exploratory positioning and tracking, visual positioning and tracking can save the time required for positioning and tracking and improve the efficiency of target positioning and tracking. On the other hand, the target can be automatically positioned and tracked without wearing any electronic devices, which can reduce the complexity of target positioning and tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a schematic flowchart of an embodiment of the target positioning and tracking method of the present application;

[0010] Figure 2 is a schematic flowchart of another embodiment of the target positioning and tracking method of the present application;

[0011] Figure 3 is a schematic diagram of an application scenario of the present application;

[0012] Figure 4 is a schematic flowchart of yet another embodiment of the target positioning and tracking method of the present application;

[0013] Figure 5 is a schematic diagram of the current image when the current image is captured by the robot;

[0014] Figure 6 is another schematic diagram of an application scenario of the present application;

[0015] Figure 7 is a schematic flowchart of a specific example of the target positioning and tracking method of the present application;

[0016] Figure 8 is a schematic flowchart of another specific example of the target positioning and tracking method of the present application;

[0017] Figure 9 is a schematic structural diagram of a target positioning and tracking system;

[0018] Figure 10 is a schematic structural diagram of an embodiment of the target positioning and tracking device of the present application;

[0019] Figure 11 is a schematic structural diagram of another embodiment of the target positioning and tracking device of the present application;

[0020] Figure 12 is a schematic structural diagram of an embodiment of the computer-readable storage medium of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts belong to the scope of protection of the present application.

[0022] The terms "first", "second", and "third" in the present application are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first", "second", and "third" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0023] Referring to "embodiments" herein means that the specific features, structures, or characteristics described in connection with the embodiments may be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that, without conflict, the embodiments described herein may be combined with other embodiments.

[0024] Figure 1 It is a schematic flowchart of an embodiment of the target positioning and tracking method of the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 1 the process sequence shown.

[0025] As Figure 1 shown, this embodiment may include:

[0026] S11: Obtain the current image.

[0027] The current image is captured by a monitoring device or a robot used to accompany the target.

[0028] The execution subject of the method embodiment of the present application is a target positioning and tracking device, and the target positioning and tracking device may be a monitoring device, other electronic devices in communication connection with the monitoring device, a robot, or other electronic devices in connection with the robot.

[0029] The target can be an animal (such as a pet cat or a pet dog), a person of a preset age group (such as a child or an elderly person), or other objects with a need for companionship. The robot for accompanying the target can include a navigation sensor, which can be an odometer, a gyroscope, an accelerometer, etc. The navigation sensor can be used to accumulate the motion data of the robot and calculate the pose of the robot in real time. The robot can also include a camera, generally with the camera facing the front of the robot (relative to the walking direction of the robot). Both the robot and the target are in the monitoring area.

[0030] The current image refers to the image captured of the monitoring area at the current moment.

[0031] The monitoring device can be an independent camera or a terminal containing a camera. The field of view of the monitoring device is wider than that of the robot, and the field of view of the monitoring device is the monitoring area where the robot and the target are located. The viewing angle of the monitoring device can be directly facing the monitoring area or not directly facing the monitoring area. In the case of not directly facing the monitoring area, if the current image is captured by the monitoring device, before using the current image in subsequent steps, it is necessary to perform a projection transformation on the current image using a projection matrix to convert its shooting angle to the angle directly facing the monitoring area.

[0032] S12: Determine whether the current image contains the target.

[0033] The target position detection algorithm can be used to detect the target in the current image to determine whether it contains the target. In the case where the target is blocked by an obstacle in the monitoring area, such as the target being under a sofa or a table, it may cause the current image not to contain the target.

[0034] If the current image contains the target, execute S13; otherwise, send a lost reminder to the user or execute S14 (continue positioning and tracking).

[0035] S13: Determine the current position relationship between the target and the robot for accompanying the target based on the current image.

[0036] The current position relationship represents the relationship between the current position of the target and the current pose of the robot. The current position of the target is the position of the target in the current image, and the current pose of the robot includes the current position of the robot and the current orientation of the robot relative to the target.

[0037] In the case where the current image is captured by the monitoring device, the current position relationship can include the current distance between the target and the robot and the current orientation of the robot relative to the target. The current distance between the target and the robot is the distance between the current position of the target and the current position of the robot. Based on this, S13 can include:

[0038] Obtain the current position of the target and the current pose of the robot in the current image; based on the current position of the target and the current position of the robot, obtain the current distance between the target and the robot.

[0039] If the current image includes not only the target but also the robot, the current position of the target can be detected from the current image using a target position detection algorithm, and the current pose of the robot can be detected from the current image using a robot pose detection algorithm.

[0040] Alternatively, regardless of whether the current image includes the robot or not, only use the target detection algorithm to detect the current position of the target from the current image, and use the navigation sensor of the robot to accumulate the current pose of the robot. The initial cumulative value of the navigation sensor can be manually input or detected from the first image captured by the monitoring device using the robot pose detection algorithm. In this way, the current pose of the robot directly accumulated by the navigation sensor throughout the positioning and tracking process can save the computing resources required to enable the robot pose detection algorithm.

[0041] Alternatively, if the current image does not include the robot, use the target position detection algorithm to detect the current position of the target from the current image, and use the navigation sensor of the robot to accumulate the current pose of the robot based on the historical pose of the robot. The historical pose of the robot is determined based on the historical images captured by the monitoring device, that is, the historical images include the robot, and the historical pose is detected from the historical images using the robot pose detection algorithm. After detecting the historical pose, align the historical pose accumulated by the navigation sensor at the historical moment to the detected historical pose. In this way, if the monitoring device captures the robot, the current pose of the robot can be determined through the detection algorithm. If the robot is blocked by an obstacle in the monitored area and the monitoring device does not capture the robot, since the historical pose accumulated by the navigation sensor will be aligned to the detected historical pose after detecting the historical pose of the robot through the historical images, the navigation sensor can further accumulate the current pose of the robot based on the aligned historical pose. Thus, through the combination of the current image and the navigation sensor, not only can the current pose of the robot be determined through the current image when the monitoring device captures the robot, but also the current pose of the robot can be accumulated by the navigation sensor based on the historical pose when the monitoring device does not capture the robot, ensuring the normal determination of the current pose of the robot without loss of accuracy.

[0042] In the case where the current image is captured by the robot, the current position relationship may include the size of the target area corresponding to the target in the current image and the distance by which the center of the target area deviates from the center of the current image. Based on this, S13 may include:

[0043] Determine the target area corresponding to the target in the current image; obtain the size of the target area and the distance by which the center of the target area deviates from the center of the current image, as the current positional relationship. The target area can be the connected domain or the circumscribed rectangle of the target in the current image.

[0044] S14: Determine the current positional relationship between the target and the robot in other ways.

[0045] In some embodiments, the target can be searched for at at least one reference position (exploratory positioning to track the target), and based on the pose of the robot and the position of the target when the target is searched for, the current positional relationship is determined. Thus, the probability of searching for the target is increased, and the probability of successfully determining the current positional relationship by target positioning and tracking is increased.

[0046] The reference position can be randomly generated. Or, the reference position can be a position where the possibility of the target appearing is relatively high.

[0047] The position where the possibility of appearance is relatively high can be several historical positions of the target. The several historical positions of the target include the positions corresponding to the target at several historical moments respectively. The several historical moments can include at least one historical moment before the current moment, for example, including a preset number of historical moments closest to the current moment. The historical position of the target can be determined by the historical images captured by the monitoring device at historical moments, that is, the position of the target in the historical images captured by the monitoring device. Or, the historical position of the target can be determined by the historical images captured by the robot, that is, obtained by converting the position of the target in the historical image according to the historical positional relationship between the robot and the target when the historical image is captured.

[0048] Or, the reference position can be the cluster center of each clustering result, and each clustering result is obtained by clustering several historical positions of the target.

[0049] Or, the reference position can be several historical positions of the robot. The method for obtaining the historical position of the robot is similar to the method for obtaining the current position of the robot. Please refer to the specific description in S13 and will not be elaborated here.

[0050] The target can be searched for at each reference position in sequence until the target is searched for. If the reference position is a position where the possibility of appearance is relatively high, the search order can be random. In the case where the position where the possibility of appearance is relatively high is several historical positions of the target, the search order can also be the order from the smallest interval between the corresponding historical moment and the current moment to the largest; in the case where the position where the possibility of appearance is relatively high is the cluster center, the search order can also be the order from the largest weight of each cluster center to the smallest. The greater the weight of the cluster center, the higher the possibility that the target appears at this cluster center.

[0051] The weight of the clustering center can be determined by the number of historical positions in the clustering result corresponding to the clustering center, that is, the weight of the clustering center is positively correlated with the number of historical positions in the corresponding clustering result, or the weight of the clustering center is equal to the number of historical positions in the corresponding clustering result. Alternatively, the weight of the clustering center is determined by the inter-class distances of the respective historical positions in the clustering result corresponding to the clustering center, that is, the weight of the clustering center is positively correlated with the inter-class distances of the respective historical positions in the corresponding clustering result, or the weight of the clustering center is equal to the inter-class distances of the respective historical positions in the corresponding clustering result.

[0052] The process of searching for a target at a reference position is illustrated by the following example: The reference positions include p1 to pn. The robot can be controlled to move to p1 based on a navigation sensor; the robot is controlled to rotate at p1, and the search image captured by the robot at the rotation angle is recorded; it is determined whether the search image contains the target; if the target is contained, the search is stopped, and the current position relationship is determined based on the search image. The method for determining the current position relationship based on the search image is similar to the method for obtaining the current position relationship based on the current image captured by the robot; if the target is not contained, the robot continues to rotate and capture a new search image at p1 until it rotates a full circle at p1; if the target is not found after rotating a full circle, p1 is marked as a reference position that has been searched, and the search continues at p2, and so on, until the target is found.

[0053] If the reference position is a position with a relatively high probability of occurrence and the target is not found at all positions with a relatively high probability of occurrence, the search can also continue at randomly generated positions until the target is found.

[0054] In some embodiments, if the current image in S11 to S13 is captured by a monitoring device, a new current image can be obtained in S14, and the new current image is captured by the robot; it is determined whether the new current image contains the target. If the target is contained, the current position relationship is determined based on the new current image; if the target is not contained, the target is searched for at the reference position to determine the current position relationship. The method for determining the current position relationship based on the new current image is similar to the method for determining the current position relationship when the current image in S13 is captured by the robot, and will not be elaborated here.

[0055] Thus, when the monitoring device cannot locate and track the target, the robot can be used to locate and track the target. When the robot cannot locate and track the target, the target can be located and tracked through exploratory positioning, so as to realize the joint positioning and tracking of the target by the monitoring device and the robot. Thereby, the range of visual positioning and tracking of the target can be expanded, the range of exploratory positioning and tracking of the target can be shortened, the positioning and tracking time can be shortened, and the target can be located and tracked more quickly.

[0056] In some embodiments, if the current image in S11 - S13 is captured by the robot, a new current image can be obtained in S14, and the new current image is captured by the monitoring device; determine whether the new current image contains the target; if it contains the target, determine the current position relationship based on the new current image; if it does not contain the target, search for the target at the reference position to determine the current position relationship. The method of determining the current position relationship based on the new current image is similar to the method of the current position relationship when the current image in S13 is captured by the monitoring device, and will not be elaborated here.

[0057] Thus, when the robot fails to locate and track the target, the monitoring device can be used to locate and track the target. When the monitoring device fails to locate and track the target, the exploratory positioning and tracking method is used to locate and track the target, realizing the joint positioning and tracking of the target by the monitoring device and the robot. Therefore, the range of visual positioning and tracking the target can be expanded, the range of exploratory positioning and tracking the target can be shortened, the positioning and tracking time can be shortened, and the target can be located and tracked more quickly.

[0058] Through the implementation of this embodiment, the present application obtains the current image captured by the monitoring device or the robot, and when the current image contains the target, determines the current position relationship between the target and the robot based on the current image (visual positioning and tracking). On the one hand, compared with exploratory positioning and tracking, visual positioning and tracking can save the time required for positioning and tracking and improve the efficiency of target positioning and tracking; on the other hand, the target can be automatically located and tracked without wearing any electronic devices, which can reduce the complexity of target positioning and tracking.

[0059] Furthermore, the aforementioned target position detection algorithm is implemented based on the trained first neural network, and the robot pose detection algorithm is implemented based on the trained second neural network.

[0060] For the training of the first neural network, a batch of training samples (training images and annotation data) can be obtained in advance. The training images contain training objects, and the training objects can be objects of the same category or similar categories as the target. The annotation data includes the positions of the training objects in the training images (such as the coordinates of the upper left vertex and the lower right vertex); use the batch of training samples to train the first neural network.

[0061] In some embodiments, if the first neural network is used for the images captured by the monitoring device, the shooting subject of the training images can be the monitoring device; if the first neural network is used for the images captured by the robot, the shooting subject of the training images can be the robot, so that the format of the training images is consistent with the image format in the application stage.

[0062] In some embodiments, regardless of whether the images used by the first neural network are captured by a monitoring device or a robot, training images can be captured by a monitoring device, a robot, or other devices, and before training, the format of the training images needs to be aligned with the image format in the application stage.

[0063] In some embodiments, a batch of training samples can be divided into three parts: a training set, a validation set, and a test set. The first neural network is trained on the training set, verified on the validation set, and finally, the network with the most accurate detection results in the test set is selected as the trained first neural network.

[0064] For the training of the second neural network, the difference from the training samples of the first neural network lies in the different annotation data, and the annotation data includes the pose of the training object in the training image (such as the coordinates of the upper left vertex, the lower right vertex, and the orientation angle).

[0065] Furthermore, since in many scenarios, the active interaction between the robot and the target needs to be carried out under an ideal position relationship. For example, a pet companion robot feeds the pet on the premise that the distance between the robot and the target is a fixed distance and the orientation of the robot relative to the target is a fixed orientation. Therefore, after determining the current position relationship, it can be judged whether the current position relationship is an ideal position relationship. If it is not an ideal position relationship, the current pose of the robot is adjusted to update the current position relationship to an ideal position relationship. Specifically, it can be as follows:

[0066] Figure 2 It is a schematic flowchart of another embodiment of the target positioning and tracking method of the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 2 the process sequence shown. In S11 - S13, the current image is captured by a monitoring device, and the current pose of the robot includes the current distance between the robot and the target and the current orientation of the robot relative to the target. As Figure 2 shown, this embodiment may include:

[0067] S21: Based on the connection line between the reference point of the current position of the target and the reference point of the current position of the robot, obtain the ideal orientation of the robot relative to the target.

[0068] The reference point of the current position can be any pixel point in the current position, such as the center, and the same applies to the reference point of the current position. The coordinates of the reference point can be expressed as (x, y), where x and y respectively represent the coordinates in the horizontal direction X and the vertical direction Y in the image coordinate system. The orientation angle of the robot relative to the preset direction (horizontal direction or vertical direction) under the ideal orientation is the orientation angle of the connection line relative to the preset direction. The current distance between the robot and the target is the length of the connection line.

[0069] S22: Determine whether the current orientation is the ideal orientation and whether the current distance is the first ideal distance.

[0070] If the orientation angle of the robot relative to the preset direction under the current orientation is equal to the orientation angle of the connection line relative to the preset direction under the ideal orientation, then it is determined that the current orientation is the ideal orientation.

[0071] If it is not the ideal orientation, then execute S23; if it is not the first ideal distance, then execute S24; if it is the ideal orientation and the first ideal distance, then no adjustment is made.

[0072] S23: Adjust the current orientation of the robot to the ideal orientation.

[0073] The adjustment angle can be obtained by subtracting the orientation angle under the current orientation from the orientation angle under the ideal orientation, and the robot is controlled to rotate this angle to update the current orientation to the ideal orientation.

[0074] S24: Adjust the current position of the robot so that the current distance between the robot and the target is the first ideal distance.

[0075] The adjustment distance can be obtained by subtracting the current distance from the first ideal distance, and the robot is controlled to move this distance to update the current distance to the first ideal distance.

[0076] As follows in combination with Figure 3 An example of S21 - S24 is given as follows:

[0077] Figure 3 It includes a pet and a pet companion robot. In the image coordinate system XOY, the current position of the pet includes the upper left vertex coordinates (x p1 , y p1 ) and the lower right vertex coordinates (x p2 , y p2 ). The current pose of the robot includes the upper left vertex coordinates (x r1 , y r1 ), the lower right vertex coordinates (x r2 , y r2 ) and the orientation angle thetar of the robot relative to the X direction under the current orientation. The connection line between the center of the current position of the robot and the center of the current position of the pet is L. Subtract thetar from the orientation angle of L relative to the X direction to obtain the adjustment angle of the robot; subtract the length of L from the first ideal distance to obtain the adjustment distance of the robot. The robot is adjusted based on the adjustment angle and the adjustment distance to update the current position relationship to the ideal position relationship.

[0078] Through the implementation in this embodiment, the present application can guide the movement of the robot based on the current position relationship between the target and the robot, and adjust the current pose of the robot to maintain the position relationship between the target and the robot as an ideal position relationship.

[0079] Figure 4 It is a schematic flowchart of another embodiment of the target positioning and tracking method of the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 4 the shown process sequence. The current image in S11 - S13 is captured by the robot, and the current position relationship includes the size of the target area and the distance that the center of the target area deviates from the center of the current image. As Figure 4 shown, this embodiment may include:

[0080] S31: Determine whether the size of the target area is the ideal size and whether the distance that the center of the target area deviates from the center of the current image is the second ideal distance.

[0081] With reference to Figure 5 , Figure 5 which is a schematic diagram of the current image when the current image is captured by the robot. As Figure 5 shown, in the current image, the pet area is A, the upper left vertex coordinates of A are (x p1 , y p1 ), the lower right vertex coordinates are (x p2 , y p2 ), and the center line of the current image is a.

[0082] The ideal size is the size of the target area when the distance between the robot and the target is the ideal orientation, and the second ideal distance is the distance that the center of the target area deviates from the center of the current image when the orientation of the robot is the ideal orientation. In some embodiments, the second ideal distance is 0, that is, the target is at the center of the current image.

[0083] If it is not the ideal size, then execute S32; if it is not the second ideal distance, then execute S33; if it is the ideal size and the second ideal distance, then no adjustment is made.

[0084] S32: Adjust the current position of the robot.

[0085] The size of the target area reflects the current distance between the robot and the target. The corresponding relationship between the size deviation and the adjustment distance can be set; the size deviation between the size of the target area in the current image and the ideal size is obtained; the adjustment distance is determined based on the corresponding relationship; and the current position of the robot is adjusted based on the determined distance.

[0086] S33: Adjust the current orientation of the robot.

[0087] The correspondence between the distance deviation and the adjustment angle can be set; obtain the distance deviation between the distance by which the center of the target area deviates from the center of the current image and the second ideal distance; determine the adjustment angle based on the correspondence; adjust the current orientation of the robot based on the determined angle.

[0088] Through the implementation in this embodiment, the present application can guide the movement of the robot based on the current position relationship between the target and the robot, and adjust the current pose of the robot to maintain the position relationship between the target and the robot as an ideal position relationship.

[0089] As follows, in combination with Figures 6 - 8 , a detailed description of the target positioning and tracking method provided by the present application is given in combination with an indoor application scenario:

[0090] With reference to Figure 6 , in this indoor application scenario, monitoring devices are arranged in the monitoring area, a pet is in the monitoring area, and the robot can travel back and forth between the charging pile and the monitoring area.

[0091] 1) Determine whether the robot is powered on; if so, proceed to 2); otherwise, end.

[0092] 2) Determine whether the robot is charging; if so, proceed to 3); otherwise, proceed to 6).

[0093] 3) Determine whether the battery of the robot is fully charged; if not, proceed to 4); otherwise, proceed to 5).

[0094] 4) Continue charging. After a preset time, proceed to 3) to repeatedly determine whether the battery is fully charged.

[0095] 5) Control the robot to leave the charging pile and move towards the monitoring area. Proceed to 6).

[0096] 6) Determine whether the battery level of the robot is lower than the preset value; if so, control the robot to return to the charging pile and proceed to 3); otherwise, proceed to 7).

[0097] 7) After the robot moves to the monitoring area, jointly use the robot and the monitoring device to perform positioning and tracking on the pet in the monitoring area:

[0098] ① Obtain the current image, which is captured by the monitoring device.

[0099] ② Determine whether the current image contains a pet and a robot; if so, proceed to ③; otherwise, proceed to ④.

[0100] ③ Determine the current position relationship between the pet and the robot based on the current image. Specifically, use the target position detection algorithm to obtain the current position of the pet in the current image, and use the robot pose detection algorithm to obtain the current pose of the robot in the current image, so as to obtain the current position relationship. Proceed to ⑧.

[0101] ④Obtain a new current image, which is captured by the robot.

[0102] ⑤Determine whether the new current image contains a pet; if so, proceed to ⑥; otherwise, proceed to ⑦.

[0103] ⑥Determine the current position relationship between the pet and the robot based on the new current image. Specifically, use the target position detection algorithm to obtain the current position of the pet in the current image, and use the navigation sensor to accumulate the current pose of the robot, so as to obtain the current position relationship. Proceed to ⑧.

[0104] ⑦Search for the target at the reference position to determine the current position relationship. Proceed to ⑧.

[0105] ⑧Determine whether the current position relationship is an ideal position relationship; if so, proceed to ⑨; otherwise, return to ① to continue positioning and tracking.

[0106] ⑨Adjust the current pose of the robot to update the current position relationship to the ideal position relationship. Return to ① to continue positioning and tracking.

[0107] Figure 9 is a schematic structural diagram of a target positioning and tracking system. As Figure 9 shown, the target positioning and tracking system includes a monitoring device, a robot, and a target positioning and tracking device.

[0108] Both the monitoring device and the robot can be used to obtain images of the monitoring area; the target positioning and tracking system can receive the images and analyze them to implement the foregoing target positioning and tracking method. For the specific implementation process, please refer to the previous method embodiments and will not be elaborated here.

[0109] In other embodiments, the target positioning and tracking device can be the monitoring device or the robot itself.

[0110] In other embodiments, the target positioning and tracking system can also include only one of the monitoring device or the robot.

[0111] Figure 10 is a schematic structural diagram of an embodiment of the target positioning and tracking device of the present application. As Figure 10 shown, the target positioning and tracking device includes an acquisition module 11, a judgment module 12, and a determination module 13.

[0112] Among them, the acquisition module 11 can be used to acquire a current image, which is captured by the monitoring device or the robot used to accompany the target.

[0113] The judgment module 12 can be used to judge whether the target is included in the current image.

[0114] The determination module 13 can be used to determine the current positional relationship between the target and the robot based on the current image when the target is included in the current image.

[0115] For other detailed descriptions of this embodiment, please refer to the foregoing method embodiments and will not be elaborated herein.

[0116] Through the implementation of this embodiment, the target positioning and tracking device of the present application uses the acquisition module to acquire the current image captured by the monitoring device or the robot, uses the judgment module to judge whether the target is included in the current image, and uses the determination module to determine the current positional relationship between the target and the robot based on the current image when the current image includes the target (visual positioning and tracking). On the one hand, compared with exploratory positioning and tracking, it can save the time required for positioning and tracking and improve the efficiency of target positioning and tracking; on the other hand, the target can automatically achieve positioning and tracking without wearing any electronic devices, which can reduce the complexity of target positioning and tracking.

[0117] Figure 11 It is a schematic structural diagram of another embodiment of the target positioning and tracking device of the present application. As Figure 11 shown, the target positioning and tracking device includes a processor 21 and a memory 22 coupled to the processor 21.

[0118] Among them, the memory 22 stores program instructions for implementing the methods of any of the foregoing embodiments; the processor 21 is configured to execute the program instructions stored in the memory 22 to implement the steps of the foregoing method embodiments. Among them, the processor 21 can also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 21 may be an integrated circuit chip with signal processing capabilities. The processor 21 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0119] Figure 12 It is a schematic structural diagram of an embodiment of the computer-readable storage medium of the present application. As Figure 12As shown, the computer-readable storage medium 30 of the embodiment of the present application stores program instructions 31, and when the program instructions 31 are executed, the method provided in the above embodiments of the present application is implemented. Among them, the program instructions 31 can form a program file and be stored in the above computer-readable storage medium 30 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor can execute all or part of the steps of the methods in various embodiments of the present application. The aforementioned computer-readable storage medium 30 includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, or terminal devices such as computers, servers, mobile phones, and tablets.

[0120] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0121] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. The above are only the embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. A target positioning and tracking method, characterized in that, Including: Obtain the current image; Determine whether the target is included in the current image; If the target is included in the current image, determine a first current position relationship between the target and a robot for accompanying the target based on the current image; wherein, the current image is captured by a monitoring device, and the field of view of the monitoring device is the monitoring area where the robot and the target are located, and the first current position relationship includes the current distance between the robot and the target and the current orientation of the robot relative to the target; If the target is not included in the current image, obtain a new current image, and the new current image is captured by the robot; Determine whether the new current image includes the target; If the new current image includes the target, determine a second current position relationship between the target and the robot based on the new current image; the second current position relationship includes the size of a target area corresponding to the target in the new current image and the distance by which the center of the target area deviates from the center of the new current image; If the target is not included in the new current image, control the robot to move to at least one reference position to capture a search image, and in the case where the search image includes the target, determine a third current position relationship between the target and the robot based on the search image; wherein, the at least one reference position is randomly generated; or the at least one reference position is several historical positions of the target; or the at least one reference position is the cluster center of each clustering result, and the each clustering result is obtained by clustering several historical positions of the target, and the several historical positions of the target include the historical positions respectively corresponding to the target at several historical moments; or the at least one reference position is several historical positions of the robot; in the case where the at least one reference position is several historical positions of the target, the search order of each reference position is the order from smallest to largest of the interval between the corresponding historical moment and the current moment, and the current moment is the moment when the current image is captured; in the case where the at least one reference position is the cluster center of each clustering result, the search order of each reference position is the order from largest to smallest of the weights of each cluster center, and the weight of the cluster center is positively correlated with the number of historical positions in the corresponding clustering result.

2. The method according to claim 1, wherein In the case where the current image further includes the robot, the determining a first current position relationship between the target and a robot for accompanying the target based on the current image includes: Obtain the current position of the target and the current pose of the robot in the current image, and the current pose of the robot includes the current position of the robot and the current orientation of the robot relative to the target; Based on the current position of the target and the current position of the robot, obtain the current distance between the target and the robot.

3. The method according to claim 2, wherein The current pose of the robot is obtained by accumulating on the basis of the historical pose of the robot by the navigation sensor of the robot, and the historical pose of the robot is determined based on the historical images captured by the monitoring device.

4. The method according to claim 1, characterized in that, The determining the second current position relationship between the target and the robot based on the new current image includes: Determining a target area corresponding to the target in the current image; Obtaining the size of the target area and the distance by which the center of the target area deviates from the center of the current image as the second current position relationship.

5. The method according to claim 1, characterized in that, The controlling the robot to move to at least one reference position to capture search images includes: Determining the weights of the respective clustering centers, where the weights of the clustering centers are positively correlated with the number of the historical positions in the corresponding clustering results; Controlling the robot to move to each of the clustering centers to capture the search images according to the order from the largest to the smallest of the weights.

6. The method according to claim 1, wherein The method further includes: If the first current position relationship or the second current position relationship is not an ideal position relationship, adjusting the current pose of the robot to update the first current position relationship or the second current position relationship to the ideal position relationship.

7. The method according to claim 6, characterized in that, The if the first current position relationship is not an ideal position relationship, adjusting the current pose of the robot includes: Obtaining an ideal orientation of the robot relative to the target based on the connection line between the reference point of the current position of the target and the reference point of the current position of the robot; If the current orientation is not the ideal orientation, adjusting the current orientation of the robot to the ideal orientation, and if the current distance is not the first ideal distance, adjusting the current position of the robot to update the current distance between the robot and the target to the first ideal distance.

8. The method according to claim 6, wherein The if the second current position relationship is not an ideal position relationship, adjusting the current pose of the robot includes: If the size of the target area is not the ideal size, adjusting the current position of the robot, and if the distance by which the center of the target area deviates from the center of the current image is not the second ideal distance, adjusting the current orientation of the robot.

9. A target positioning and tracking device, characterized in that, Including a processor and a memory connected to the processor, where The memory stores program instructions; The processor is configured to execute the program instructions stored in the memory to implement the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions, and the program instructions can be executed by a processor and, when executed, implement the method according to any one of claims 1-8.

Images