Target following method, device, robot and readable storage medium

Abstract

The present application discloses a target following method, device, robot and readable storage medium. The method comprises: determining whether the current image frame includes the tracking target according to the image frame related to the current image frame; if the tracking target is included, controlling the robot to follow the tracking target according to the target center coordinates of the tracking target and the image center coordinates of the current image frame; if the tracking target is not included, repeatedly controlling the head rotation of the robot according to the recorded optical flow moving direction and optical flow moving speed and determining whether the next image frame includes the tracking target, until an image including the tracking target is obtained, and using the image including the tracking target as the current image frame to return to the step of controlling the robot to follow the tracking target. The present application can improve the sensitivity of the robot, and when the tracking target is lost, the tracking target can be automatically retrieved to avoid tracking failure.

Description

Technical Field

[0001] The present invention relates to the field of robot control technology, and in particular to a target following method, device, robot and readable storage medium. Background Art

[0002] In recent years, as a high-tech, robotics has gradually penetrated into all aspects of our lives, from production workshops to hospitals, and the role played by robots is immeasurable. In order to meet people's higher needs and improve human-computer interaction technology, robots need to be able to automatically identify the tracking target and follow the tracking target based on the recognition results.

[0003] However, in scenarios such as factories, when the robot is performing a target following task, the robot maintains a certain safe distance from the target. When encountering a T-shaped intersection, a crossroads, or a more complex road fork, after following the target and turning, it may be blocked by obstacles such as walls, and the followed target will temporarily disappear from the robot's field of view. If the robot cannot retrieve the tracking target in time, the target will eventually completely leave the robot's field of view, causing tracking failure. Summary of the invention

[0004] In view of the above problems, the present application proposes a target following method, device, robot and readable storage medium to improve the sensitivity of the robot and avoid tracking failure.

[0005] In a first aspect, an embodiment of the present application provides a target following method, the method comprising:

[0006] Determining whether the current image frame includes a tracking target according to image frames related to the current image frame;

[0007] In the case where the tracking target is included, controlling the robot to follow the tracking target according to the target center coordinates of the tracking target and the image center coordinates of the current image frame;

[0008] In the case where the tracking target is not included, the steps of controlling the rotation of the robot's head according to the recorded optical flow movement direction and optical flow movement speed and determining whether the tracking target is included in the next image frame are repeatedly executed until an image including the tracking target is obtained, and then the image including the tracking target is used as the current image frame to return to the step of controlling the robot to follow the movement of the tracking target.

[0009] The target following method described in the embodiment of the present application, wherein the robot is controlled to follow the movement of the tracking target according to the target center coordinates of the tracking target and the image center coordinates of the current image frame, comprises:

[0010] Determine whether a first distance between the target center coordinates of the tracking target and the image center coordinates of the current image frame is less than or equal to a first threshold;

[0011] If the first distance is less than or equal to the first threshold, taking the next image frame as the current image frame, and returning to the step of determining whether the current image frame includes the tracking target according to the image frames related to the current image frame;

[0012] If the first distance is greater than the first threshold, it is determined whether to control the robot to follow the tracking target according to a second distance between the current position of the robot and the target edge position of the activity area.

[0013] The target following method described in the embodiment of the present application, wherein determining whether to control the robot to follow the movement of the tracking target according to the second distance between the current position of the robot and the target edge position of the activity area includes:

[0014] Predicting the movement direction of the robot according to the target center coordinates and the image center coordinates;

[0015] Determine the target edge position on the edge of the activity area according to the motion direction;

[0016] Determining whether a second distance between the current position of the robot and the target edge position is less than or equal to a second threshold;

[0017] If the second distance is greater than the second threshold, the robot is controlled to move in the movement direction within the activity area; or, if the second distance is less than or equal to the second threshold, an optical flow algorithm is started to determine and record the optical flow moving direction and optical flow moving speed of the tracking target based on the current image frame and the next image frame through the optical flow algorithm;

[0018] The next image frame is used as the current image frame, and the step of determining whether the current image frame includes the tracking target according to the image frames related to the current image frame is returned.

[0019] The target following method described in the embodiment of the present application further includes:

[0020] After starting the optical flow algorithm, if the first distance between the target center coordinates of the tracked target and the image center coordinates of the current image frame is less than or equal to the first threshold, the next image frame is used as the current image frame and the optical flow algorithm is stopped.

[0021] The target following method described in the embodiment of the present application, wherein determining whether the current image frame includes the tracking target according to the image frame related to the current image frame, comprises:

[0022] Determine whether the total number of image frames before the current image frame is greater than or equal to N;

[0023] If the total number of image frames before the current image frame is greater than or equal to N, the Nth image frame before the current image frame is used as the relevant image frame; or, if the total number of image frames before the current image frame is less than N, a pre-acquired initial image frame is used as the relevant image frame;

[0024] A pre-trained target tracking model is used to determine whether the current image frame includes the tracking target based on the relevant image frame and the current image frame.

[0025] The target tracking method described in the embodiment of the present application, wherein the target tracking model includes a first feature extraction module, a second feature extraction module and a classifier, and wherein the use of the pre-trained target tracking model to determine whether the current image frame includes the tracking target based on the relevant image frame and the current image frame includes:

[0026] The first feature extraction module is used to extract a first feature vector from the relevant image frame;

[0027] The second feature extraction module is used to extract a second feature vector from the current image frame;

[0028] The classifier is used to determine whether the current image frame includes the target center coordinates of the tracked target according to the first feature vector and the second feature vector.

[0029] The target following method described in the embodiment of the present application, the target tracking model also includes an intersection-over-union ratio calculation module, and also includes:

[0030] In the case of including the target center coordinates of the tracking target, the intersection-over-union calculation module is used to determine the target tracking frame corresponding to the tracking target from multiple candidate frames according to the first feature vector and the second feature vector, and the multiple candidate frames are determined based on the target center coordinates and multiple preset candidate frame sizes.

[0031] The target following method described in the embodiment of the present application further includes:

[0032] The target tracking model is pre-trained using a plurality of background image samples that do not include the tracking target and a plurality of target image samples that include the tracking target.

[0033] In a second aspect, an embodiment of the present application further provides a target following device, the device comprising:

[0034] A target determination module, used for determining whether the current image frame includes a tracking target according to image frames related to the current image frame;

[0035] A first control module, configured to control the robot to follow the tracking target according to the target center coordinates of the tracking target and the image center coordinates of the current image frame when the tracking target is included;

[0036] The second control module is used to repeatedly control the rotation of the robot's head according to the recorded optical flow movement direction and optical flow movement speed and determine whether the tracking target is included in the next image frame until an image including the tracking target is obtained, and then use the image including the tracking target as the current image frame to return to the step of controlling the robot to follow the movement of the tracking target.

[0037] In a third aspect, an embodiment of the present application further proposes a robot, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program runs on the processor, the target following method described in the embodiment of the present application is executed.

[0038] In a fourth aspect, an embodiment of the present application further proposes a readable storage medium storing a computer program, which executes the target following method described in the embodiment of the present application when running on a processor.

[0039] The present application determines whether the current image frame includes the tracking target based on the image frames related to the current image frame; if the tracking target is included, the robot is controlled to follow the tracking target according to the target center coordinates of the tracking target and the image center coordinates of the current image frame; if the tracking target is not included, the robot's head rotation is repeatedly controlled according to the recorded optical flow movement direction and optical flow movement speed, and it is determined whether the next image frame includes the tracking target until an image including the tracking target is obtained, and the image including the tracking target is used as the current image frame to return to the step of controlling the robot to follow the tracking target. The present application can improve the sensitivity of the robot, and when the tracking target is lost, the tracking target can be automatically retrieved to avoid tracking failure. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solution of the present invention, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope of protection of the present invention. In each of the drawings, similar components are numbered similarly.

[0041] Figure 1 A schematic diagram of a process flow of a target following method proposed in an embodiment of the present application is shown;

[0042] Figure 2 A schematic diagram showing a process of determining related image frames in a target following method proposed in an embodiment of the present application is shown;

[0043] Figure 3 A schematic diagram of the structure of a target tracking model proposed in an embodiment of the present application is shown;

[0044] Figure 4 A schematic diagram of a flow chart of a following process in a target following method proposed in an embodiment of the present application is shown;

[0045] Figure 5 A schematic flow chart of another following process in a target following method proposed in an embodiment of the present application is shown;

[0046] Figure 6 A schematic diagram of the structure of a target following device proposed in an embodiment of the present application is shown;

[0047] Figure 7 A schematic structural diagram of a robot proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0049] The components of the embodiments of the present invention generally described and shown in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0050] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present invention, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or adding the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0051] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0052] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present invention belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present invention.

[0053] In order to ensure the successful completion of long-term target following tasks, this application proposes a target following solution that combines an optical flow algorithm and a target tracking algorithm, which effectively solves the problem of insufficient accuracy of real-time tracking algorithms on existing embedded devices. By introducing an optical flow algorithm to determine the target's moving direction, the problem of target loss that may occur during long-term following is solved.

[0054] The target tracking method proposed in this application will be explained in detail with specific embodiments below.

[0055] Example 1

[0056] See also Figure 1 In one embodiment of the present application, a target following method is provided, the method comprising the following steps:

[0057] Step S100: determining whether the current image frame includes a tracking target according to image frames related to the current image frame.

[0058] For example, see Figure 2 , the related image frames of the current image frame can be determined by following steps S110 to S130:

[0059] Step S110, determining whether the total number of image frames before the current image frame is greater than or equal to N.

[0060] If the total number of image frames before the current image frame is greater than or equal to N, step S120 is executed; or if the total number of image frames before the current image frame is less than N, step S130 is executed.

[0061] Step S120: taking the Nth image frame before the current image frame as the related image frame.

[0062] Step S130: Using the pre-acquired initial image frame as the related image frame.

[0063] For example, if N is equal to 60, that is, when the total number of image frames before the current image frame is greater than or equal to 60, the 60th frame before the current frame is used as the current image frame of the current frame; when the total number of image frames before the current frame is less than 60, the pre-acquired initial image frame is used as the related image frame.

[0064] The tracking target in the initial image frame is pre-marked. The tracking target may be manually marked by a user or may be marked after automatic recognition by a recognition model.

[0065] After determining the image frame of the current image, in this embodiment, one implementation method is to use an existing target tracking algorithm, such as SINT (Siamese Instance Search for Tracking) and SINT++ (optimized Siamese Instance Search for Tracking), to determine whether the current image frame includes the tracking target based on related image frames and the current image frame.

[0066] However, considering that the existing target tracking algorithms have insufficient accuracy when the robot performs following tasks, in this embodiment, another implementation method is given. After determining the image frame of the current image, a pre-trained target tracking model can be used to determine whether the current image frame includes the tracking target based on the related image frames and the current image frame.

[0067] See also Figure 3 The target tracking model includes a first feature extraction module, a second feature extraction module and a classifier. The first feature extraction module is used to extract a first feature vector from the relevant image frame; the second feature extraction module is used to extract a second feature vector from the current image frame; the classifier is used to determine whether the current image frame includes the target center coordinates of the tracking target based on the first feature vector and the second feature vector.

[0068] Among them, the first feature extraction module can be a ResNet18, and the second feature extraction module can be another ResNet18.

[0069] Furthermore, in this embodiment, the target tracking model also includes an intersection-and-union calculation module. When the classifier determines that the current image frame includes the target center coordinates of the tracking target, the intersection-and-union calculation module can determine the target tracking frame corresponding to the tracking target from multiple candidate frames based on the first feature vector and the second feature vector, and the multiple candidate frames are determined based on the target center coordinates and multiple preset candidate frame sizes.

[0070] It should be noted that in order to improve the accuracy of the target tracking model, this embodiment uses multiple background image samples that do not include the tracking target and multiple target image samples that include the tracking target to pre-train the target tracking model, so that the trained target tracking model can accurately identify the tracking target in the current image frame.

[0071] Further, when it is determined in step S100 that the current image frame includes the tracking target, that is, the tracking target is not lost, step S200 is executed; when the current image frame does not include the tracking target, that is, the tracking target is lost, step S300 is executed.

[0072] Step S200: controlling the robot to follow the tracking target according to the target center coordinates of the tracking target and the image center coordinates of the current image frame.

[0073] Step S300, repeatedly controlling the rotation of the robot's head according to the recorded optical flow moving direction and optical flow moving speed and determining whether the next image frame includes the tracking target, until an image including the tracking target is obtained, and taking the image including the tracking target as the current image frame.

[0074] After taking the image including the tracking target as the current image frame, the process returns to the above step S200 to enable the robot to follow the tracking target and perform a long-term target following task.

[0075] The target following method proposed in this embodiment is a single target following method for embedded devices (e.g., robots). Unlike most single target following methods, the target following method proposed in this embodiment combines an optical flow algorithm with a single target tracking algorithm. This embodiment introduces a target tracking model and adds background information near the target as a negative sample to train the classifier of the target tracking model, thereby improving the tracking accuracy of the algorithm. In addition, while improving the accuracy of the model, the target tracking model also has good expansion capabilities. The target tracking model can be trained by updating the training samples so that the target tracking model can be quickly applied to different scenes such as factories, homes, and outdoors. The target tracking model has good versatility and practicality.

[0076] In addition, this embodiment introduces an optical flow algorithm to determine and record the optical flow moving direction and optical flow moving speed, so that the robot's field of vision can find the tracking target that has left the field of vision again, thereby achieving the purpose of long-term tracking of the target.

[0077] Example 2

[0078] It should be noted that in order to make the robot more sensitive and the tracked target less likely to be lost, the robot can be controlled to ensure that the target center coordinates of the tracked target are near the image center coordinates of the current image frame to reduce the loss rate of the tracked target.

[0079] For example, see Figure 4 In order to reduce the loss rate of the tracked target, the above step S200 may include the following steps S210 to S230:

[0080] Step S210, determining whether a first distance between the target center coordinates of the tracking target and the image center coordinates of the current image frame is less than or equal to a first threshold.

[0081] If the first distance is less than or equal to the first threshold, step S220 is executed to use the next image frame as the current image frame and return to the above step S100. If the first distance is greater than the first threshold, step S230 is executed.

[0082] Step S230: determining whether to control the robot to follow the tracking target according to a second distance between the current position of the robot and the target edge position of the activity area.

[0083] In order to avoid collision, in this embodiment, the robot is controlled not to move to the target edge position where a collision may occur.

[0084] For example, see Figure 5 , the step S230 includes the following steps:

[0085] Step S231, predicting the movement direction of the robot according to the target center coordinates and the image center coordinates.

[0086] Step S232: determining the target edge position on the edge of the activity area according to the movement direction.

[0087] Step S233, determining whether a second distance between the current position of the robot and the target edge position is less than or equal to a second threshold.

[0088] If the second distance is greater than the second threshold, step S234 is executed; or if the second distance is less than or equal to the second threshold, step S235 is executed.

[0089] Step S234, controlling the robot to move in the movement direction within the activity area.

[0090] Step S235 , starting the optical flow algorithm to determine and record the optical flow moving direction and optical flow moving speed of the tracking target based on the current image frame and the next image frame through the optical flow algorithm.

[0091] It should be noted that the Lucas-Kanade optical flow algorithm has three major assumptions:

[0092] (1) Constant brightness: When a pixel moves between two consecutive frames of a video, the brightness (or color) of the pixel in the image remains unchanged.

[0093] (2) Spatial consistency: Pixels that are adjacent in three-dimensional space remain adjacent in a two-dimensional plane, and the movement speeds of the two adjacent points are similar.

[0094] (3) The motion is small: In an image, when an object undergoes optical flow motion, the movement speed of the pixels between two adjacent frames is small and the range of motion is limited.

[0095] Based on the above assumptions, let f(x,y,t) be the color feature value of the pixel point (x,y) in the image at time t, dx be the offset distance in the x direction, and dy be the offset distance in the y direction, and the optical flow limiting equation is obtained:

[0096]

[0097] set up:

[0098]

[0099] available:

[0100] Based on the spatial consistency assumption of the optical flow method, two adjacent points have similar motion in their three-dimensional space and are mapped to two adjacent points in the two-dimensional plane. Therefore, a set of equations can be formed using the data of the pixels around the central pixel:

[0101]

[0102] By calculation, we can get the v of the center pixel x , v y , that is, the direction and speed of the optical flow following the target.

[0103] Step S236: taking the next image frame as the current image frame.

[0104] After executing step S236, return to the above step S100. After returning to the above step S100, the optical flow algorithm is started and is in the start-up state. If the first distance between the target center coordinates of the tracked target and the image center coordinates of the current image frame is less than or equal to the first threshold, the next image frame is used as the current image frame and the optical flow algorithm is stopped. This is to avoid wasting the robot's computing power and occupying the robot's resources by starting the optical flow algorithm for a long time.

[0105] Example 3

[0106] See also Figure 6 In another embodiment of the present application, a target following device 10 is provided. The device includes: a target determination module 11, a first control module 12 and a second control module 13.

[0107] A target determination module 11 is used to determine whether the current image frame includes the tracking target based on the related image frames of the current image frame; a first control module 12 is used to control the robot to follow the tracking target according to the target center coordinates of the tracking target and the image center coordinates of the current image frame when the tracking target is included; a second control module 13 is used to repeatedly control the head rotation of the robot according to the recorded optical flow movement direction and optical flow movement speed and determine whether the next image frame includes the tracking target until an image including the tracking target is obtained, and then use the image including the tracking target as the current image frame to return to the first control module 12 that controls the robot to follow the tracking target.

[0108] In this embodiment, the target following device 10 uses the target determination module 11, the first control module 12 and the second control module 13 in cooperation to execute the target following method described in the above embodiment. The implementation scheme and beneficial effects involved in the above embodiment are also applicable to this embodiment and will not be repeated here.

[0109] Example 4

[0110] See also Figure 7 The fourth embodiment of the present application further proposes a robot 100, including a memory 110 and a processor 120, wherein the memory 110 stores a computer program, and when the computer program runs on the processor 120, the target following method described in the above embodiment is executed.

[0111] Example 5

[0112] A fifth embodiment of the present application further proposes a readable storage medium storing a computer program, wherein the computer program, when executed on a processor, executes the target following method described in the above embodiment of the present application.

[0113] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow chart, and the combination of boxes in the structure diagram and / or the flow chart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0114] In addition, the functional modules or units in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0115] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned readable storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0116] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A target following method, characterized in that: The method comprises: Determining whether the current image frame includes a tracking target according to image frames related to the current image frame; In the case of including the tracking target, controlling the robot to follow the tracking target according to whether a first distance between the target center coordinates of the tracking target and the image center coordinates of the current image frame is less than or equal to a first threshold; In the case where the tracking target is not included, repeatedly controlling the head rotation of the robot according to the recorded optical flow moving direction and optical flow moving speed and determining whether the tracking target is included in the next image frame until an image including the tracking target is obtained, and taking the image including the tracking target as the current image frame to return to the step of controlling the robot to follow the movement of the tracking target; If the first distance is less than or equal to the first threshold, taking the next image frame as the current image frame, and returning to the step of determining whether the current image frame includes the tracking target according to the image frames related to the current image frame; If the first distance is greater than the first threshold, the movement direction of the robot is predicted according to the target center coordinates and the image center coordinates; and the target edge position is determined on the edge of the activity area according to the movement direction; it is determined whether the second distance between the current position of the robot and the target edge position is less than or equal to a second threshold; if the second distance is greater than the second threshold, the robot is controlled to move in the movement direction within the activity area; or, if the second distance is less than or equal to the second threshold, the optical flow algorithm is started to determine and record the optical flow movement direction and optical flow movement speed of the tracked target based on the current image frame and the next image frame through the optical flow algorithm; and the next image frame is used as the current image frame.

2. The target following method according to claim 1, characterized in that: Also includes: After starting the optical flow algorithm, if the first distance between the target center coordinates of the tracked target and the image center coordinates of the current image frame is less than or equal to the first threshold, the next image frame is used as the current image frame and the optical flow algorithm is stopped.

3. The target following method according to claim 1 or 2, characterized in that: The determining, according to the image frames related to the current image frame, whether the current image frame includes the tracking target comprises: Determine whether the total number of image frames before the current image frame is greater than or equal to N; If the total number of image frames before the current image frame is greater than or equal to N, the Nth image frame before the current image frame is used as the relevant image frame; or, if the total number of image frames before the current image frame is less than N, a pre-acquired initial image frame is used as the relevant image frame; A pre-trained target tracking model is used to determine whether the current image frame includes the tracking target based on the relevant image frame and the current image frame.

4. The target following method according to claim 3, characterized in that: The target tracking model includes a first feature extraction module, a second feature extraction module and a classifier, and the using of the pre-trained target tracking model to determine whether the current image frame includes the tracking target based on the related image frame and the current image frame includes: The first feature extraction module is used to extract a first feature vector from the relevant image frame; The second feature extraction module is used to extract a second feature vector from the current image frame; The classifier is used to determine whether the current image frame includes the target center coordinates of the tracked target according to the first feature vector and the second feature vector.

5. The target following method according to claim 4, characterized in that: The target tracking model also includes an intersection-over-union calculation module, and also includes: In the case of including the target center coordinates of the tracking target, the intersection-over-union calculation module is used to determine the target tracking frame corresponding to the tracking target from multiple candidate frames according to the first feature vector and the second feature vector, and the multiple candidate frames are determined based on the target center coordinates and multiple preset candidate frame sizes.

6. The target following method according to claim 3, characterized in that: Also includes: The target tracking model is pre-trained using a plurality of background image samples that do not include the tracking target and a plurality of target image samples that include the tracking target.

7. A target following device, characterized in that: The device comprises: A target determination module, used for determining whether the current image frame includes a tracking target according to image frames related to the current image frame; a first control module, configured to coordinately control the robot to follow the tracking target according to whether a first distance between the target center coordinates of the tracking target and the image center of the current image frame is less than or equal to a first threshold value when the tracking target is included; a second control module, configured to repeatedly control the rotation of the robot's head according to the recorded optical flow moving direction and optical flow moving speed and determine whether the next image frame includes the tracking target, until an image including the tracking target is obtained, and then use the image including the tracking target as the current image frame to return to the step of controlling the robot to follow the movement of the tracking target; If the first distance is less than or equal to the first threshold, taking the next image frame as the current image frame, and returning to the step of determining whether the current image frame includes the tracking target according to the image frames related to the current image frame; If the first distance is greater than the first threshold, the movement direction of the robot is predicted according to the target center coordinates and the image center coordinates; and the target edge position is determined on the edge of the activity area according to the movement direction; it is determined whether the second distance between the current position of the robot and the target edge position is less than or equal to a second threshold; if the second distance is greater than the second threshold, the robot is controlled to move in the movement direction within the activity area; or, if the second distance is less than or equal to the second threshold, the optical flow algorithm is started to determine and record the optical flow movement direction and optical flow movement speed of the tracked target based on the current image frame and the next image frame through the optical flow algorithm; and the next image frame is used as the current image frame.

8. A robot, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is run on the processor, the target following method according to any one of claims 1 to 6 is executed.

9. A readable storage medium, characterized in that: The computer program stores a computer program, and when the computer program is run on a processor, the target following method according to any one of claims 1 to 6 is executed.

Images