Robot tracking method, control terminal, robot, and storage medium

Abstract

The application provides a robot tracking method, a control terminal, a robot and a storage medium. The method comprises the following steps: if the confidence of a target tracker is less than a first preset value, determining that an initial target is lost; if the confidence of the target tracker is not less than the first preset value, acquiring an HSV feature of a current frame image, a previous frame image and an initial target, and determining a tracking probability based on the HSV feature of the current frame image, the previous frame image and the initial target; if the tracking probability is less than a second preset value, determining that the initial target is lost; and if the tracking probability is not less than the second preset value, determining that the initial target is not lost. According to the application, whether the target is lost is preliminarily determined according to the confidence, and whether the target is lost is determined again through the HSV feature, so that whether the tracking target is lost can be accurately determined, and the accuracy of the determination is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a robot tracking method, a control terminal, a robot, and a storage medium. Background Technology

[0002] With the continuous development of robotics technology, intelligent service robots are widely used in factories, homes, hospitals, and other scenarios. Target following is an important function of intelligent service robots, which use computer algorithms to identify targets, avoid obstacles, and automatically follow them.

[0003] In existing technologies, neural network models are typically used for machine vision target tracking, with confidence levels used to determine whether the target has been lost. However, confidence level determination is only feasible in relatively obvious situations; it becomes unusable in slightly more complex environments. Summary of the Invention

[0004] This invention provides a robot tracking method, a control terminal, a robot, and a storage medium to solve the problem that existing trackers cannot effectively determine whether they have lost track of the target.

[0005] In a first aspect, embodiments of the present invention provide a robot tracking method applied to a robot, wherein the robot uses a target tracker to track a target; the method includes:

[0006] If the confidence level of the target tracker is less than the first preset value, then the initial target is determined to be lost;

[0007] If the confidence level of the target tracker is not less than the first preset value, the HSV features of the current frame image, the previous frame image and the initial target are obtained, and the tracking probability is determined based on the HSV features of the current frame image, the previous frame image and the initial target.

[0008] If the tracking probability is less than the second preset value, then the initial target is determined to be lost;

[0009] If the tracking probability is not less than the second preset value, then it is determined that the initial target has not been lost.

[0010] Secondly, embodiments of the present invention provide a control terminal, including a processor and a memory, wherein the memory is used to store computer programs, and the processor is used to call and run the computer programs stored in the memory to execute the robot tracking method provided in the first aspect of the present invention.

[0011] Thirdly, embodiments of the present invention provide a robot, including: a control terminal and a robot body provided in the second aspect of the present invention;

[0012] The control terminal is used to control the robot body to track the target.

[0013] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the robot tracking method provided in the first aspect or any possible implementation thereof.

[0014] This invention provides a robot tracking method, a control terminal, a robot, and a storage medium. The method includes: if the confidence level of the target tracker is less than a first preset value, determining that the initial target is lost; if the confidence level of the target tracker is not less than the first preset value, acquiring the HSV features of the current frame image, the previous frame image, and the initial target, and determining a tracking probability based on the HSV features of the current frame image, the previous frame image, and the initial target; if the tracking probability is less than a second preset value, determining that the initial target is lost; if the tracking probability is not less than the second preset value, determining that the initial target is not lost. This invention initially determines whether the target is lost based on the confidence level. If it is confirmed that the target is not lost, it further determines whether the target is indeed not lost by using the HSV features. Combining confidence level with HSV features can accurately determine whether the tracked target is lost, effectively improving the accuracy of the determination. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the implementation of a robot tracking method according to an embodiment of the present invention;

[0017] Figure 2 This is a flowchart illustrating the implementation of another robot tracking method provided in this embodiment of the invention;

[0018] Figure 3 This is a schematic diagram of the robot tracking device provided in an embodiment of the present invention;

[0019] Figure 4 This is a schematic diagram of the control terminal provided in an embodiment of the present invention. Detailed Implementation

[0020] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0022] See Figure 1 The diagram illustrates a flowchart of the robot tracking method provided in this embodiment of the invention. The method is applied to a robot, which uses a target tracker to track a target. For example, the target tracker can be a KCF tracker, a TLD tracker, or other single-target tracker. The executing entity of the method can be the robot's control terminal, such as a central controller carried on the robot, or other controllers capable of controlling the robot. The method includes:

[0023] S101: If the confidence level of the target tracker is less than the first preset value, then the initial target is determined to be lost;

[0024] S102: If the confidence level of the target tracker is not less than the first preset value, then obtain the HSV features of the current frame image, the previous frame image and the initial target, and determine the tracking probability based on the HSV features of the current frame image, the previous frame image and the initial target.

[0025] S103: If the tracking probability is less than the second preset value, then the initial target is determined to be lost;

[0026] S104: If the tracking probability is not less than the second preset value, then it is determined that the initial target has not been lost.

[0027] Confidence level is used to evaluate the accuracy of the target tracker's detection results. A low confidence level directly indicates target loss. When the confidence level is relatively high, to avoid false positives and improve accuracy, HSV (Hue Saturation Value) features can be used for secondary judgment to determine whether the tracked target is lost, thus overcoming the inaccuracy of determining target loss solely through confidence level. In this embodiment of the invention, based on the confidence level judgment, the initial target's HSV features are combined for secondary judgment. The two are used together to determine the target's loss status, effectively improving the accuracy of target tracking loss judgment. Simultaneously, because HSV features are relatively simple, this ensures accuracy while reducing computational power and improving real-time detection.

[0028] It should be noted that the HSV features of the initial target can be extracted using a feature extractor, for example,

[0029] In one possible implementation, S102 may include:

[0030] S1021: Determine the intersection-union ratio (IoU) between the detection bounding box and the tracking bounding box of the current frame image;

[0031] S1022: Determine the scale ratio of the tracking box in the previous frame to the tracking box in the current frame;

[0032] S1023: Determine the cosine similarity between the HSV features of the current target and the HSV features of the initial target;

[0033] S1024: Determine the tracking probability based on the intersection-union ratio, scale ratio, and cosine similarity.

[0034] Intersection over Union (IOU) is the ratio of the intersection to the union of two geometric shapes, used to evaluate the degree of overlap in the areas of two geometric figures. In this embodiment of the invention, it is used to calculate the error between the detection box and the tracking box; the smaller the error, the more likely they overlap, indicating that the target is in the current frame image, and it can evaluate the tracking status of the target to a certain extent.

[0035] The closer the scale ratio of the tracking box in the previous frame to the tracking box in the current frame is to 1, the more consistent the target is in the two frames and the target has not been lost. This can also be used to evaluate the tracking status of the target to some extent.

[0036] Cosine similarity is used to evaluate similarity. In this embodiment of the invention, the cosine similarity between the HSV features of the current target and the HSV features of the initial target is determined, thereby judging the similarity between the current target and the initial target and reflecting the tracking status of the target.

[0037] In this embodiment of the invention, the tracking probability is determined by combining the above three parameters, and the tracking status of the target—whether it has been lost or not—is determined by the tracking probability. The comprehensive judgment based on these three parameters effectively improves the accuracy of the judgment.

[0038] In one possible implementation, S1024 may include:

[0039] 1. Based on the intersection-union ratio, scale ratio, and cosine similarity, and combined with the first formula, determine the tracking probability;

[0040] The first formula can be:

[0041]

[0042] Where P is the tracking probability, x1, x2, and x3 are the intersection-union ratio, scale ratio, and cosine similarity, respectively, and w1, w2, and w3 are the weights corresponding to the intersection-union ratio, scale ratio, and cosine similarity, respectively.

[0043] In this embodiment of the invention, a weight is set for each parameter, and the tracking probability is determined by logistic regression by combining the three parameters. The calculation results are accurate and the calculation process is simple.

[0044] The weights of the three parameters are preset values ​​and can be set according to actual application needs; no limitation is made here. For example, w1, w2, and w3 are 0.2, 0.2, and 0.6, respectively.

[0045] Furthermore, the weights of the three parameters can be determined by learning from the sample set.

[0046] Based on the above, in one possible implementation, the second preset value can be 0.5.

[0047] Specifically, the second preset value can be set according to actual application needs, and is not limited here.

[0048] In one possible implementation, after S101, the above method may further include:

[0049] S105: If the confidence level of the target tracker is less than the first preset value, the depth features of the initial target are obtained, the initial target is found based on the depth features of the initial target, and the initial target is re-locked and tracked.

[0050] If the confidence level is less than the first preset value, it indicates that the tracking target has been lost, and the lost initial target needs to be found. In this embodiment of the invention, since the initial target has been lost, depth features are directly used to find the initial target. Depth features include the target's spatial location, color, scale, and other features. Compared to HSV features, depth features have more features and higher accuracy, and can accurately find the initial target.

[0051] Specifically, when searching for the initial target, the robot can be controlled to rotate in the direction in which the target was lost, and the depth features of the initial target can be matched with the depth features of each suspicious target in the image to find the initial target.

[0052] In one possible implementation, after S103, the above method may further include:

[0053] S106: If the tracking probability is less than the second preset value, then obtain the HSV features of the initial target and find the initial target based on the HSV features of the initial target;

[0054] S107: If the initial target is relocked within the preset time period, the target tracker is updated, and the updated target tracker is used to control the robot to track the relocked target.

[0055] S108: If the initial target is not relocked within the preset time period, the depth features of the initial target are obtained, the initial target is found based on the depth features of the initial target, and the initial target is relocked and tracked.

[0056] When the confidence level of the target tracker is not less than a first preset value and the tracking probability is less than a second preset value, the initial target is confirmed to be lost. Since the initial target is usually nearby and easy to retrieve in the case of short-term loss, this embodiment of the invention uses HSV features with lower computational power to find the initial target, which has a faster calculation speed and is used to quickly retrieve the initial target in the case of short-term loss. If the initial target cannot be re-locked within a preset time period, that is, the loss time is long or it cannot be retrieved for a long time, the initial target may be far away from the robot and difficult to find. In this case, real-time performance is no longer considered, and depth features are used to accurately retrieve the initial target.

[0057] In this embodiment of the invention, HSV features, which require less computational power, are first used to locate the target, ensuring the real-time performance and accuracy of target retrieval. Furthermore, if the target cannot be retrieved quickly, depth features are used to ensure the accuracy and effectiveness of target retrieval. Similarly, the robot is controlled to rotate along the direction in which the initial target was lost to match HSV or depth features.

[0058] For example, the preset duration is the duration for processing a preset number of image frames, such as the duration for 30 image frames.

[0059] It should be noted that after S108, if the initial target still cannot be re-locked using depth features, the robot can be controlled to return to the base station and wait for subsequent instructions.

[0060] In one possible implementation, S108 may include:

[0061] S1081: Acquire the image of the current frame and determine each target in the current frame image;

[0062] S1082: For each target, determine its depth and scale information, and determine whether the target is occluded based on its depth and scale information; if the target is not occluded, use the first feature extractor to extract the target's depth features; if the target is occluded, use the second feature extractor to extract the target's depth features; match the extracted target's depth features with the initial target's depth features to determine whether the target is the initial target.

[0063] When the robot is far from the target and there are no obstructions, the image can cover the entire target, resulting in good sample quality and complete feature extraction. However, when the target is occluded or the robot is too close to the target, the image cannot cover the entire target, resulting in occluded portions, poor sample quality, and incomplete feature extraction. In both cases, using a single model for feature extraction may introduce errors. Therefore, two feature extractors are designed for each target: the first focuses on global features, while the second focuses on local features. This effectively improves the accuracy of feature extraction and recognition, ensuring accurate target identification even when the target undergoes drastic changes or the environment changes.

[0064] Specifically, occluded samples and unoccluded samples can be used to extract and identify features for the two feature extractors respectively.

[0065] In one possible implementation, after S107, S108, or S104, the above method may further include:

[0066] S109: Obtain the depth and scale information of the current target, and determine whether the current target is occluded based on the depth and scale information of the current target;

[0067] S1010: If the current target is unobstructed, add the image of the current target to the first sample set to train the model and obtain the first feature extractor;

[0068] S1011: If the current target is occluded, add the image of the current target to the second sample set to train the model and obtain the second feature extractor.

[0069] Based on the above, after S107, S108, or S104, once the target is re-locked, the currently tracked target becomes the initial target. An image of the current target can be added to the sample set to further train the model (first feature extractor and second feature extractor), expanding the sample set and increasing its quantity. This strengthens the model's training, thereby improving its accuracy. Simultaneously, through continuous learning, it can still accurately identify and track targets even when they change.

[0070] It should be noted that S109 to S1011 can run on the online server without affecting the normal use of the robot.

[0071] In one possible implementation, the first feature extractor and the second feature extractor can be the OSNet model.

[0072] refer to Figure 2 The above method will be described below with reference to specific embodiments.

[0073] 1. Initialize the target tracker by collecting the initial target's HSV and depth features;

[0074] 2. Control the robot to track the initial target;

[0075] 3. Determine whether the confidence level is ≥ TH1; if it is less than the first preset value TH1, select a feature extractor (first feature extractor or second feature extractor) to extract depth features based on the depth information and scale information, lock the target based on the depth features, initialize the target tracker to continue tracking, and add the target image to the sample set at the same time.

[0076] 4. If ≥ TH1, calculate the tracking probability P; if P ≥ 0.5, it means the target has not been lost, continue tracking the initial target, and add the target image to the sample set.

[0077] 5. If P < 0.5, perform HSV feature matching first; if the matching is successful, initialize the target tracker to continue tracking, and add the target image to the sample set.

[0078] 6. If the matching fails, select the feature extractor to extract depth features, lock the target based on the depth features, initialize the target tracker to continue tracking, and add the target image to the sample set.

[0079] In this embodiment of the invention, HSV feature verification is used when the confidence level is high, which can accurately determine whether the target is lost. At the same time, HSV feature computation is low and has good real-time performance. Furthermore, this embodiment of the invention uses HSV to retrieve the target when it is lost for a short time, ensuring real-time performance and accuracy; and when HSV features cannot retrieve the target for a long time, deep feature retrieval is further used to ensure that the target can be retrieved in real time and accurately when it is lost. In addition, two feature extractors are set up to extract local features and global features respectively, ensuring that the target can be accurately retrieved when the target undergoes drastic changes or significant changes in the environment, thus improving the accuracy of tracking.

[0080] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0081] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.

[0082] Figure 3 The diagram illustrates the structure of a robot tracking device provided in an embodiment of the present invention. This device is applied to a robot, which uses a target tracker to track a target. For ease of explanation, only the parts relevant to the embodiment of the present invention are shown, and are detailed below:

[0083] like Figure 3 As shown, the robot tracking device includes:

[0084] The first judgment module 21 is used to determine that the initial target is lost if the confidence level of the target tracker is less than a first preset value;

[0085] The second judgment module 22 is used to obtain the HSV features of the current frame image, the previous frame image and the initial target if the confidence of the target tracker is not less than the first preset value, and determine the tracking probability based on the HSV features of the current frame image, the previous frame image and the initial target.

[0086] The third judgment module 23 is used to determine that the initial target is lost if the tracking probability is less than the second preset value;

[0087] The fourth judgment module 24 is used to determine that the initial target has not been lost if the tracking probability is not less than the second preset value.

[0088] In one possible implementation, the second determination module 22 may include:

[0089] The first parameter determination unit is used to determine the intersection-union ratio of the detection box and the tracking box of the current frame image;

[0090] The second parameter determination unit is used to determine the scale ratio of the tracking box of the previous frame image to the tracking box of the current frame image.

[0091] The third parameter determination unit is used to determine the cosine similarity between the HSV features of the current target and the HSV features of the initial target;

[0092] The probability output unit is used to determine the tracking probability based on the intersection-union ratio, scale ratio, and cosine similarity.

[0093] In one possible implementation, the probability output unit can be specifically used to: determine the tracking probability based on the intersection-union ratio, scale ratio, and cosine similarity, combined with the first formula;

[0094] The first formula can be:

[0095]

[0096] Where P is the tracking probability, x1, x2, and x3 are the intersection-union ratio, scale ratio, and cosine similarity, respectively, and w1, w2, and w3 are the weights corresponding to the intersection-union ratio, scale ratio, and cosine similarity, respectively.

[0097] In one possible implementation, the above-described apparatus may further include:

[0098] The fifth judgment module is used to obtain the depth features of the initial target if the confidence level of the target tracker is less than the first preset value, find the initial target based on the depth features of the initial target, and re-lock and track the initial target.

[0099] In one possible implementation, the above-described apparatus may further include:

[0100] The sixth judgment module is used to obtain the HSV features of the initial target if the tracking probability is less than the second preset value, and to find the initial target based on the HSV features of the initial target.

[0101] The seventh judgment module is used to update the target tracker if the initial target is relocked within a preset time period, and to use the updated target tracker to control the robot to track the relocked target.

[0102] The eighth judgment module is used to obtain the depth features of the initial target if the initial target is not relocked within a preset time period, find the initial target based on the depth features of the initial target, relock the initial target and track it.

[0103] In one possible implementation, the eighth determination module may include:

[0104] The detection unit is used to acquire the image of the current frame and determine each target in the current frame image;

[0105] The target determination unit is used to determine the depth and scale information of each target, and determine whether the target is occluded based on the depth and scale information. If the target is unoccluded, the first feature extractor is used to extract the depth features of the target. If the target is occluded, the second feature extractor is used to extract the depth features of the target. The extracted depth features of the target are matched with the depth features of the initial target to determine whether the target is the initial target.

[0106] In one possible implementation, the above-described apparatus may further include:

[0107] The occlusion determination module is used to acquire the depth and scale information of the current target, and determine whether the current target is occluded based on the depth and scale information of the current target.

[0108] The ninth judgment module is used when the current target is unobstructed. If so, the image of the current target is added to the first sample set to train the model and obtain the first feature extractor.

[0109] The tenth judgment module is used to add the image of the current target to the second sample set to train the model and obtain the second feature extractor if the current target is occluded.

[0110] Figure 4This is a schematic diagram of the control terminal provided in an embodiment of the present invention. Figure 4 As shown, the control terminal 3 in this embodiment includes a processor 30 and a memory 31. The memory 31 is used to store a computer program 32, and the processor 30 is used to call and run the computer program 32 stored in the memory 31 to execute the steps in the various robot tracking method embodiments described above, for example... Figure 1 The steps S101 to S104 are shown. Alternatively, the processor 30 is used to call and run the computer program 32 stored in the memory 31 to implement the functions of each module / unit in the above-described device embodiments, for example... Figure 3 The functions of modules 21 to 24 are shown.

[0111] For example, computer program 32 can be divided into one or more modules / units, one or more of which are stored in memory 31 and executed by processor 30 to complete the present invention. One or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 32 in control terminal 3. For example, computer program 32 can be divided into... Figure 3 Modules / units 21 to 24 are shown.

[0112] The control terminal 3 can be a central controller for the robot, a desktop computer, a laptop, a handheld computer, or a cloud server, etc. The control terminal 3 may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will understand that... Figure 4 This is merely an example of control terminal 3 and does not constitute a limitation on control terminal 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal may also include input / output devices, network access devices, buses, etc.

[0113] The processor 30 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0114] The memory 31 can be an internal storage unit of the control terminal 3, such as a hard disk or RAM of the control terminal 3. The memory 31 can also be an external storage device of the control terminal 3, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the control terminal 3. Furthermore, the memory 31 can include both internal and external storage units of the control terminal 3. The memory 31 is used to store computer programs and other programs and data required by the terminal. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0115] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0116] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0117] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0118] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0119] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0121] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0122] This invention also provides a robot, including the control terminal and robot body provided in the above embodiments;

[0123] The control terminal is used to control the robot body to track the target.

[0124] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A robot tracking method, characterized in that, Applied to robots, the robots use target trackers to track targets; the above method includes: If the confidence level of the target tracker is less than a first preset value, then the initial target is determined to be lost; If the confidence level of the target tracker is not less than the first preset value, then the HSV features of the current frame image, the previous frame image and the initial target are obtained, and the tracking probability is determined based on the HSV features of the current frame image, the previous frame image and the initial target. If the tracking probability is less than the second preset value, then the initial target is determined to be lost; If the tracking probability is not less than the second preset value, then it is determined that the initial target has not been lost; The step of determining the tracking probability based on the HSV features of the current frame image, the previous frame image, and the initial target includes: Determine the intersection-union ratio (IoU) between the detection bounding box and the tracking bounding box of the current frame image; Determine the scale ratio of the tracking box in the previous frame image to the tracking box in the current frame image; Determine the cosine similarity between the HSV features of the current target and the HSV features of the initial target; The tracking probability is determined based on the intersection-union ratio, the scale ratio, and the cosine similarity, combined with the first formula. The first formula is: in, The tracking probability is... These are the intersection-union ratio, the scale ratio, and the cosine similarity, respectively. These are the weights corresponding to the intersection-union ratio, the scale ratio, and the cosine similarity, respectively.

2. The robot tracking method according to claim 1, characterized in that, After determining that the initial target is lost if the confidence level of the target tracker is less than a first preset value, the method further includes: If the confidence level of the target tracker is less than the first preset value, the depth features of the initial target are obtained, the initial target is located based on the depth features of the initial target, and the initial target is re-locked and tracked.

3. The robot tracking method according to claim 2, characterized in that, After determining that the initial target is lost if the tracking probability is less than a second preset value, the method further includes: If the tracking probability is less than the second preset value, then the HSV feature of the initial target is obtained, and the initial target is found based on the HSV feature of the initial target; If the initial target is relocked within a preset time period, the target tracker is updated, and the updated target tracker is used to control the robot to track the relocked target. If the initial target is not relocked within the preset time period, the depth features of the initial target are obtained, the initial target is located based on the depth features, and the initial target is relocked and tracked.

4. The robot tracking method according to claim 3, characterized in that, The step of finding the initial target based on the depth features of the initial target includes: Acquire the image of the current frame and identify each target in the current frame image; For each target, its depth and scale information are determined, and whether the target is occluded is determined based on the depth and scale information. If the target is not occluded, the first feature extractor is used to extract the depth features of the target. If the target is occluded, the second feature extractor is used to extract the depth features of the target. The extracted depth features of the target are matched with the depth features of the initial target to determine whether the target is the initial target.

5. The robot tracking method according to claim 4, characterized in that, If the initial target is not relocked within the preset time period, the depth features of the initial target are obtained, the initial target is located based on the depth features, and the initial target is relocked and tracked; if the initial target is relocked within the preset time period, the target tracker is updated, and the updated target tracker is used to control the robot to track the relocked target; or if the tracking probability is not less than the second preset value, it is determined that the initial target has not been lost. The method further includes: Obtain the depth and scale information of the current target, and determine whether the current target is occluded based on the depth and scale information of the current target; If the current target is unobstructed, the image of the current target is added to the first sample set to train the model and obtain the first feature extractor; If the current target is occluded, the image of the current target is added to the second sample set to train the model and obtain the second feature extractor.

6. A control terminal, characterized in that, It includes a processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to perform the robot tracking method as described in any one of claims 1 to 5.

7. A robot, characterized in that, include: The control terminal and robot body as described in claim 6; The control terminal is used to control the robot body to track the target.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the robot tracking method as described in any one of claims 1 to 5.

Images