Tracking method, device, equipment and storage medium

Abstract

The present application provides a tracking method, apparatus, device and storage medium, wherein the method comprises: performing object recognition on a current frame of an image collected by a tracking device, and generating corresponding recognition frames for the recognized objects respectively, wherein the objects are located in the recognition frames, and determining a target recognition frame from each recognition frame according to image features corresponding to each recognition frame and image features corresponding to at least one historical recognition frame, wherein the historical recognition frame is a recognition frame of the target object to be tracked in the historical image collected by the tracking device, and determining the motion control parameters of the tracking device according to the attribute information of the target recognition frame and the attribute information of the reference recognition frame, wherein the reference recognition frame is obtained based on the historical recognition frame, and the tracking device is controlled to move according to the motion control parameters to track the target object. The tracking cost is reduced, and the real-time performance and tracking accuracy of the tracking are improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a tracking method, apparatus, device and storage medium. Background Art

[0002] Autonomous vehicles, also known as driverless cars, tracking devices, computer-driven cars, and wheeled mobile robots, are intelligent cars that achieve unmanned driving through computer systems.

[0003] Currently, in visual tracking scenarios, tracking devices usually use ultra-wideband (UWB) to calculate the target's position, or use real-time kinematic (RTK) or global positioning system (GPS) point tracking solutions to achieve visual tracking.

[0004] However, the above visual tracking solutions have high hardware costs and poor tracking accuracy. Summary of the Invention

[0005] The purpose of this application is to provide a tracking method, device, equipment and storage medium to address the deficiencies in the above-mentioned prior art, so as to solve the problems of high hardware cost and poor tracking accuracy in the prior art.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of the present application are as follows:

[0007] In a first aspect, an embodiment of the present application provides a tracking method, including:

[0008] Performing object recognition on a current frame of image captured by the tracking device, and generating corresponding identification frames for the identified objects, wherein the objects are located within the identification frames;

[0009] Determining a target recognition frame from each recognition frame based on image features corresponding to each recognition frame and image features corresponding to at least one historical recognition frame, wherein the historical recognition frame is a recognition frame of the target object to be tracked in historical images captured by the tracking device;

[0010] determining motion control parameters of the tracking device according to attribute information of the target recognition frame and attribute information of a reference recognition frame, wherein the reference recognition frame is obtained based on each of the historical recognition frames;

[0011] The tracking device is controlled to move according to the motion control parameters to track the target object.

[0012] Optionally, determining the target recognition frame from each recognition frame according to the image features corresponding to each recognition frame and the image features corresponding to at least one historical recognition frame includes:

[0013] performing a feature comparison between the recognition frame and each of the historical recognition frames based on image features corresponding to the recognition frame and image features corresponding to each of the historical recognition frames to obtain a comparison result, wherein the comparison result is used to indicate whether the recognition frame and each of the historical recognition frames belong to the same target object;

[0014] If the comparison result indicates that the recognition frame and each of the historical recognition frames belong to the same target object, the recognition frame is used as the target recognition frame.

[0015] Optionally, if the comparison result indicates that the recognition frame and each of the historical recognition frames belong to the same target object, using the recognition frame as the target recognition frame includes:

[0016] If the comparison result indicates that the recognition frame and each of the historical recognition frames belong to the same target object, and the sum of the number of the recognition frame and each of the historical recognition frames is greater than or equal to a preset number threshold, the recognition frame is used as the target recognition frame.

[0017] Optionally, performing feature comparison on the recognition frame and each of the historical recognition frames based on the image features corresponding to the recognition frame and the image features corresponding to each of the historical recognition frames to obtain a comparison result includes:

[0018] Determining similarities between the image features corresponding to the recognition frame and the image features corresponding to each of the historical recognition frames;

[0019] The comparison result is obtained according to each similarity.

[0020] Optionally, obtaining the comparison result according to each similarity includes:

[0021] If each of the similarities exceeds a preset similarity threshold, and the deviation of each similarity does not exceed a preset deviation, it is determined that the recognition frame and each of the historical recognition frames belong to the same target object.

[0022] Optionally, after performing feature comparison between the recognition frame and each of the historical recognition frames based on the image features corresponding to the recognition frame and the image features corresponding to each of the historical recognition frames and obtaining the comparison result, the method further includes:

[0023] If the comparison result indicates that the recognition frame and each of the historical recognition frames belong to different objects, then the operating parameters of the tracking device are adjusted, where the operating parameters include: a shooting angle.

[0024] Optionally, after performing feature comparison between the recognition frame and each of the historical recognition frames based on the image features corresponding to the recognition frame and the image features corresponding to each of the historical recognition frames and obtaining the comparison result, the method further includes:

[0025] If the comparison result indicates that the identification frame and each of the historical identification frames belong to different objects, adjusting the value of the tracking loss flag;

[0026] If the value of the tracking loss flag is greater than a preset threshold, the operating parameters of the tracking device are adjusted, and the operating parameters include: a shooting angle.

[0027] Optionally, before performing feature comparison between the recognition frame and each of the historical recognition frames based on the image features corresponding to the recognition frame and the image features corresponding to each of the historical recognition frames, the method further includes:

[0028] Filtering selectable identification frames from the identification frames according to attribute information of the identification frames, wherein the attribute information includes at least one of length, width, and area;

[0029] The performing feature comparison on the recognition frame and each of the historical recognition frames based on the image features corresponding to the recognition frame and the image features corresponding to each of the historical recognition frames to obtain a comparison result includes:

[0030] Based on the image features corresponding to the optional identification frame and the image features corresponding to each of the historical identification frames, a feature comparison is performed on the optional identification frame and each of the historical identification frames to obtain the comparison result, which is used to indicate whether the optional identification frame and each of the historical identification frames belong to the same target object.

[0031] Optionally, the filtering out selectable identification boxes from the identification boxes according to the attribute information of the identification boxes includes:

[0032] The identification frame with the largest area is selected from the identification frames as the optional identification frame.

[0033] Optionally, the filtering out selectable identification boxes from the identification boxes according to the attribute information of the identification boxes includes:

[0034] Determine the identification frame with the largest area among the identification frames;

[0035] If the area of ​​the identification frame with the largest area is greater than or equal to a preset area threshold, the identification frame with the largest area is determined to be the selectable identification frame.

[0036] Optionally, the filtering out selectable identification boxes from the identification boxes according to the attribute information of the identification boxes includes:

[0037] If the number of the identification frames in the current frame image is within a preset number range, selectable identification frames are selected from the identification frames according to attribute information of the identification frames.

[0038] Optionally, determining the motion control parameters of the tracking device according to the attribute information of the target recognition frame and the attribute information of the reference recognition frame includes:

[0039] determining a size difference between the reference recognition frame and the target recognition frame according to the size of the target recognition frame and the size of the reference recognition frame;

[0040] The motion control parameter is determined according to the size difference.

[0041] Optionally, the attribute information further includes: location;

[0042] The determining the motion control parameter according to the size difference includes:

[0043] The motion control parameter is determined according to the size difference, the position of the target recognition frame, and the position of the reference recognition frame.

[0044] Optionally, determining the motion control parameter according to the size difference, the position of the target recognition frame, and the position of the reference recognition frame includes:

[0045] determining a position difference between the reference recognition frame and the target recognition frame according to the position of the target recognition frame and the position of the reference recognition frame;

[0046] Determining an angular velocity corresponding to the position difference from a preset angular velocity control curve, wherein the angular velocity control curve is determined based on a preset maximum angular velocity, a preset maximum position difference, and a preset minimum position difference;

[0047] Determining a linear speed corresponding to the size difference from a preset linear speed control curve, wherein the linear speed control curve is determined based on a preset maximum linear speed, a preset maximum size difference, and a preset minimum size difference;

[0048] The linear velocity and the angular velocity are used as the motion control parameters.

[0049] Optionally, the center point position of the reference identification frame is the center point position of the current frame image.

[0050] Optionally, performing object recognition on a current frame of image captured by the tracking device includes:

[0051] The object of the current frame image is recognized according to the recognition network obtained by pre-training, and corresponding recognition frames are generated for the recognized objects, wherein the recognition network is trained based on multiple image samples, and the multiple image samples are respectively marked with sample recognition frames corresponding to each sample object.

[0052] Optionally, before determining the motion control parameters of the tracking device based on the attribute information of the target recognition frame and the attribute information of the reference recognition frame, the method further includes:

[0053] Fitting is performed on each of the historical recognition frames to obtain the reference recognition frame.

[0054] In a second aspect, another embodiment of the present application provides a tracking device, comprising:

[0055] a recognition module, configured to perform object recognition on a current frame of image captured by the tracking device, and generate corresponding recognition frames for the recognized objects, wherein the objects are located within the recognition frames;

[0056] a determination module configured to determine a target recognition frame from each recognition frame based on image features corresponding to each recognition frame and image features corresponding to at least one historical recognition frame, wherein the historical recognition frame is a recognition frame of the target object to be tracked in historical images captured by the tracking device;

[0057] The determination module is further configured to determine motion control parameters of the tracking device based on attribute information of the target recognition frame and attribute information of a reference recognition frame, wherein the reference recognition frame is obtained based on each of the historical recognition frames;

[0058] A control module is configured to control the movement of the tracking device according to the motion control parameters to track the target object.

[0059] Optionally, the determining module is specifically configured to:

[0060] performing a feature comparison between the recognition frame and each of the historical recognition frames based on image features corresponding to the recognition frame and image features corresponding to each of the historical recognition frames to obtain a comparison result, wherein the comparison result is used to indicate whether the recognition frame and each of the historical recognition frames belong to the same target object;

[0061] If the comparison result indicates that the recognition frame and each of the historical recognition frames belong to the same target object, the recognition frame is used as the target recognition frame.

[0062] Optionally, the determining module is specifically configured to:

[0063] If the comparison result indicates that the recognition frame and each of the historical recognition frames belong to the same target object, and the sum of the number of the recognition frame and each of the historical recognition frames is greater than or equal to a preset number threshold, the recognition frame is used as the target recognition frame.

[0064] Optionally, the determining module is specifically configured to:

[0065] Determining similarities between the image features corresponding to the recognition frame and the image features corresponding to each of the historical recognition frames;

[0066] The comparison result is obtained according to each similarity.

[0067] Optionally, the determining module is specifically configured to:

[0068] If each of the similarities exceeds a preset similarity threshold, and the deviation of each similarity does not exceed a preset deviation, it is determined that the recognition frame and each of the historical recognition frames belong to the same target object.

[0069] Optionally, it also includes:

[0070] The adjustment module is configured to adjust operating parameters of the tracking device if the comparison result indicates that the recognition frame and each of the historical recognition frames belong to different objects, the operating parameters including a shooting angle.

[0071] Optionally, the adjustment module is further configured to:

[0072] If the comparison result indicates that the identification frame and each of the historical identification frames belong to different objects, adjusting the value of the tracking loss flag;

[0073] If the value of the tracking loss flag is greater than a preset threshold, the operating parameters of the tracking device are adjusted, and the operating parameters include: a shooting angle.

[0074] Optionally, the determining module is further configured to:

[0075] Filtering selectable identification frames from the identification frames according to attribute information of the identification frames, wherein the attribute information includes at least one of length, width, and area;

[0076] The determining module is specifically configured to:

[0077] Based on the image features corresponding to the optional identification frame and the image features corresponding to each of the historical identification frames, a feature comparison is performed on the optional identification frame and each of the historical identification frames to obtain the comparison result, which is used to indicate whether the optional identification frame and each of the historical identification frames belong to the same target object.

[0078] Optionally, the determining module is specifically configured to:

[0079] The identification frame with the largest area is selected from the identification frames as the optional identification frame.

[0080] Optionally, the determining module is specifically configured to:

[0081] Determine the identification frame with the largest area among the identification frames;

[0082] If the area of ​​the identification frame with the largest area is greater than or equal to a preset area threshold, the identification frame with the largest area is determined to be the selectable identification frame.

[0083] Optionally, the determining module is specifically configured to:

[0084] If the number of the identification frames in the current frame image is within a preset number range, selectable identification frames are selected from the identification frames according to attribute information of the identification frames.

[0085] Optionally, the determining module is specifically configured to:

[0086] determining a size difference between the reference recognition frame and the target recognition frame according to the size of the target recognition frame and the size of the reference recognition frame;

[0087] The motion control parameter is determined according to the size difference.

[0088] Optionally, the attribute information further includes: a location; and the determining module is specifically configured to:

[0089] The motion control parameter is determined according to the size difference, the position of the target recognition frame, and the position of the reference recognition frame.

[0090] Optionally, the determining module is specifically configured to:

[0091] determining a position difference between the reference recognition frame and the target recognition frame according to the position of the target recognition frame and the position of the reference recognition frame;

[0092] Determining an angular velocity corresponding to the position difference from a preset angular velocity control curve, wherein the angular velocity control curve is determined based on a preset maximum angular velocity, a preset maximum position difference, and a preset minimum position difference;

[0093] Determining a linear speed corresponding to the size difference from a preset linear speed control curve, wherein the linear speed control curve is determined based on a preset maximum linear speed, a preset maximum size difference, and a preset minimum size difference;

[0094] The linear velocity and the angular velocity are used as the motion control parameters.

[0095] Optionally, the center point position of the reference identification frame is the center point position of the current frame image.

[0096] Optionally, the identification module is specifically configured to:

[0097] The object of the current frame image is recognized according to the recognition network obtained by pre-training, and corresponding recognition frames are generated for the recognized objects, wherein the recognition network is trained based on multiple image samples, and the multiple image samples are respectively marked with sample recognition frames corresponding to each sample object.

[0098] Optionally, it also includes:

[0099] The processing module is used to perform fitting processing on each of the historical recognition frames to obtain the reference recognition frame.

[0100] In the third aspect, another embodiment of the present application provides a tracking device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the tracking device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to execute any one of the methods described in the first aspect.

[0101] In a fourth aspect, another embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in any one of the first aspects is executed.

[0102] The beneficial effects of this application are:

[0103] The present application provides a tracking method, apparatus, device, and storage medium, wherein the method includes: performing object recognition on a current frame of an image captured by a tracking device, and generating corresponding recognition frames for the recognized objects, wherein the objects are located within the recognition frames; determining a target recognition frame from each of the recognition frames based on image features corresponding to each recognition frame and image features corresponding to at least one historical recognition frame; the historical recognition frame is a recognition frame of the target object to be tracked in a historical image captured by the tracking device; determining motion control parameters of the tracking device based on attribute information of the target recognition frame and attribute information of the reference recognition frame; wherein the reference recognition frame is obtained based on the historical recognition frame; and controlling the movement of the tracking device based on the motion control parameters to track the target object. This reduces the cost of tracking and improves the real-time performance and accuracy of tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0105] Figure 1 The following is a schematic diagram of the tracking method provided in the embodiment of the present application. Figure 1 ;

[0106] Figure 2 The following is a schematic diagram of the tracking method provided in the embodiment of the present application. Figure 2 ;

[0107] Figure 3 The following is a schematic diagram of the tracking method provided in the embodiment of the present application. Figure 3 ;

[0108] Figure 4 The schematic diagram of the current frame image provided by the embodiment of the present application is shown Figure 1 ;

[0109] Figure 5 The following is a schematic diagram of the tracking method provided in the embodiment of the present application. Figure 4 ;

[0110] Figure 6 The following is a schematic diagram of the tracking method provided in the embodiment of the present application. Figure 5 ;

[0111] Figure 7 The following is a schematic diagram of the tracking method provided in the embodiment of the present application. Figure 6 ;

[0112] Figure 8 The schematic diagram of the current frame image provided by the embodiment of the present application is shown Figure 2 ;

[0113] Figure 9 A schematic diagram of a line speed control curve provided in an embodiment of the present application is shown;

[0114] Figure 10 A schematic diagram of an angular velocity control curve provided in an embodiment of the present application is shown;

[0115] Figure 11 A schematic structural diagram of a tracking device provided in an embodiment of the present application is shown;

[0116] Figure 12 A schematic structural diagram of a tracking device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0117] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0118] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0119] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0120] Existing technologies offer two visual tracking solutions. One uses ultra-wideband (UWB) to calculate the target's position, but this solution has extremely high hardware costs, requires at least four UWB base stations, and suffers from large errors. The other uses real-time kinematic (RTK) or global positioning system (GPS) tracking. However, RTK equipment is expensive and has limited accuracy, especially when the positioning signal is blocked. GPS also has poor accuracy. Both tracking solutions have a 5Hz frequency, so their real-time performance is poor. Furthermore, poor accuracy can cause the tracking device to collide with the object.

[0121] Based on this, the present application provides a tracking method, which performs object recognition on the current frame image captured by the tracking device, generates corresponding recognition frames for the recognized objects, and then determines the target recognition frame from each recognition frame based on the image features corresponding to each recognition frame and the image features corresponding to at least one historical recognition frame. Then, based on the attribute information of the target recognition frame and the attribute information of the reference recognition frame, the motion control parameters of the tracking device are determined, and the movement of the tracking device is controlled according to the motion control parameters to track the target object. No additional hardware equipment is required, which reduces the cost of tracking and improves the real-time performance and tracking accuracy of tracking. A low-cost, highly robust monocular visual tracking is achieved.

[0122] The tracking device in this application can be an unmanned vehicle, drone, or other device. In some scenarios, the tracking device needs to track an object, such as a person or an animal. For example, in an agricultural transport scenario, the tracking device is loaded with transported objects. By tracking the person, the tracking device transports the transported objects to the location the person passed, thereby reducing the cost of manual transportation. Alternatively, in a route reconstruction scenario, the tracking device tracks the person to record the person's walking trajectory so that the route can be reconstructed later based on the recorded trajectory.

[0123] Taking the agricultural transportation scenario as an example, in the tracking preparation stage, the tracking device can prompt the target object to be tracked (such as the person to be tracked) to stand at a preset distance from the device (such as 1 meter from the tracking device). The tracking device is provided with a shooting camera to capture at least one frame of the target object image (historical image), and then identify the target object in each historical image and generate a corresponding historical identification frame. The tracking device performs fitting processing based on at least one historical identification frame to obtain a benchmark identification frame. After obtaining the benchmark identification frame, the tracking scheme provided by this application can be executed, and the motion control parameters of the tracking device can be dynamically adjusted based on the benchmark identification frame to avoid the tracking device being too close or too far from the target object, so as to improve the tracking accuracy.

[0124] Among them, the preset distance can be the optimal distance for the tracking device to track the target object. If it is greater than the preset distance, the target object may be lost. If it is less than the preset distance, the target object may be hit. The preset distance in this embodiment can be selected based on experience and is not particularly limited in this embodiment.

[0125] The technical solution of the tracking method of the present application is described in detail below with reference to several specific embodiments.

[0126] Figure 1 The following is a schematic diagram of the tracking method provided in the embodiment of the present application. Figure 1 , the execution subject of this embodiment can be a tracking device, such as an unmanned vehicle, a drone and other devices. Figure 1 As shown, the method includes:

[0127] S101: Perform object recognition on a current frame of image captured by a tracking device, and generate corresponding recognition frames for the recognized objects.

[0128] The tracking device may be provided with a shooting camera, which may capture a current frame of image, and then perform object recognition on the captured current frame of image, and generate corresponding identification frames for the identified objects, wherein the objects are located within the identification frames. The current frame of image may include but is not limited to one object, and each object corresponds to an identification frame, wherein the objects may be, for example, people, animals, buildings, roads, etc.

[0129] S102 : Determine a target recognition frame from each recognition frame according to image features corresponding to each recognition frame and image features corresponding to at least one historical recognition frame.

[0130] The historical recognition frame is the recognition frame of the target object to be tracked in the historical images collected by the tracking device, and the target recognition frame is the recognition frame of the target object in the current frame image. That is, the historical recognition frame and the target recognition frame are both recognition frames corresponding to the target object, wherein the historical image can be at least one frame of historical image collected by the tracking device before the current frame image.

[0131] The image features corresponding to each recognition frame may be image features of the image data in each recognition frame, and the image features corresponding to the historical recognition frame may be image features of the image data in the historical recognition frame.

[0132] In some embodiments, a feature extraction algorithm can be used to extract image features corresponding to each recognition frame and image features corresponding to at least one historical recognition frame, and then a target recognition frame can be determined from each recognition frame based on the image features corresponding to each recognition frame and the image features corresponding to at least one historical recognition frame. The target recognition frame can be determined from each recognition frame by comparing the image features corresponding to each recognition frame and the image features corresponding to at least one historical recognition frame, wherein the target recognition frame and the historical recognition frames belong to the same target object, that is, the similarity between the image features corresponding to the target recognition frame and the image features corresponding to the historical recognition frames can exceed a certain preset threshold.

[0133] S103: Determine motion control parameters of the tracking device according to the attribute information of the target recognition frame and the attribute information of the reference recognition frame.

[0134] S104: Control the movement of the tracking device according to the motion control parameters to track the target object.

[0135] The reference recognition frame is derived from each historical recognition frame. The difference between the target recognition frame and the reference recognition frame can be determined based on the attribute information of the target recognition frame and the attribute information of the reference recognition frame. This difference can then be used to determine the motion control parameters of the tracking device. The tracking device is then controlled based on the motion control parameters to track the target object. The attribute information can include size and / or area. The recognition frame can be a rectangular recognition frame, and the size of the recognition frame includes length and width.

[0136] Since the distance between the tracking device and the target object changes dynamically during the tracking process, generally the longer the distance, the smaller the size and area of ​​the generated target recognition frame, and the closer the distance, the larger the size and area of ​​the generated target recognition frame. Therefore, during the tracking process, the motion control parameters of the tracking device are dynamically adjusted according to the size difference or area difference of the target recognition frame relative to the reference recognition frame, so that the distance between the tracking device and the target object during the tracking process is as close as possible to the distance between the tracking device and the target object corresponding to the reference recognition frame, avoiding the tracking device being too close to the target object (possibly colliding) or too far away (possibly losing it), thereby improving the tracking accuracy.

[0137] The tracking method of this embodiment performs object recognition on the current frame of an image captured by the tracking device and generates corresponding identification frames for each identified object. The object is located within the identification frame. Based on the image features corresponding to each identification frame and the image features corresponding to at least one historical identification frame, a target identification frame is determined from each of the identification frames. The historical identification frame is the identification frame of the target object to be tracked in historical images captured by the tracking device. The motion control parameters of the tracking device are determined based on the attribute information of the target identification frame and the attribute information of the reference identification frame. The reference identification frame is obtained based on the historical identification frame, and the tracking device is controlled to move according to the motion control parameters to track the target object. This reduces the tracking cost and improves the real-time tracking performance and tracking accuracy.

[0138] Optionally, before step S103, based on the attribute information of the target recognition frame and the attribute information of the reference recognition frame, the method further includes:

[0139] Perform fitting processing on each historical recognition frame to obtain the benchmark recognition frame.

[0140] The historical recognition frame is the recognition frame of the target object to be tracked in the historical images collected by the tracking device. Each historical recognition frame is fitted to obtain the reference recognition frame. Both the reference recognition frame and the target recognition frame can be displayed in the current frame image.

[0141] Optionally, the center point position of the reference recognition frame is the center point position of the current frame image.

[0142] The following describes the process of fitting and obtaining the reference recognition frame with an example.

[0143] Assume that there are i historical recognition frames, i is greater than 1, and the coordinates of the center point of each historical recognition frame in the pixel coordinate system are (x n ,y n ), n is an integer in (1, i], assuming the center point of the reference recognition box is (x, y), then the minimized objective function is:

[0144]

[0145] In order to optimize the objective function, we calculate the partial derivatives of x and y respectively, and get:

[0146]

[0147] Therefore, the center point (x, y) of the reference recognition frame can be obtained, and the distance between the center point and the center point of each historical recognition frame is the shortest.

[0148] If the length and width of each history recognition frame are (w n ,h n ), then the length and width of the reference recognition frame are (w,h), and the formula is:

[0149]

[0150]

[0151] Optionally, step S101, performing object recognition on a current frame of image captured by the tracking device, includes:

[0152] The object recognition of the current frame image is performed based on the pre-trained recognition network, and corresponding recognition frames are generated for the recognized objects.

[0153] The recognition network is trained using multiple image samples, each labeled with a corresponding sample recognition box. This network is an improvement on the Yolox network, replacing upsampling with deconvolution to improve recognition accuracy.

[0154] Figure 2 The following is a schematic diagram of the tracking method provided in the embodiment of the present application. Figure 2 ,like Figure 2 As shown, step S102, based on the image features corresponding to each recognition frame and the image features corresponding to at least one historical recognition frame, determines a target recognition frame from each recognition frame, including:

[0155] S201 : performing feature comparison between the optional recognition frame and each historical recognition frame based on image features corresponding to the recognition frame and image features corresponding to each historical recognition frame to obtain a comparison result.

[0156] The image features corresponding to each recognition frame and the image features corresponding to each historical recognition frame are extracted, and then feature comparison is performed to obtain a comparison result, wherein the comparison result is used to indicate whether the recognition frame undergoing feature comparison and each historical recognition frame belong to the same target object.

[0157] Optionally, the similarities between the image features corresponding to the recognition frame and the image features corresponding to each historical recognition frame can be determined, and a comparison result can be obtained based on the similarities. If each similarity exceeds a certain preset similarity threshold, or a preset ratio of similarities (such as 90% similarity among the similarities) exceeds a certain preset similarity threshold, it indicates that the recognition frame and the historical recognition frame belong to the same target object; otherwise, it indicates that the recognition frame and the historical recognition frame belong to different objects. The similarities between the image features corresponding to the recognition frame and the image features corresponding to each historical recognition frame can be calculated using the cosine distance.

[0158] Optionally, a comparison result is obtained based on each similarity, including:

[0159] If each similarity exceeds a preset similarity threshold and the deviation of each similarity does not exceed a preset deviation, it is determined that the recognition frame and each historical recognition frame belong to the same target object.

[0160] Determine whether each similarity exceeds a preset similarity threshold and whether the deviation between each similarity exceeds a preset deviation. If each similarity exceeds the preset similarity threshold and the deviation of each similarity (the deviation between each similarity) does not exceed the preset deviation, then determine that the recognition frame and each historical recognition frame belong to the same target object, that is, if the recognition frame also belongs to the target object, each similarity between the recognition frame and each historical recognition frame exceeds the preset similarity threshold, and the deviation of each similarity does not exceed the preset deviation.

[0161] If there is a similarity among the similarities that does not exceed the preset similarity threshold, and there is a similarity among the similarities that has a deviation that exceeds the preset deviation, then the recognition frame and the historical recognition frame belong to different objects.

[0162] It should be noted that a feature extraction network can be used to extract image data in the recognition frame to obtain image features corresponding to the recognition frame, and a feature extraction network can be used to extract image data in each historical recognition frame to obtain image features corresponding to each historical recognition frame. Among them, the feature extraction network can be a person re-identification (ReID) network, which can be trained based on each sample recognition frame, and each sample recognition frame is pre-labeled with the corresponding sample image features.

[0163] In addition, when the embedded neural network processor (NPU) of the tracking device does not support the fully connected layer Dense10 in the ReID network, the fully connected layer Dense10 can also be modified to an average pooling layer (avgpool).

[0164] S202: If the comparison result indicates that the recognition frame and each historical recognition frame belong to the same target object, then the recognition frame is used as the target recognition frame.

[0165] If the comparison result indicates that the recognition frame and each historical recognition frame belong to the same target object, the recognition frame is used as the target recognition frame, that is, the target object corresponding to the target recognition frame is tracked in the subsequent tracking process.

[0166] Optionally, in step S202, if the comparison result indicates that the recognition frame and each historical recognition frame belong to the same target object, then the recognition frame is used as the target recognition frame, including:

[0167] If the comparison result indicates that the recognition frame and each historical recognition frame belong to the same target object, and the sum of the number of the recognition frame and each historical recognition frame is greater than or equal to a preset number threshold, the recognition frame is used as the target recognition frame.

[0168] Since each historical frame is the target object's frame, the comparison results confirm that the frame and each historical frame belong to the same target object. Taking into account the number of frames, if the sum of the number of frames is greater than or equal to a preset number threshold, the frame is considered the target frame. This ensures that the target object is stably tracked and avoids losing the target object if it is unstable.

[0169] The tracking method of this embodiment compares the image features of the recognition frame with those of each historical recognition frame based on the image features corresponding to the recognition frame. If the comparison result indicates that the recognition frame and each historical recognition frame belong to the same target object, the recognition frame is used as the target recognition frame. This achieves single-target tracking, reduces tracking costs, ensures the accuracy of the target recognition frame, and improves tracking precision.

[0170] Figure 3 The following is a schematic diagram of the tracking method provided in the embodiment of the present application. Figure 3 ,like Figure 3 As shown, in step S201, based on the image features corresponding to the recognition frame and the image features corresponding to each historical recognition frame, feature comparison is performed on the optional recognition frame and each historical recognition frame. Before obtaining the comparison result, the method further includes:

[0171] S301 : Filtering selectable identification boxes from the identification boxes according to the attribute information of the identification boxes.

[0172] The attribute information includes at least one of length, width, and area. Based on the attribute information of each identification frame, an identification frame whose area satisfies a preset area condition is determined as a selectable identification frame. The preset area condition may include: an area greater than a preset area threshold or a maximum area. Alternatively, an identification frame that satisfies a preset length condition is determined as a selectable identification frame. The preset length condition may include: a length greater than a preset length or a maximum length. Alternatively, an identification frame that satisfies a preset width condition is determined as a selectable identification frame. The preset width condition may include: a width greater than a preset width or a maximum width.

[0173] Since the larger the area, length, and width of the identification box, the closer the object corresponding to the identification box is to the tracking device, and the smaller the area, length, and width of the identification box, the farther the object corresponding to the identification box is from the tracking device, to avoid losing the object due to being too far away, the identification box whose area meets the preset area condition, or meets the preset length condition, or meets the preset width condition can be selected as an optional identification box.

[0174] Optionally, step S301, screening selectable identification boxes from the identification boxes according to attribute information of the identification boxes, includes:

[0175] The recognition frame with the largest area is selected from each recognition frame as the optional recognition frame.

[0176] In some embodiments, the areas of the recognition frames in the current frame image may be sorted from large to small, and then the recognition frame ranked first may be selected as the selectable recognition frame, that is, the recognition frame with the largest area among the recognition frames may be selected as the selectable recognition frame.

[0177] Figure 4 The schematic diagram of the current frame image provided by the embodiment of the present application is shown Figure 1 ,like Figure 4 As shown, the current frame image includes four recognition frames (recognition frame 1, recognition frame 2, recognition frame 3, and recognition frame 4). The area of ​​each recognition frame is determined, and the areas of each recognition frame are sorted from large to small. The recognition frame with the largest area (recognition frame 1) is selected as the optional recognition frame.

[0178] Optionally, step S301, screening selectable identification boxes from the identification boxes according to attribute information of the identification boxes, includes:

[0179] Determine the recognition box with the largest area among all recognition boxes;

[0180] If the area of ​​the largest recognition frame is greater than or equal to the preset area threshold, the recognition frame with the largest area is determined to be a selectable recognition frame.

[0181] In some embodiments, the area of ​​each identification frame is calculated, and the identification frame with the largest area among the identification frames is determined, and it is determined whether the area of ​​the identification frame with the largest area is greater than or equal to a preset area threshold. If it is greater than or equal to the preset area threshold, the identification frame with the largest area is used as an optional identification frame. That is, the optional identification frame satisfies the requirements of having the largest area among all the identification frames, and the area of ​​the optional identification frame is greater than or equal to the preset area threshold. Since the larger the area of ​​the identification frame, the closer the object corresponding to the identification frame is to the tracking device, and the smaller the area of ​​the identification frame, the farther the object corresponding to the identification frame is from the tracking device, this method can avoid the situation of being lost due to being too far away.

[0182] It should be noted that the preset area threshold can be selected based on experience or actual conditions, and this embodiment does not impose any special limitation on this.

[0183] Optionally, step S301, screening selectable identification boxes from the identification boxes according to attribute information of the identification boxes, includes:

[0184] If the number of the recognition frames in the current frame image is within a preset number range, selectable recognition frames are selected from the recognition frames according to the attribute information of the recognition frames.

[0185] Considering the performance of the tracking device, the step of selecting optional identification frames from each identification frame is performed only when the number of each identification frame in the current frame image is within a preset number range. If the number of each identification frame in the current frame image is outside the preset number range, such as greater than the upper limit of the preset number range, for example, the number of each identification frame is 25 and the upper limit is 20, this step may not be performed due to the limitation of the computing resources of the board of the tracking device. If the number is less than the lower limit of the preset number range, for example, the number of each identification frame is 1 and the lower limit is 2, there is no need to execute this solution, and the object corresponding to the identification frame can be directly used as the tracking target.

[0186] It can be seen from this that when determining the target recognition frame from each recognition frame, the number of each recognition frame in the current frame image and the area of ​​each recognition frame can be comprehensively considered. When the number of each recognition frame in the current frame image is within a preset number range, the recognition frame with the largest area among the recognition frames is determined as the optional recognition frame. Alternatively, when the number of each recognition frame in the current frame image is within a preset number range, the recognition frame with the largest area among the recognition frames is determined, and when the area of ​​the recognition frame with the largest area is greater than or equal to a preset area threshold, the recognition frame with the largest area is determined as the optional recognition frame.

[0187] Accordingly, in step S201, based on the image features corresponding to the recognition frame and the image features corresponding to each historical recognition frame, feature comparison is performed on the optional recognition frame and each historical recognition frame to obtain a comparison result, including:

[0188] S302 : performing feature comparison between the optional recognition frame and each historical recognition frame based on the image features corresponding to the optional recognition frame and the image features corresponding to each historical recognition frame to obtain a comparison result.

[0189] The image features corresponding to the optional recognition frame and the image features corresponding to each historical recognition frame are extracted, and then feature comparison is performed to obtain a comparison result. The comparison result is used to indicate whether the optional recognition frame and each historical recognition frame belong to the same target object.

[0190] Optionally, the similarities between the image features corresponding to the optional identification frame and the image features corresponding to each historical identification frame can be determined, and a comparison result can be obtained based on the similarities. For example, if each similarity exceeds a preset similarity threshold, and the deviation of each similarity does not exceed a preset deviation, then it is determined that the optional identification frame and each historical identification frame belong to the same target object. If there is a similarity among the similarities that does not exceed the preset similarity threshold, and there is a similarity among the similarities that has a deviation that exceeds the preset deviation, then the optional identification frame and the historical identification frame belong to different objects.

[0191] Accordingly, in step S202, if the comparison result indicates that the recognition frame and each historical recognition frame belong to the same target object, the recognition frame is used as the target recognition frame, including:

[0192] S303: If the comparison result indicates that the selectable recognition frame and each historical recognition frame belong to the same target object, the selectable recognition frame is used as the target recognition frame.

[0193] If the comparison result indicates that the optional identification box and each historical identification box belong to the same target object, then the optional identification box is used as the target identification box. In some embodiments, if the comparison result indicates that the optional identification box and each historical identification box belong to the same target object, and the sum of the number of optional identification boxes and each historical identification box is greater than or equal to a preset number threshold, then the optional identification box is used as the target identification box.

[0194] Since each historical frame is the target object's frame, the comparison results confirm that the selectable frame and each historical frame belong to the same target object. Taking into account the number of frames, if the sum of the selectable frame and each historical frame is greater than or equal to a preset number threshold, the selectable frame is selected as the target frame. This ensures that the target object is stably tracked and avoids losing the target object if it is unstable.

[0195] Figure 5 The following is a schematic diagram of the tracking method provided in the embodiment of the present application. Figure 4 ,like Figure 5As shown, in step S201, based on the image features corresponding to the recognition frame and the image features corresponding to each historical recognition frame, feature comparison is performed on the recognition frame and each historical recognition frame. After obtaining the comparison result, the method further includes step S401 or steps S402-S403.

[0196] S401: If the comparison result indicates that the recognition frame and each historical recognition frame belong to a different object, adjust the working parameters of the tracking device.

[0197] If the comparison result indicates that the recognition frame and each historical recognition frame belong to different objects, that is, the object corresponding to the recognition frame is not the target object, the working parameters of the tracking device can be adjusted, and the working parameters include: shooting angle.

[0198] That is to say, if the target object is temporarily lost, the tracking device can swing left and right to adjust the shooting angle, skip the current frame, capture the next frame, identify the object in the next frame, and generate the corresponding identification frame for the object, and repeat the process. Figure 1 In this embodiment, the target object is searched again until the determined recognition frame and each historical recognition frame belong to the target object.

[0199] If the target object is not found again within a preset time after the current frame image, the target object is determined to be permanently lost. Alternatively, if the target object is not found again after a preset number of frame images are collected after the current frame image, the target object is determined to be permanently lost.

[0200] S402: If the comparison result indicates that the recognition frame and each historical recognition frame belong to a different object, adjust the value of the tracking loss flag.

[0201] S403: If the value of the tracking loss flag is greater than a preset threshold, adjust the working parameters of the tracking device.

[0202] If the comparison result indicates that the recognition frame and each historical recognition frame belong to different objects, it means that the target object is temporarily lost, skip the current frame image, and increase the value of the tracking loss flag by 1. The tracking device collects the next frame image and repeats the process. Figure 1 In an embodiment, if the value of the tracking loss flag is greater than a preset threshold, the operating parameters of the tracking device are adjusted, the operating parameters including the shooting angle, so that the tracking device re-searches for the target object according to the adjusted operating parameters.

[0203] In some embodiments, in step S302, based on the image features corresponding to the selectable identification box and the image features corresponding to the historical identification boxes, feature comparison is performed on the selectable identification box and the historical identification boxes. After obtaining the comparison results, the method may further include:

[0204] If the comparison result indicates that the selectable identification box and each historical identification box belong to a different object, the operating parameters of the tracking device are adjusted, or the value of the tracking loss flag is adjusted. If the value of the tracking loss flag is greater than a preset threshold, the operating parameters of the tracking device are adjusted. The specific implementation process is similar to steps S401-S403 and will not be repeated here.

[0205] In the tracking method of this embodiment, if the comparison result indicates that the identification frame and each historical identification frame belong to a different object, the operating parameters of the tracking device are adjusted. If the comparison result indicates that the identification frame and each historical identification frame belong to a different object, the value of the tracking loss flag is adjusted. If the value of the tracking loss flag is greater than a preset threshold, the operating parameters of the tracking device are adjusted. If the target object is temporarily lost, the operating parameters of the tracking device can be adjusted to facilitate a re-search for the target object.

[0206] Figure 6 The following is a schematic diagram of the tracking method provided in the embodiment of the present application. Figure 5 ,like Figure 6 As shown, step S103, determining the motion control parameters of the tracking device according to the attribute information of the target recognition frame and the attribute information of the reference recognition frame, includes:

[0207] S501 : Determine a size difference between the reference recognition frame and the target recognition frame according to the size of the target recognition frame and the size of the reference recognition frame.

[0208] S502: Determine motion control parameters according to the size difference.

[0209] Among them, the attribute information may include size, the size of the target recognition frame includes length and width, and the size of the reference recognition frame includes length and width. Since the distance between the tracking device and the target object changes dynamically during the tracking process, generally the longer the distance, the smaller the size of the generated target recognition frame, and the closer the distance, the larger the size of the generated target recognition frame. Therefore, the size difference between the reference recognition frame and the target recognition frame can be calculated, and the size difference includes the length difference and / or the width difference. Then, the motion control parameters are determined based on the size difference. In this way, during the tracking process, the motion control parameters can be dynamically adjusted based on the size difference, so that the distance between the tracking device and the target object during the tracking process is as close as possible to the distance between the tracking device and the target object corresponding to the reference recognition frame, avoiding the tracking device being too close to the target object (possibly colliding) or too far away (possibly losing it), thereby improving the tracking accuracy.

[0210] Optionally, the attribute information further includes: location;

[0211] Step S502, determining motion control parameters based on the size difference, includes:

[0212] The motion control parameters are determined according to the size difference, the position of the target recognition frame and the position of the reference recognition frame.

[0213] Among them, the center point position of the reference identification frame can be the center point position of the current frame image, that is, the reference identification frame is displayed in the middle of the current frame image. Then, while considering the size difference, the position difference between the reference identification frame and the target identification frame is determined to determine the motion control parameters. In this way, when the tracking device is controlled to move according to the motion parameters, the position of the target identification frame identified during the tracking process is made as close to the position of the reference identification frame as possible, that is, the target identification frame is placed in the middle of the current frame image as much as possible, avoiding the current frame image not displaying the target identification frame (losing the target object), thereby improving the tracking accuracy.

[0214] Among them, a pixel coordinate system can be established, the horizontal coordinate corresponds to the length of the identification frame, the vertical coordinate corresponds to the width of the identification frame, and the position difference between the reference identification frame and the target identification frame can be the distance difference between the reference identification frame and the target identification frame on the horizontal coordinate in the pixel coordinate system.

[0215] The tracking method of this embodiment determines the size difference between the reference recognition frame and the target recognition frame based on the size of the target recognition frame and the size of the reference recognition frame, and determines the motion control parameters based on the size difference, thereby improving tracking accuracy.

[0216] Figure 7 The following is a schematic diagram of the tracking method provided in the embodiment of the present application. Figure 6 ,like Figure 7 As shown, the motion control parameters are determined according to the size difference, the position of the target recognition frame, and the position of the reference recognition frame, including:

[0217] S601 : Determine a position difference between the reference recognition frame and the target recognition frame according to the position of the target recognition frame and the position of the reference recognition frame.

[0218] Figure 8 The schematic diagram of the current frame image provided by the embodiment of the present application is shown Figure 2 ,like Figure 8 As shown, the center point position of the reference recognition frame is the center point position of the current frame image, and the position difference (xError) between the reference recognition frame and the target recognition frame can be the distance between the horizontal coordinate (x_GT) of the center point of the reference recognition frame and the horizontal coordinate (x_target) of the center point of the target recognition frame in the pixel coordinate system, that is, xError = x_GT-x_target.

[0219] The size difference (hError) between the reference recognition frame and the target recognition frame may be the difference between the width of the reference recognition frame (h_GT) and the width of the target recognition frame (h_target), ie, hError=h_GT-h_target, where xError and hError are both in pixels.

[0220] S602: Determine the angular velocity corresponding to the position difference from a preset angular velocity control curve.

[0221] S603: Determine the linear speed corresponding to the size difference from a preset linear speed control curve.

[0222] S604: Use the linear velocity and angular velocity as motion control parameters.

[0223] The angular velocity control curve is determined based on the preset maximum angular velocity, the preset maximum position difference, and the preset minimum position difference; the linear velocity control curve is determined based on the preset maximum linear velocity, the preset maximum dimension difference, and the preset minimum dimension difference.

[0224] Figure 9 Schematic diagram of the line speed control curve provided by the embodiment of the present application is shown in FIG. Figure 9 As shown, the slope of the linear speed control area

[0225] k_verlocity = (MAX_Verlocity - 0) / (MAX_H_ERROR - MIN_H_ERROR), where MAX_Verlocity is the preset maximum linear velocity, which can be adjusted according to the user's walking speed and is generally set to 1200 mm / s based on the walking speed of an adult. MAX_H_ERROR is the preset maximum dimensional difference, and MIN_H_ERROR is the preset minimum dimensional difference, which can be selected based on experience and is not particularly limited in this embodiment. The unit of linear velocity (verlocity) is mm / s.

[0226] Figure 10 FIG. 1 shows a schematic diagram of an angular velocity control curve provided by an embodiment of the present application, as shown in FIG. Figure 10 shown

[0227] k_angleSpeed ​​= (MAX_AngleSpeed ​​- 0) / (MAX_X_ERROR - MIN_X_ERROR), where MAX_AngleSpeed ​​is the preset maximum angular velocity, MAX_X_ERROR is the preset maximum position difference, and MIN_X_ERROR is the preset minimum position difference. These values ​​can be selected based on experience and are not particularly limited in this embodiment. The unit of angular velocity (angleSpeed) is deg / s.

[0228] The angular velocity corresponding to the position difference can be determined from the preset angular velocity control curve, and the linear velocity corresponding to the size difference can be determined from the preset linear velocity control curve. The angular velocity and linear velocity are then used as motion control parameters of the tracking device. The angular velocity is used to control the steering angle of the tracking device, and the linear velocity is used to control the forward speed of the tracking device. Through this linear velocity, the distance between the tracking device and the target object during the tracking process can be made as close as possible to the distance between the tracking device and the target object corresponding to the reference identification frame, avoiding the tracking device being too close to the target object (possibly colliding) or too far away (possibly losing it). The angular velocity can be used to make the target identification frame as close as possible to the center of the current frame image, avoiding the current frame image not displaying the target identification frame (losing the target object).

[0229] The tracking method of this embodiment determines the position difference between the reference and target recognition frames based on the positions of the target recognition frame and the reference recognition frame. The angular velocity corresponding to the position difference is determined from a preset angular velocity control curve, and the linear velocity corresponding to the size difference is determined from a preset linear velocity control curve. These linear and angular velocities are used as motion control parameters. This achieves single-target tracking, reduces tracking costs, and improves tracking accuracy.

[0230] Figure 11 FIG1 shows a schematic diagram of the structure of the tracking device provided by the embodiment of the present application, which can be integrated into the tracking device. Figure 11 As shown, the device includes:

[0231] The recognition module 701 is used to perform object recognition on the current frame of the image captured by the tracking device and generate corresponding recognition frames for the recognized objects, wherein the objects are located within the recognition frames;

[0232] a determination module 702 configured to determine a target recognition frame from each recognition frame based on image features corresponding to each recognition frame and image features corresponding to at least one historical recognition frame, wherein the historical recognition frame is a recognition frame of the target object to be tracked in historical images captured by the tracking device;

[0233] The determining module 702 is further configured to determine motion control parameters of the tracking device based on the attribute information of the target recognition frame and the attribute information of the reference recognition frame, wherein the reference recognition frame is obtained based on each of the historical recognition frames;

[0234] The control module 703 is configured to control the movement of the tracking device according to the motion control parameters to track the target object.

[0235] Optionally, the determining module 702 is specifically configured to:

[0236] performing a feature comparison between the recognition frame and each of the historical recognition frames based on image features corresponding to the recognition frame and image features corresponding to each of the historical recognition frames to obtain a comparison result, wherein the comparison result is used to indicate whether the recognition frame and each of the historical recognition frames belong to the same target object;

[0237] If the comparison result indicates that the recognition frame and each of the historical recognition frames belong to the same target object, the recognition frame is used as the target recognition frame.

[0238] Optionally, the determining module 702 is specifically configured to:

[0239] If the comparison result indicates that the recognition frame and each of the historical recognition frames belong to the same target object, and the sum of the number of the recognition frame and each of the historical recognition frames is greater than or equal to a preset number threshold, the recognition frame is used as the target recognition frame.

[0240] Optionally, the determining module 702 is specifically configured to:

[0241] Determining similarities between the image features corresponding to the recognition frame and the image features corresponding to each of the historical recognition frames;

[0242] The comparison result is obtained according to each similarity.

[0243] Optionally, the determining module 702 is specifically configured to:

[0244] If each of the similarities exceeds a preset similarity threshold, and the deviation of each similarity does not exceed a preset deviation, it is determined that the recognition frame and each of the historical recognition frames belong to the same target object.

[0245] Optionally, it also includes:

[0246] The adjustment module 704 is configured to adjust operating parameters of the tracking device if the comparison result indicates that the recognition frame and each of the historical recognition frames belong to different objects. The operating parameters include a shooting angle.

[0247] Optionally, the adjustment module 704 is further configured to:

[0248] If the comparison result indicates that the identification frame and each of the historical identification frames belong to different objects, adjusting the value of the tracking loss flag;

[0249] If the value of the tracking loss flag is greater than a preset threshold, the operating parameters of the tracking device are adjusted, and the operating parameters include: a shooting angle.

[0250] Optionally, the determining module 702 is further configured to:

[0251] Filtering selectable identification frames from the identification frames according to attribute information of the identification frames, wherein the attribute information includes at least one of length, width, and area;

[0252] The determining module 702 is specifically configured to:

[0253] Based on the image features corresponding to the optional identification frame and the image features corresponding to each of the historical identification frames, a feature comparison is performed on the optional identification frame and each of the historical identification frames to obtain the comparison result, which is used to indicate whether the optional identification frame and each of the historical identification frames belong to the same target object.

[0254] Optionally, the determining module 702 is specifically configured to:

[0255] The identification frame with the largest area is selected from the identification frames as the optional identification frame.

[0256] Optionally, the determining module 702 is specifically configured to:

[0257] Determine the identification frame with the largest area among the identification frames;

[0258] If the area of ​​the identification frame with the largest area is greater than or equal to a preset area threshold, the identification frame with the largest area is determined to be the selectable identification frame.

[0259] Optionally, the determining module 702 is specifically configured to:

[0260] If the number of the identification frames in the current frame image is within a preset number range, selectable identification frames are selected from the identification frames according to attribute information of the identification frames.

[0261] Optionally, the determining module 702 is specifically configured to:

[0262] determining a size difference between the reference recognition frame and the target recognition frame according to the size of the target recognition frame and the size of the reference recognition frame;

[0263] The motion control parameter is determined according to the size difference.

[0264] Optionally, the attribute information further includes: location; the determining module 702 is specifically configured to:

[0265] The motion control parameter is determined according to the size difference, the position of the target recognition frame, and the position of the reference recognition frame.

[0266] Optionally, the determining module 702 is specifically configured to:

[0267] determining a position difference between the reference recognition frame and the target recognition frame according to the position of the target recognition frame and the position of the reference recognition frame;

[0268] Determining an angular velocity corresponding to the position difference from a preset angular velocity control curve, wherein the angular velocity control curve is determined based on a preset maximum angular velocity, a preset maximum position difference, and a preset minimum position difference;

[0269] Determining a linear speed corresponding to the size difference from a preset linear speed control curve, wherein the linear speed control curve is determined based on a preset maximum linear speed, a preset maximum size difference, and a preset minimum size difference;

[0270] The linear velocity and the angular velocity are used as the motion control parameters.

[0271] Optionally, the center point position of the reference identification frame is the center point position of the current frame image.

[0272] Optionally, the identification module 701 is specifically configured to:

[0273] The object of the current frame image is recognized according to the recognition network obtained by pre-training, and corresponding recognition frames are generated for the recognized objects, wherein the recognition network is trained based on multiple image samples, and the multiple image samples are respectively marked with sample recognition frames corresponding to each sample object.

[0274] Optionally, it also includes:

[0275] The processing module 705 is used to perform fitting processing on each of the historical recognition frames to obtain the reference recognition frame.

[0276] The tracking device of this embodiment, the implementation process and the implementation principle are similar to the tracking method provided in the above method embodiment, and will not be repeated here.

[0277] Figure 12 A schematic diagram of the structure of the tracking device provided in the embodiment of the present application is shown in FIG. Figure 12 As shown, the device includes: a processor 801, a memory 802 and a bus 803. The memory 802 stores machine-readable instructions executable by the processor 801. When the tracking device is running, the processor 801 communicates with the memory 802 through the bus 803, and the processor executes the machine-readable instructions to perform the above method embodiment.

[0278] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above method embodiment is executed.

[0279] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0280] In addition, the functional units in the various embodiments of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0281] The above is only a specific implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the protection scope of the present application.

Claims

1. A tracking method, characterized in that: include: Performing object recognition on a current frame of image collected by the tracking device, and generating corresponding recognition frames for the recognized objects, wherein the objects are located within the recognition frames; Determine a target recognition frame from each recognition frame according to image features corresponding to each recognition frame and image features corresponding to at least one historical recognition frame, wherein the historical recognition frame is a recognition frame of the target object to be tracked in the historical image collected by the tracking device; Determining the motion control parameters of the tracking device according to the attribute information of the target recognition frame and the attribute information of the reference recognition frame, wherein the reference recognition frame is obtained based on each of the historical recognition frames; Controlling the movement of the tracking device according to the motion control parameter to track the target object; The step of determining the motion control parameters of the tracking device according to the attribute information of the target recognition frame and the attribute information of the reference recognition frame comprises: Determining a size difference between the reference recognition frame and the target recognition frame according to the size of the target recognition frame and the size of the reference recognition frame; determining the motion control parameter according to the size difference; The attribute information also includes: location; The step of determining the motion control parameter according to the size difference comprises: determining the motion control parameter according to the size difference, the position of the target recognition frame, and the position of the reference recognition frame; The step of determining the motion control parameter according to the size difference, the position of the target recognition frame, and the position of the reference recognition frame includes: Determining a position difference between the reference recognition frame and the target recognition frame according to the position of the target recognition frame and the position of the reference recognition frame; Determining the angular velocity corresponding to the position difference from a preset angular velocity control curve, wherein the angular velocity control curve is determined according to a preset maximum angular velocity, a preset maximum position difference, and a preset minimum position difference; Determining a line speed corresponding to the size difference from a preset line speed control curve, wherein the line speed control curve is determined according to a preset maximum line speed, a preset maximum size difference, and a preset minimum size difference; The linear velocity and the angular velocity are used as the motion control parameters.

2. The method according to claim 1, characterized in that The step of determining a target recognition frame from each recognition frame according to image features corresponding to each recognition frame and image features corresponding to at least one historical recognition frame includes: According to the image features corresponding to the recognition frame and the image features corresponding to each of the historical recognition frames, feature comparison is performed on the recognition frame and each of the historical recognition frames to obtain a comparison result, wherein the comparison result is used to indicate whether the recognition frame and each of the historical recognition frames belong to the same target object; If the comparison result indicates that the recognition frame and each of the historical recognition frames belong to the same target object, the recognition frame is used as the target recognition frame.

3. The method according to claim 2, characterized in that If the comparison result indicates that the recognition frame and each of the historical recognition frames belong to the same target object, taking the recognition frame as the target recognition frame includes: If the comparison result indicates that the recognition frame and each of the historical recognition frames belong to the same target object, and the sum of the number of the recognition frame and each of the historical recognition frames is greater than or equal to a preset number threshold, the recognition frame is used as the target recognition frame.

4. The method according to claim 2, characterized in that: The step of performing feature comparison between the recognition frame and each of the historical recognition frames according to the image features corresponding to the recognition frame and the image features corresponding to each of the historical recognition frames to obtain a comparison result includes: Determine the similarities between the image features corresponding to the recognition frame and the image features corresponding to the historical recognition frames; The comparison result is obtained according to each similarity.

5. The method according to claim 4, characterized in that Obtaining the comparison result according to each similarity includes: If each of the similarities exceeds a preset similarity threshold, and the deviation of each of the similarities does not exceed a preset deviation, it is determined that the recognition frame and each of the historical recognition frames belong to the same target object.

6. The method according to claim 2, characterized in that After performing feature comparison between the recognition frame and each of the historical recognition frames based on the image features corresponding to the recognition frame and the image features corresponding to each of the historical recognition frames and obtaining the comparison result, the method further includes: If the comparison result indicates that the identification frame and each of the historical identification frames belong to different objects, then the working parameters of the tracking device are adjusted, and the working parameters include: shooting angle.

7. The method according to claim 2, characterized in that After performing feature comparison between the recognition frame and each of the historical recognition frames based on the image features corresponding to the recognition frame and the image features corresponding to each of the historical recognition frames and obtaining the comparison result, the method further includes: If the comparison result indicates that the identification frame and each of the historical identification frames belong to different objects, adjusting the value of the tracking loss flag; If the value of the tracking loss flag is greater than a preset threshold, the working parameters of the tracking device are adjusted, and the working parameters include: shooting angle.

8. The method according to claim 2, characterized in that: Before comparing the features of the recognition frame with the historical recognition frames according to the image features corresponding to the recognition frame and the image features corresponding to the historical recognition frames, the method further includes: Filtering optional identification frames from the identification frames according to attribute information of the identification frames, wherein the attribute information includes at least one of length, width and area; The step of performing feature comparison between the recognition frame and each of the historical recognition frames according to the image features corresponding to the recognition frame and the image features corresponding to each of the historical recognition frames to obtain a comparison result includes: Based on the image features corresponding to the optional identification box and the image features corresponding to each of the historical identification boxes, feature comparison is performed on the optional identification box and each of the historical identification boxes to obtain the comparison result, and the comparison result is used to indicate whether the optional identification box and each of the historical identification boxes belong to the same target object.

9. The method according to claim 8, characterized in that The selecting a selectable identification box from each identification box according to the attribute information of each identification box includes: The identification frame with the largest area is selected from the identification frames as the optional identification frame.

10. The method according to claim 8, characterized in that The selecting a selectable identification box from each identification box according to the attribute information of each identification box includes: Determine the identification frame with the largest area among the identification frames; If the area of ​​the identification frame with the largest area is greater than or equal to a preset area threshold, the identification frame with the largest area is determined to be the selectable identification frame.

11. The method according to claim 8, characterized in that The selecting a selectable identification box from each identification box according to the attribute information of each identification box includes: If the number of the identification frames in the current frame image is within a preset number range, selectable identification frames are selected from the identification frames according to attribute information of the identification frames.

12. The method according to claim 1, characterized in that The center point position of the reference recognition frame is the center point position of the current frame image.

13. The method according to claim 1, characterized in that The performing object recognition on the current frame of image collected by the tracking device includes: The object recognition is performed on the current frame image according to the pre-trained recognition network, and corresponding recognition frames are generated for the recognized objects, wherein the recognition network is trained according to multiple image samples, and the multiple image samples are respectively annotated with sample recognition frames corresponding to each sample object.

14. The method according to claim 1, characterized in that Before determining the motion control parameters of the tracking device according to the attribute information of the target recognition frame and the attribute information of the reference recognition frame, the method further includes: Fitting is performed on each of the historical recognition frames to obtain the reference recognition frame.

15. A tracking device, characterized in that: include: A recognition module, used to perform object recognition on a current frame of image collected by the tracking device, and generate corresponding recognition frames for the recognized objects, wherein the objects are located within the recognition frames; A determination module, configured to determine a target recognition frame from each recognition frame according to image features corresponding to each recognition frame and image features corresponding to at least one historical recognition frame, wherein the historical recognition frame is a recognition frame of a target object to be tracked in a historical image collected by the tracking device; The determination module is further used to determine the motion control parameters of the tracking device according to the attribute information of the target recognition frame and the attribute information of the reference recognition frame, wherein the reference recognition frame is obtained based on each of the historical recognition frames; A control module, configured to control the movement of the tracking device according to the motion control parameter to track the target object; The determination module is specifically used for: Determining a size difference between the reference recognition frame and the target recognition frame according to the size of the target recognition frame and the size of the reference recognition frame; determining the motion control parameter according to the size difference; The attribute information also includes: location; the determination module is specifically used to: determining the motion control parameter according to the size difference, the position of the target recognition frame, and the position of the reference recognition frame; The determination module is specifically used for: Determining a position difference between the reference recognition frame and the target recognition frame according to the position of the target recognition frame and the position of the reference recognition frame; Determining the angular velocity corresponding to the position difference from a preset angular velocity control curve, wherein the angular velocity control curve is determined according to a preset maximum angular velocity, a preset maximum position difference, and a preset minimum position difference; Determining a line speed corresponding to the size difference from a preset line speed control curve, wherein the line speed control curve is determined according to a preset maximum line speed, a preset maximum size difference, and a preset minimum size difference; The linear velocity and the angular velocity are used as the motion control parameters.

16. A tracking device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the tracking device is running, the processor and the memory communicate via the bus, and the processor executes the machine-readable instructions to perform the method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 14 is executed.

Images