Object tracking method, device, terminal device and storage medium
By obtaining the predicted boundary detection box of traffic signal objects and combining the intersection ratio and angle relationship of the current boundary detection box, using the Hungarian algorithm to match, the problem of poor tracking of traffic signal objects in the existing technology is solved, and the abnormality is achieved Accurate tracking in the situation enhances the stability of autonomous driving control.
Patent Information
- Application Number
- CN202310477945.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-27
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-04-27
AI Technical Summary
The existing traffic signal object tracking method is poor in abnormal situations (such as unstable detection of front and back frames, excessive change in the position of the boundary detection frame, obstruction, etc.), making it difficult to achieve stable tracking.
By obtaining the predicted boundary detection box of the target traffic signal object, based on the intersection ratio, angle relationship and Hungarian algorithm matching of the current boundary detection box and the predicted boundary detection box, it is judged and tracked to adapt to various abnormal situations.
It improves the robustness of the traffic signal object tracking process, ensures that the target can be accurately tracked under various abnormal situations, and enhances the stability of autonomous driving control.
Smart Images

Figure CN116580051B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of autonomous driving technology, and in particular to an object tracking method, apparatus, terminal device and storage medium. Background Art
[0002] In autonomous driving, the tracking of traffic signal objects is crucial. Traffic signal objects refer to objects such as traffic lights, traffic signs, and traffic markings that are used to convey traffic signals. Accurately tracking traffic signal objects is conducive to improving the control effect of autonomous driving.
[0003] At present, the process of tracking traffic signal objects generally detects the bounding box (BBox) corresponding to the traffic signal object, and then tracks the traffic signal object according to the size of the intersection over union (IOU) of the bounding box between the previous and next frames. However, in actual applications, there may be abnormal situations such as unstable detection of previous and next frames, excessive change in the position of the bounding box, and occlusion by obstacles. The current tracking method of traffic signal objects is prone to tracking failure or misassociation in these abnormal situations. In other words, the current tracking method of traffic signal objects has poor robustness. Summary of the invention
[0004] The main purpose of the present application is to provide an object tracking method, apparatus, terminal device and storage medium, aiming to solve the problem of poor robustness of the current tracking method of traffic signal objects.
[0005] To achieve the above object, the present application provides an object tracking method, which includes:
[0006] Obtaining a predicted boundary detection box of a target traffic signal object in a target picture;
[0007] The target traffic signal object is tracked based on the current boundary detection frame and the predicted boundary detection frame of the target image.
[0008] Optionally, the step of obtaining a predicted boundary detection box of the target traffic signal object in the target picture includes:
[0009] Acquire historical tracking information of the target traffic signal object in the target picture;
[0010] A predicted boundary detection box of the target traffic signal object in the target picture is calculated based on the historical tracking information.
[0011] Optionally, the step of obtaining the historical tracking information of the target traffic signal object in the target picture includes:
[0012] Obtaining a first detection time corresponding to a historical boundary detection frame of the target traffic signal object in the target picture;
[0013] The step of calculating and obtaining a predicted boundary detection frame of the target traffic signal object in the target picture based on the historical tracking information comprises:
[0014] Calculate the relative motion information of the camera according to the position information of the camera corresponding to the target image at the first detection time and the second detection time corresponding to the current boundary detection frame;
[0015] Calculating depth information of the target traffic signal object according to preset specification parameters of the target traffic signal object and relative motion information of the camera;
[0016] According to the depth information of the target traffic signal object, a predicted boundary detection box of the target traffic signal object in the target picture is calculated.
[0017] Optionally, the step of tracking the target traffic signal object based on the current boundary detection frame and the predicted boundary detection frame of the target picture includes:
[0018] Based on the current boundary detection frame and the predicted boundary detection frame of the target image, determining whether at least one preset tracking condition is met;
[0019] If so, determining that the current boundary detection box of the target image belongs to the target traffic signal object;
[0020] If not, it is determined that the current boundary detection box of the target picture does not belong to the target traffic signal object.
[0021] Optionally, the step of judging whether the current boundary detection frame and the predicted boundary detection frame of the target image meet at least one preset tracking condition comprises:
[0022] Performing an intersection-and-union ratio calculation on the current boundary detection frame of the target image and the predicted boundary detection frame to obtain an intersection-and-union ratio calculation result;
[0023] It is determined whether a preset first tracking condition is met according to the intersection-over-union ratio calculation result to obtain a corresponding first determination result.
[0024] Optionally, the step of judging whether the current boundary detection frame and the predicted boundary detection frame of the target image meet at least one preset tracking condition comprises:
[0025] Acquire a first angle relationship between the predicted boundary detection frame and a historical boundary detection frame of the target traffic signal object in the target picture, and a second angle relationship between the current boundary detection frame and a historical boundary detection frame of the target traffic signal object in the target picture;
[0026] Whether a preset second tracking condition is satisfied is determined according to the first angle relationship and the second angle relationship, and a corresponding second determination result is obtained.
[0027] Optionally, the step of tracking the target traffic signal object based on the current boundary detection frame and the predicted boundary detection frame of the target picture includes:
[0028] Matching the current boundary detection frame of the target image with the predicted boundary detection frame based on a preset Hungarian algorithm to obtain a corresponding matching result;
[0029] The target traffic signal object is tracked according to the matching result.
[0030] The present application also provides an object tracking device, the object tracking device comprising:
[0031] An acquisition module, used to acquire a predicted boundary detection frame of a target traffic signal object in a target picture;
[0032] A tracking module is used to track the target traffic signal object based on the current boundary detection frame and the predicted boundary detection frame of the target image.
[0033] An embodiment of the present application further proposes a terminal device, which includes a memory, a processor, and an object tracking program stored in the memory and executable on the processor, wherein the object tracking program implements the steps of the object tracking method described above when executed by the processor.
[0034] An embodiment of the present application further provides a computer-readable storage medium, on which an object tracking program is stored. When the object tracking program is executed by a processor, the steps of the object tracking method described above are implemented.
[0035] The object tracking method, apparatus, terminal device and storage medium proposed in the embodiments of the present application obtain the predicted boundary detection frame of the target traffic signal object in the target picture; based on the current boundary detection frame of the target picture and the predicted boundary detection frame, the target traffic signal object is tracked. Based on the scheme of the present application, the possible position of the target traffic signal object in the target picture is represented by the predicted boundary detection frame, and the target traffic signal object can be tracked based on the current boundary detection frame and the predicted boundary detection frame of the target picture, and can adapt to various abnormal situations in the tracking process, thereby improving the robustness of the tracking process of the target traffic signal object. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 This is a schematic diagram of the functional modules of the terminal device to which the object tracking device of this application belongs;
[0037] Figure 2 This is a flowchart of a first exemplary embodiment of the object tracking method of the present application;
[0038] Figure 3 This is a flow chart of a second exemplary embodiment of the object tracking method of the present application;
[0039] Figure 4 This is a flowchart of a third exemplary embodiment of the object tracking method of the present application;
[0040] Figure 5 This is a flowchart of a fourth exemplary embodiment of the object tracking method of the present application;
[0041] Figure 6 This is a flowchart of a fifth exemplary embodiment of the object tracking method of the present application;
[0042] Figure 7 This is a flowchart of a sixth exemplary embodiment of the object tracking method of the present application;
[0043] Figure 8 This is a flowchart of the seventh exemplary embodiment of the object tracking method of the present application.
[0044] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0045] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0046] The main solution of the embodiment of the present application is: obtaining the predicted boundary detection frame of the target traffic signal object in the target picture; tracking the target traffic signal object based on the current boundary detection frame of the target picture and the predicted boundary detection frame. Based on the solution of the present application, the possible position of the target traffic signal object in the target picture is represented by the predicted boundary detection frame, and the target traffic signal object can be tracked based on the current boundary detection frame and the predicted boundary detection frame of the target picture, and can adapt to various abnormal situations in the tracking process, thereby improving the robustness of the tracking process of the target traffic signal object.
[0047] Specifically, refer to Figure 1 , Figure 1 This is a functional module diagram of the terminal device to which the object tracking device of this application belongs. The object tracking device can be a device that is independent of the terminal device and can track objects, and it can be carried on the terminal device in the form of hardware or software. The terminal device can be a smart mobile terminal with data processing functions such as a mobile phone or a tablet computer, or a fixed terminal device or server with data processing functions.
[0048] In this embodiment, the terminal device to which the object tracking apparatus belongs includes at least an output module 110 , a processor 120 , a memory 130 and a communication module 140 .
[0049] The memory 130 stores an operating system and an object tracking program. The object tracking device can obtain the predicted boundary detection frame of the target traffic signal object in the target screen; based on the current boundary detection frame of the target screen and the predicted boundary detection frame, the tracking result of the target traffic signal object is stored in the memory 130; the output module 110 can be a display screen, etc. The communication module 140 can include a WIFI module, a mobile communication module, a Bluetooth module, etc., and communicates with an external device or server through the communication module 140.
[0050] When the object tracking program in the memory 130 is executed by the processor, the following steps are implemented:
[0051] Obtaining a predicted boundary detection box of a target traffic signal object in a target picture;
[0052] The target traffic signal object is tracked based on the current boundary detection frame and the predicted boundary detection frame of the target image.
[0053] Furthermore, when the object tracking program in the memory 130 is executed by the processor, the following steps are also implemented:
[0054] Acquire historical tracking information of the target traffic signal object in the target picture;
[0055] A predicted boundary detection box of the target traffic signal object in the target picture is calculated based on the historical tracking information.
[0056] Furthermore, when the object tracking program in the memory 130 is executed by the processor, the following steps are also implemented:
[0057] Obtaining a first detection time corresponding to a historical boundary detection frame of the target traffic signal object in the target picture;
[0058] The step of calculating and obtaining a predicted boundary detection frame of the target traffic signal object in the target picture based on the historical tracking information comprises:
[0059] Calculate the relative motion information of the camera according to the position information of the camera corresponding to the target image at the first detection time and the second detection time corresponding to the current boundary detection frame;
[0060] Calculating depth information of the target traffic signal object according to preset specification parameters of the target traffic signal object and relative motion information of the camera;
[0061] According to the depth information of the target traffic signal object, a predicted boundary detection box of the target traffic signal object in the target picture is calculated.
[0062] Furthermore, when the object tracking program in the memory 130 is executed by the processor, the following steps are also implemented:
[0063] Based on the current boundary detection frame and the predicted boundary detection frame of the target image, determining whether at least one preset tracking condition is met;
[0064] If so, determining that the current boundary detection box of the target image belongs to the target traffic signal object;
[0065] If not, it is determined that the current boundary detection box of the target picture does not belong to the target traffic signal object.
[0066] Furthermore, when the object tracking program in the memory 130 is executed by the processor, the following steps are also implemented:
[0067] Performing an intersection-and-union ratio calculation on the current boundary detection frame of the target image and the predicted boundary detection frame to obtain an intersection-and-union ratio calculation result;
[0068] It is determined whether a preset first tracking condition is met according to the intersection-over-union ratio calculation result to obtain a corresponding first determination result.
[0069] Furthermore, when the object tracking program in the memory 130 is executed by the processor, the following steps are also implemented:
[0070] Acquire a first angle relationship between the predicted boundary detection frame and a historical boundary detection frame of the target traffic signal object in the target picture, and a second angle relationship between the current boundary detection frame and a historical boundary detection frame of the target traffic signal object in the target picture;
[0071] Whether a preset second tracking condition is satisfied is determined according to the first angle relationship and the second angle relationship, and a corresponding second determination result is obtained.
[0072] Furthermore, when the object tracking program in the memory 130 is executed by the processor, the following steps are also implemented:
[0073] Matching the current boundary detection frame of the target image with the predicted boundary detection frame based on a preset Hungarian algorithm to obtain a corresponding matching result;
[0074] The target traffic signal object is tracked according to the matching result.
[0075] This embodiment uses the above scheme to obtain the predicted boundary detection frame of the target traffic signal object in the target picture; based on the current boundary detection frame of the target picture and the predicted boundary detection frame, the target traffic signal object is tracked. In this embodiment, the predicted boundary detection frame is used to represent the possible position of the target traffic signal object in the target picture. Based on the current boundary detection frame and the predicted boundary detection frame of the target picture, the target traffic signal object can be tracked, and it can adapt to various abnormal situations in the tracking process, thereby improving the robustness of the tracking process of the target traffic signal object.
[0076] Reference Figure 2 The first embodiment of the object tracking method of the present application provides a flowchart, and the object tracking method includes:
[0077] Step S10, obtaining a predicted boundary detection box of the target traffic signal object in the target picture.
[0078] Specifically, the traffic signal object involved in this embodiment refers to objects such as traffic lights, traffic signs, and traffic markings used to convey traffic signals. For example, a traffic light commonly seen in an autonomous driving scenario is a type of traffic signal object.
[0079] The target vehicle obtains a picture of the road based on the on-board target camera, that is, the target picture. There may be several traffic signal objects in the target picture. The target vehicle detects boundary detection frames corresponding to several traffic signal objects in the target picture based on a preset detection algorithm, including the boundary detection frame of the target traffic signal object.
[0080] Usually, the target vehicle is in a state of continuous motion, and the target traffic signal object may appear in continuous frames of the target screen (i.e., multiple continuous image frames). The position of the target traffic signal object in the continuous frames may also change at any time. By associating the target traffic signal object with the boundary detection box of the continuous frames, the target traffic signal object can be tracked.
[0081] However, in actual applications, abnormal situations may occur, such as unstable detection of previous and next frames, excessive changes in the position of the boundary detection box, and occlusion by obstacles. The current tracking method of traffic signal objects cannot achieve stable tracking of the target traffic signal object, and the tracking process has poor robustness.
[0082] To this end, this embodiment proposes a traffic signal object tracking method based on a predicted boundary detection frame. More specifically, it is necessary to obtain the historical boundary detection frame of the target traffic signal object in the historical frame (i.e., any frame before the current frame) and the relative motion information of the target vehicle / target camera, and calculate the predicted boundary detection frame of the target traffic signal object based on the historical boundary detection frame of the target traffic signal object and the motion information of the target vehicle / target camera.
[0083] Step S20: Tracking the target traffic signal object based on the current boundary detection frame and the predicted boundary detection frame of the target image.
[0084] Specifically, there are several boundary detection frames corresponding to traffic signal objects in the current frame of the target picture, namely, the current boundary detection frame. The current boundary detection frame in the target picture may belong to the target traffic signal object, or it may not belong to the target traffic signal object. In order to verify the ownership of the above-mentioned current boundary detection frame and track the target traffic signal object, the predicted boundary detection frame can be used as a reference, and whether the preset tracking conditions are met based on the predicted boundary detection frame and the current boundary detection frame. If the judgment result shows that the preset tracking conditions are met, then it can be determined that the current boundary detection frame belongs to the target traffic signal object, and the relevant information of the current boundary detection frame can be added to the tracking information of the target traffic signal object; if the judgment result shows that the preset tracking conditions are not met, then it can be determined that the current boundary detection frame does not belong to the target traffic signal object.
[0085] The predicted boundary detection box is a virtual boundary detection box. The current boundary detection box involved in this embodiment is not directly compared with the historical boundary detection box, that is, it does not involve the comparison of the actual boundary detection boxes of the previous and next frames. It can effectively overcome abnormal situations such as unstable detection of previous and next frames, excessive change in the position of the boundary detection box, and occlusion by obstacles, and can effectively improve the accuracy of the tracking results of traffic signal objects.
[0086] This embodiment uses the above scheme to obtain the predicted boundary detection frame of the target traffic signal object in the target picture; based on the current boundary detection frame of the target picture and the predicted boundary detection frame, the target traffic signal object is tracked. In this embodiment, the predicted boundary detection frame is used to represent the possible position of the target traffic signal object in the target picture. Based on the current boundary detection frame and the predicted boundary detection frame of the target picture, the target traffic signal object can be tracked, and it can adapt to various abnormal situations in the tracking process, thereby improving the robustness of the tracking process of the target traffic signal object.
[0087] Further, refer to Figure 3 The second embodiment of the object tracking method of the present application provides a flow chart based on the above Figure 2 In the embodiment shown, step S10, obtaining a predicted boundary detection frame of the target traffic signal object in the target picture and further refining the predicted boundary detection frame includes:
[0088] Step S101, obtaining historical tracking information of the target traffic signal object in the target picture.
[0089] Specifically, the predicted boundary detection frame involved in this embodiment needs to be constructed based on the historical tracking information of the target traffic signal object in the target picture, so it is necessary to obtain the historical tracking information of the target traffic signal object in the target picture. The historical tracking information includes the historical boundary detection frame of the target traffic signal object in the historical frame (i.e., any frame before the current frame), the detection time corresponding to the historical boundary detection frame, and other information.
[0090] Step S102: Calculate and obtain a predicted boundary detection box of the target traffic signal object in the target picture based on the historical tracking information.
[0091] Specifically, after obtaining the historical tracking information of the target traffic signal object in the target picture, the predicted boundary detection box of the target traffic signal object in the target picture can be calculated in combination with the motion information of the target vehicle / target camera.
[0092] It can be understood that the predicted boundary detection frame refers to the boundary detection frame located in the current frame predicted when the target traffic signal object is in the motion state corresponding to the motion information of the target vehicle / target camera, and can be used to represent the expected state of the target traffic signal object in the target picture. In addition, there are also several boundary detection frames corresponding to the traffic signal object in the current frame of the target picture, namely, the current boundary detection frame, which can be used to represent the actual state of the traffic signal object in the target picture.
[0093] This embodiment adopts the above scheme, specifically by obtaining the historical tracking information of the target traffic signal object in the target picture; and calculating the predicted boundary detection frame of the target traffic signal object in the target picture based on the historical tracking information. In this embodiment, the historical tracking information of the target traffic signal object in the target picture can be used to obtain an accurate predicted boundary detection frame, and the target traffic signal object can be tracked based on the current boundary detection frame and the predicted boundary detection frame of the target picture, and can adapt to various abnormal situations in the tracking process, thereby improving the robustness of the tracking process of the target traffic signal object.
[0094] Further, refer to Figure 4 The third embodiment of the object tracking method of the present application provides a flow chart based on the above Figure 3 In the embodiment shown, step S101, obtaining the historical tracking information of the target traffic signal object in the target picture is further refined, including:
[0095] Step S1011, obtaining a first detection time corresponding to a historical boundary detection frame of the target traffic signal object in a target picture.
[0096] Specifically, in order to accurately calculate the predicted boundary detection frame of the target traffic signal object in the target picture, it is necessary to know the movement state of the target vehicle in the past period of time. First, it is necessary to obtain the first detection time corresponding to the historical boundary detection frame of the target traffic signal object in the target picture, and the first detection time can also be the generation time of the historical frame.
[0097] Step S102, further refining the predicted boundary detection frame of the target traffic signal object in the target picture calculated based on the historical tracking information, includes:
[0098] Step S1021 , calculating relative motion information of the camera according to position information of the camera corresponding to the target image at the first detection moment and at the second detection moment corresponding to the current boundary detection frame.
[0099] Specifically, the second detection moment corresponding to the current boundary detection frame is determined. The first detection moment may also be the generation moment of the current frame, and the current frame refers to the latest frame. The position information of the camera corresponding to the target screen (i.e., the target camera) is retrieved according to the first detection moment and the second detection moment, that is, the first position information and the second position information of the target camera are obtained respectively. Then, the relative motion information of the target camera between the first detection moment and the second detection moment is calculated according to the first position information and the second position information. The relative motion information can represent a displacement.
[0100] Step S1022: Calculate the depth information of the target traffic signal object according to the preset specification parameters of the target traffic signal object and the relative motion information of the camera.
[0101] Specifically, the preset specification parameters are used to represent the actual specifications of the target traffic signal object, which are generally defined by road laws and regulations, national standards, local standards, industry standards, etc. Therefore, the preset specification parameters of the target traffic signal object can be extracted from the relevant documents recording the road laws and regulations, national standards, local standards, and industry standards. For example, parameters such as the height or width of a traffic light can be extracted from the relevant documents.
[0102] After the relative motion information of the target camera is obtained, the depth information of the target traffic signal object can be restored according to the preset specification parameters of the target traffic signal object and the relative motion information of the target camera. The depth information can be the depth value of the target traffic signal object in the vision of the target camera.
[0103] Step S1023: Calculate and obtain a predicted boundary detection box of the target traffic signal object in the target picture according to the depth information of the target traffic signal object.
[0104] Specifically, after obtaining the depth information of the target traffic signal object, the predicted position of the target traffic signal object in the target picture can be calculated according to the depth information of the target traffic signal object, and the predicted boundary detection box can be restored based on the predicted position of the target traffic signal object in the target picture.
[0105] This embodiment adopts the above scheme, specifically by obtaining the first detection moment corresponding to the historical boundary detection frame of the target traffic signal object in the target picture; calculating the relative motion information of the camera according to the position information of the camera corresponding to the target picture at the first detection moment and the second detection moment corresponding to the current boundary detection frame; calculating the depth information of the target traffic signal object according to the preset specification parameters of the target traffic signal object and the relative motion information of the camera; and calculating the predicted boundary detection frame of the target traffic signal object in the target picture according to the depth information of the target traffic signal object. In this embodiment, the relative motion information of the target camera is calculated based on the first detection moment and the second detection moment, and the depth information of the target traffic signal object is calculated based on the relative motion information of the target camera and the preset specification parameters of the target traffic signal object, and then the predicted boundary detection frame is restored based on the depth information. The predicted boundary detection frame can effectively reflect the predicted position of the target traffic signal object in the current frame, thereby improving the accuracy of the tracking result of the traffic signal object.
[0106] Further, refer to Figure 5The fourth embodiment of the object tracking method of the present application provides a flow chart based on the above Figure 2 In the embodiment shown, step S20, based on the current boundary detection frame and the predicted boundary detection frame of the target image, further refining the tracking of the target traffic signal object includes:
[0107] Step S201 : judging whether at least one preset tracking condition is met based on the current boundary detection frame and the predicted boundary detection frame of the target image.
[0108] Specifically, there are several boundary detection frames corresponding to traffic signal objects in the current frame of the target image, namely, the current boundary detection frame. The current boundary detection frame in the target image may belong to the target traffic signal object, or may not belong to the target traffic signal object. In order to verify the ownership of the above-mentioned current boundary detection frame and track the target traffic signal object, the predicted boundary detection frame can be used as a reference, and based on the predicted boundary detection frame and the current boundary detection frame, it is judged whether at least one preset tracking condition is met. For example, tracking condition A is set, and based on the predicted boundary detection frame and the current boundary detection frame, it is judged whether tracking condition A is met, and the corresponding judgment result is obtained; or, tracking condition B and tracking condition C are set, and based on the predicted boundary detection frame and the current boundary detection frame, it is judged whether tracking condition B and tracking condition C are met at the same time, and the corresponding judgment result is obtained.
[0109] Step S202: If yes, determine whether the current boundary detection box of the target image belongs to the target traffic signal object.
[0110] Specifically, if the above results indicate that at least one preset tracking condition is met, then it can be determined that the current boundary detection box of the target image belongs to the target traffic signal object, and the relevant information of the current boundary detection box can be added to the tracking information of the target traffic signal object to track the target traffic signal object.
[0111] Step S203: If not, it is determined that the current boundary detection frame of the target image does not belong to the target traffic signal object.
[0112] Specifically, if the aforementioned result indicates that at least one preset tracking condition is not met, then it can be determined that the current boundary detection frame of the target picture does not belong to the target traffic signal object.
[0113] This embodiment uses the above scheme to determine whether at least one preset tracking condition is met based on the current boundary detection frame and the predicted boundary detection frame of the target image; if so, it is determined that the current boundary detection frame of the target image belongs to the target traffic signal object; if not, it is determined that the current boundary detection frame of the target image does not belong to the target traffic signal object. This embodiment determines whether the current boundary detection frame belongs to the target traffic signal object by judging at least one tracking condition under the condition of obtaining the current boundary detection frame and the predicted boundary detection frame. This embodiment can adapt to various abnormal situations in the tracking process and improve the robustness of the tracking process of the target traffic signal object.
[0114] Further, refer to Figure 6 The fifth embodiment of the object tracking method of the present application provides a flowchart based on the above Figure 5 In the illustrated embodiment, step S201, based on the current boundary detection frame and the predicted boundary detection frame of the target image, determining whether at least one preset tracking condition is met is further refined, including:
[0115] Step S2011, performing an IoU calculation on the current boundary detection frame of the target image and the predicted boundary detection frame to obtain an IoU calculation result.
[0116] Specifically, after obtaining the predicted boundary detection frame, the intersection over union (IOU) can be further calculated based on the current boundary detection frame and the predicted boundary detection frame of the target image, that is, the overlap rate of the current boundary detection frame and the predicted boundary detection frame is calculated to obtain the ratio of the intersection and union of the current boundary detection frame and the predicted boundary detection frame. The ideal situation is that the current boundary detection frame and the predicted boundary detection frame completely overlap, that is, the ratio is 1.
[0117] Step S2012: judging whether a preset first tracking condition is satisfied according to the intersection-over-union ratio calculation result, and obtaining a corresponding first judgment result.
[0118] Specifically, the first tracking condition can be set based on a preset intersection-and-union ratio threshold. If the intersection-and-union ratio of the current boundary detection frame and the predicted boundary detection frame obtained by the aforementioned intersection-and-union ratio calculation process is greater than the intersection-and-union ratio threshold, then it can be considered that the preset first tracking condition is met; if the intersection-and-union ratio of the current boundary detection frame and the predicted boundary detection frame obtained by the aforementioned intersection-and-union ratio calculation process is less than the intersection-and-union ratio threshold, then it can be considered that the preset first tracking condition is not met.
[0119] This embodiment adopts the above scheme, specifically by calculating the intersection-and-union ratio of the current boundary detection frame and the predicted boundary detection frame of the target image, to obtain the intersection-and-union ratio calculation result; and judges whether the preset first tracking condition is met according to the intersection-and-union ratio calculation result to obtain the corresponding first judgment result. This embodiment adopts the intersection-and-union ratio calculation method to judge whether the intersection-and-union ratio of the current boundary detection frame and the predicted boundary detection frame meets the preset first tracking condition, and determines whether the current boundary detection frame belongs to the target traffic signal object according to the judgment result, which can adapt to various abnormal situations in the tracking process and improve the robustness of the tracking process of the target traffic signal object.
[0120] Further, refer to Figure 7 The sixth embodiment of the object tracking method of the present application provides a flow chart based on the above Figure 5 In the illustrated embodiment, step S201, based on the current boundary detection frame and the predicted boundary detection frame of the target image, determining whether at least one preset tracking condition is met is further refined, including:
[0121] Step S2013, obtaining a first angle relationship between the predicted boundary detection box and a historical boundary detection box of the target traffic signal object in the target picture, and a second angle relationship between the current boundary detection box and a historical boundary detection box of the target traffic signal object in the target picture.
[0122] Specifically, the predicted boundary detection frame and the historical boundary detection frame of the target traffic signal object in the target picture will form a certain angle relationship, that is, a first angle relationship; the current boundary detection frame and the historical boundary detection frame of the target traffic signal object in the target picture will also form a certain angle relationship, that is, a second angle relationship. This embodiment determines whether the current boundary detection frame belongs to the target traffic signal object by comparing the angle information, and it is necessary to first obtain the first angle relationship and the second angle relationship.
[0123] Step S2014: determine whether a preset second tracking condition is satisfied according to the first angle relationship and the second angle relationship, and obtain a corresponding second determination result.
[0124] Specifically, the second tracking condition can be set based on a preset angle difference threshold. In an ideal case, the first angle relationship and the second angle relationship are close or equal, that is, the current boundary detection frame is close to or overlaps with the predicted boundary detection frame. If the angle difference calculated based on the first angle relationship and the second angle relationship is less than the angle difference threshold, then it can be considered that the preset second tracking condition is met; if the angle difference calculated based on the first angle relationship and the second angle relationship is greater than the angle difference threshold, then it can be considered that the preset second tracking condition is not met.
[0125] This embodiment adopts the above scheme, specifically by obtaining the first angle relationship between the predicted boundary detection box and the historical boundary detection box of the target traffic signal object in the target picture, and the second angle relationship between the current boundary detection box and the historical boundary detection box of the target traffic signal object in the target picture; judging whether the preset second tracking condition is met according to the first angle relationship and the second angle relationship, and obtaining the corresponding second judgment result. This embodiment uses angle information to judge whether the current boundary detection box belongs to the target traffic signal object, and effectively improves the accuracy of the tracking result of the traffic signal object. This embodiment can especially overcome the abnormal situation of obstacle occlusion and improve the robustness of the tracking process of the traffic signal object.
[0126] Further, refer to Figure 8 The seventh embodiment of the object tracking method of the present application provides a flowchart based on the above Figure 2 In the embodiment shown, step S20, based on the current boundary detection frame and the predicted boundary detection frame of the target image, further refining the tracking of the target traffic signal object includes:
[0127] Step S204 , matching the current boundary detection frame of the target image with the predicted boundary detection frame based on a preset Hungarian algorithm to obtain a corresponding matching result.
[0128] Specifically, the Hungarian algorithm is a combinatorial optimization algorithm for solving the task allocation problem in polynomial time. Applied to this embodiment, at least one current boundary detection frame and a predicted boundary detection frame of the target image are matched based on the preset Hungarian algorithm, and the matching result obtained is the optimal current boundary detection frame. It can be understood that the optimal current boundary detection frame is the boundary detection frame that is closest to the predicted boundary detection frame.
[0129] Step S205: Track the target traffic signal object according to the matching result.
[0130] Specifically, it can be determined that the optimal current boundary detection frame belongs to the target traffic signal object, and relevant information of the optimal current boundary detection frame can be added to the tracking information of the target traffic signal object to track the target traffic signal object.
[0131] In this embodiment, the current boundary detection frame of the target image and the predicted boundary detection frame are matched based on a preset Hungarian algorithm to obtain a corresponding matching result; and the target traffic signal object is tracked according to the matching result. In this embodiment, the Hungarian algorithm can be used to match the optimal current boundary detection frame as the actual boundary detection frame of the target traffic signal object in the current frame, which can effectively improve the accuracy and robustness of the tracking process of the target traffic signal object.
[0132] In addition, an embodiment of the present application further provides an object tracking device, the object tracking device comprising:
[0133] An acquisition module, used to acquire a predicted boundary detection frame of a target traffic signal object in a target picture;
[0134] A tracking module is used to track the target traffic signal object based on the current boundary detection frame and the predicted boundary detection frame of the target image.
[0135] For the principle and implementation process of object tracking in this embodiment, please refer to the above embodiments and will not be repeated here.
[0136] In addition, an embodiment of the present application also proposes a terminal device, which includes a memory, a processor, and an object tracking program stored in the memory and executable on the processor, wherein the object tracking program implements the steps of the object tracking method described above when executed by the processor.
[0137] Since all the technical solutions of all the aforementioned embodiments are adopted when the object tracking program is executed by the processor, it has at least all the beneficial effects brought by all the technical solutions of all the aforementioned embodiments, which will not be described one by one here.
[0138] In addition, an embodiment of the present application further proposes a computer-readable storage medium, on which an object tracking program is stored. When the object tracking program is executed by a processor, the steps of the object tracking method described above are implemented.
[0139] Since all the technical solutions of all the aforementioned embodiments are adopted when the object tracking program is executed by the processor, it has at least all the beneficial effects brought by all the technical solutions of all the aforementioned embodiments, which will not be described one by one here.
[0140] Compared with the prior art, the object tracking method, apparatus, terminal device and storage medium proposed in the embodiments of the present application obtain the predicted boundary detection frame of the target traffic signal object in the target picture; based on the current boundary detection frame of the target picture and the predicted boundary detection frame, the target traffic signal object is tracked. Based on the scheme of the present application, the possible position of the target traffic signal object in the target picture is represented by the predicted boundary detection frame, and the target traffic signal object can be tracked based on the current boundary detection frame and the predicted boundary detection frame of the target picture, and can adapt to various abnormal situations in the tracking process, thereby improving the robustness of the tracking process of the target traffic signal object.
[0141] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0142] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present application.
[0144] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. An object tracking method, characterized in that: The object tracking method comprises: Obtaining a predicted boundary detection box of a target traffic signal object in a target picture; Tracking the target traffic signal object based on the current boundary detection frame and the predicted boundary detection frame of the target picture; The step of tracking the target traffic signal object based on the current boundary detection frame and the predicted boundary detection frame of the target picture includes: Based on the current boundary detection frame and the predicted boundary detection frame of the target image, determining whether at least one preset tracking condition is met; If so, determining that the current boundary detection box of the target image belongs to the target traffic signal object; If not, determining that the current boundary detection frame of the target picture does not belong to the target traffic signal object; The step of judging whether the current boundary detection frame and the predicted boundary detection frame of the target image meet at least one preset tracking condition comprises: Acquire a first angle relationship between the predicted boundary detection frame and a historical boundary detection frame of the target traffic signal object in the target picture, and a second angle relationship between the current boundary detection frame and a historical boundary detection frame of the target traffic signal object in the target picture; Determining whether a preset second tracking condition is satisfied according to the first angle relationship and the second angle relationship, and obtaining a corresponding second determination result, wherein the second tracking condition is set based on a preset angle difference threshold; The step of obtaining a predicted boundary detection frame of the target traffic signal object in the target picture includes: obtaining historical tracking information of the target traffic signal object in the target picture; calculating the predicted boundary detection frame of the target traffic signal object in the target picture based on the historical tracking information; The step of obtaining the historical tracking information of the target traffic signal object in the target picture includes: obtaining a first detection moment corresponding to a historical boundary detection frame of the target traffic signal object in the target picture; The step of calculating the predicted boundary detection frame of the target traffic signal object in the target picture based on the historical tracking information includes: calculating the relative motion information of the camera according to the position information of the camera corresponding to the target picture at the first detection moment and the second detection moment corresponding to the current boundary detection frame; calculating the depth information of the target traffic signal object according to the preset specification parameters of the target traffic signal object and the relative motion information of the camera; and calculating the predicted boundary detection frame of the target traffic signal object in the target picture according to the depth information of the target traffic signal object.
2. The object tracking method according to claim 1, characterized in that: The step of judging whether the current boundary detection frame and the predicted boundary detection frame of the target image meet at least one preset tracking condition comprises: Performing an intersection-and-union ratio calculation on the current boundary detection frame of the target image and the predicted boundary detection frame to obtain an intersection-and-union ratio calculation result; It is determined whether a preset first tracking condition is met according to the intersection-over-union ratio calculation result to obtain a corresponding first determination result.
3. The object tracking method according to claim 1, characterized in that: The method further comprises: Matching the current boundary detection frame of the target image with the predicted boundary detection frame based on a preset Hungarian algorithm to obtain a corresponding matching result; The target traffic signal object is tracked according to the matching result.
4. An object tracking device, characterized in that: The object tracking device comprises: An acquisition module, used to acquire a predicted boundary detection frame of a target traffic signal object in a target picture; A tracking module, configured to track the target traffic signal object based on the current boundary detection frame and the predicted boundary detection frame of the target image; The tracking module is further used to determine whether the current boundary detection frame of the target picture and the predicted boundary detection frame meet at least one preset tracking condition based on the current boundary detection frame of the target picture; if so, determine that the current boundary detection frame of the target picture belongs to the target traffic signal object; if not, determine that the current boundary detection frame of the target picture does not belong to the target traffic signal object; The tracking module is further used to obtain a first angle relationship between the predicted boundary detection frame and a historical boundary detection frame of the target traffic signal object in the target picture, and a second angle relationship between the current boundary detection frame and a historical boundary detection frame of the target traffic signal object in the target picture; determine whether a preset second tracking condition is met according to the first angle relationship and the second angle relationship, and obtain a corresponding second determination result, wherein the second tracking condition is set based on a preset angle difference threshold; The acquisition module is further used to acquire the historical tracking information of the target traffic signal object in the target picture; and calculate the predicted boundary detection frame of the target traffic signal object in the target picture based on the historical tracking information; The acquisition module is further used to acquire the first detection moment corresponding to the historical boundary detection frame of the target traffic signal object in the target picture; The acquisition module is further used to calculate the relative motion information of the camera according to the position information of the camera corresponding to the target screen at the first detection moment and the second detection moment corresponding to the current boundary detection frame; calculate the depth information of the target traffic signal object according to the preset specification parameters of the target traffic signal object and the relative motion information of the camera; and calculate the predicted boundary detection frame of the target traffic signal object in the target screen according to the depth information of the target traffic signal object.
5. A terminal device, characterized in that: The terminal device includes a memory, a processor, and an object tracking program stored in the memory and executable on the processor. When the object tracking program is executed by the processor, the steps of the object tracking method according to any one of claims 1 to 3 are implemented.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an object tracking program, and when the object tracking program is executed by a processor, the steps of the object tracking method according to any one of claims 1 to 3 are implemented.
Citation Information
Patent Citations
A multi-target tracking method and system based on depth features
CN109816690A
Face tracking method, device and equipment and storage medium
CN110705478A
Target matching method and device, equipment and storage medium
CN114140730A
Method for detecting traffic lights and electronic equipment
CN114529883A
Traffic light detection method and device for automatic driving and electronic equipment
CN114782924A
