Target tracking method and device and computer storage medium

Abstract

The invention provides a target tracking method and device and a computer storage medium, and the method comprises the steps: obtaining two adjacent frames of traffic light images, and the two adjacent frames of traffic light images comprise a first traffic light image and a second traffic light image; acquiring an image transformation matrix of the first traffic light image and the second traffic light image; projecting a first target detection frame of the first traffic light image to the second traffic light image by using the image transformation matrix to obtain a projection detection frame; and determining target detection values of the two adjacent frames of traffic light images based on the projection detection frame and a second target detection frame of the second traffic light image. By means of the mode, the influence caused by irregular movement of the traffic light target in the vehicle-mounted narrow-angle camera is effectively relieved, and therefore the tracking accuracy and robustness are improved.

Description

Technical Field

[0001] The present application relates to the field of target recognition technology, and in particular to a target tracking method, device and computer storage medium. Background Art

[0002] When using the autonomous driving function in the city, you will inevitably encounter intersections with traffic lights. The perception of the traffic light status at this time affects the smoothness and safety of the autonomous vehicle passing through the intersection. Therefore, it is necessary to perceive the status of the traffic light in advance at a far enough distance so that reasonable operations such as deceleration, braking, and turning can be made in advance.

[0003] In the prior art, since the size of the traffic light target in the field of view of the vehicle-mounted camera is relatively small, false detection and missed detection are prone to occur. Summary of the invention

[0004] The present application provides a target tracking method, device and computer storage medium.

[0005] To solve the above technical problems, the present application proposes a target tracking method, which includes: obtaining two adjacent frames of traffic light images, wherein the two adjacent frames of traffic light images include a first traffic light image and a second traffic light image; obtaining an image transformation matrix of the first traffic light image and the second traffic light image; using the image transformation matrix to project a first target detection frame of the first traffic light image onto the second traffic light image to obtain a projected detection frame; based on the projected detection frame and the second target detection frame of the second traffic light image, determining the target detection values ​​of the two adjacent frames of traffic light images.

[0006] Wherein, the target tracking method further includes: enlarging the first target detection frame, the second target detection frame, and / or the projection detection frame according to a preset ratio.

[0007] Among them, the step of enlarging the first target detection frame, the second target detection frame, and / or the projected detection frame according to a preset ratio includes: obtaining the coordinates of the corner points of the first target detection frame, the second target detection frame, and / or the projected detection frame; taking the center point of the detection frame as a reference, enlarging the horizontal coordinates of the corner point coordinates according to a first preset multiple, and enlarging the vertical coordinates of the corner point coordinates according to a second preset multiple; wherein the second preset multiple is greater than the first preset multiple.

[0008] Among them, the obtaining of the image transformation matrix of the first traffic light image and the second traffic light image includes: obtaining first vehicle posture information of the first traffic light image and second vehicle posture information of the second traffic light image; converting the first vehicle posture information into a first rotation matrix; converting the second vehicle posture information into a second rotation matrix; and generating the image transformation matrix using the first rotation matrix and the second rotation matrix.

[0009] The method of using the first rotation matrix and the second rotation matrix to generate the image transformation matrix includes: obtaining a vehicle transformation matrix based on the first rotation matrix, the second rotation matrix and a transformation matrix from a world coordinate system to a vehicle coordinate system; generating a camera transformation matrix based on the vehicle transformation matrix and the offset information between the camera and the vehicle; and generating the image transformation matrix based on the camera transformation matrix and camera intrinsic parameters.

[0010] The step of generating a camera transformation matrix based on the vehicle transformation matrix and the offset information between the camera and the vehicle includes: obtaining rotation information in a three-dimensional space based on the vehicle transformation matrix; fusing the rotation information with the offset information to obtain fused information; and generating the camera transformation matrix using the fused information.

[0011] Among them, using the image transformation matrix to project the first target detection frame of the first traffic light image onto the second traffic light image to obtain the projected detection frame includes: using the image transformation matrix to perform matrix transformation on the first target detection frame to obtain a transformed detection frame; normalizing the transformed detection frame to generate the projected detection frame.

[0012] Among them, determining the target detection value of the two adjacent frames of traffic light images based on the projection detection frame and the second target detection frame of the second traffic light image includes: obtaining the intersection-and-union ratio of the projection detection frame and the second target detection frame; in response to the intersection-and-union ratio being greater than a preset threshold, determining that the second target detection frame is the target detection value of the two adjacent frames of traffic light images; in response to the intersection-and-union ratio being less than or equal to the preset threshold, determining that the projection detection frame is the target detection value of the two adjacent frames of traffic light images.

[0013] In order to solve the above technical problems, the present application proposes a target tracking device, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above target tracking method.

[0014] In order to solve the above technical problems, the present application proposes a computer storage medium, which is used to store program data. When the program data is executed by a computer, it is used to implement the above target tracking method.

[0015] Different from the prior art, the beneficial effects of the present application are as follows: the target tracking device obtains two adjacent frames of traffic light images, wherein the two adjacent frames of traffic light images include a first traffic light image in front and a second traffic light image in the back; obtains the image transformation matrix of the first traffic light image and the second traffic light image; uses the image transformation matrix to project the first target detection frame of the first traffic light image onto the second traffic light image to obtain the projection detection frame; based on the projection detection frame and the second target detection frame of the second traffic light image, determines the target detection value of the two adjacent frames of traffic light images. Through the above method, the influence of irregular motion of the traffic light target in the vehicle-mounted narrow-angle camera is effectively alleviated, thereby improving the accuracy and robustness of tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 is a flowchart of a first embodiment of a target tracking method provided by the present application;

[0018] Figure 2 It is a schematic diagram of the overall process of the target tracking method provided by this application;

[0019] Figure 3 This is the target tracking method provided by this application. Figure 1 Schematic diagram of the flow of sub-steps of step S12;

[0020] Figure 4 It is a structural schematic diagram of an embodiment of a target tracking device provided by the present application;

[0021] Figure 5 It is a structural diagram of an embodiment of a computer storage medium provided by the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] The target tracking method of the present application is applied to a target tracking device, wherein the target tracking device of the present application can be a server, or a system composed of a server and a local terminal cooperating with each other. Accordingly, the various parts of the target tracking device, such as various units, subunits, modules, and submodules, can all be set in the server, or can be set in the server and the local terminal respectively.

[0024] Furthermore, the above-mentioned server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules, such as software or software modules for providing distributed servers, or it can be implemented as a single software or software module, which is not specifically limited here. In some possible implementations, the target tracking method of the embodiment of the present application can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0025] The main traffic light tracking methods in the existing technology can be divided into two types: the first is to use a predictor to predict the position of the traffic light in the next frame based on the historical movement trend of the target, and match the predicted frame with the detection frame of the next frame. Even if the target has a large displacement between frames, it can be compensated by the prediction process. Currently commonly used predictors include Kalman filtering, etc.; first, the traffic light detection frame is divided into high and low score frames, and then the prediction frame of the current frame image is obtained, and the prediction frame is matched with the high-score and low-score detection frames respectively, and finally the multiple matching results are integrated to obtain the final traffic light tracking result.

[0026] Another method is to use more matching criteria to improve the accuracy of target association between adjacent frames, such as IOU (Intersection over Union), appearance features, and some attributes that can uniquely identify the target itself. These attributes have their own advantages and can complement each other to a certain extent; IOU is the intersection over union ratio between detection frames. It is the most commonly used matching basis that can be obtained without additional operations, but it is easily affected by the movement and deformation of the target. Appearance features will not be affected by these behaviors, but additional algorithmic overhead is required to extract these features. Currently, common features include HOG features, appearance features extracted by CNN, etc.; during matching, attributes such as the category of traffic lights, the position in the target image, the confidence level, and the feature description vector are added to improve the accuracy of target association.

[0027] The existing technology has the following defects: The first traffic light tracking technology attempts to use the historical movement trend of the target to predict its next position, so as to alleviate the impact of the displacement of the target between frames on the tracking. However, this method can only play a certain role when the target moves regularly. Because 2D images are perspective views, the movement trend of the target in them is mostly non-linear. In addition, the traffic light target itself is small, and a slight visual jitter of the on-board camera will cause a huge displacement of the target's position in the image. In such a scenario, it is impossible to effectively predict the movement trend of the target, and it may even have a negative impact.

[0028] The second technology introduces a large number of matching criteria to improve the accuracy of target association. First, this requires the addition of a feature extraction model, which will bring additional algorithm overhead. Secondly, the appearance similarity between traffic light targets is high, and it is impossible to extract sufficiently reliable features to uniquely identify traffic light targets.

[0029] In view of the defects of the prior art, this application proposes a target tracking method, which will first improve the iou calculation method and expand the detection frame in the horizontal and vertical directions of the vehicle-mounted camera image. This is to adapt to the scene where the traffic light target undergoes a large displacement in the field of view of the vehicle-mounted narrow-angle camera when the vehicle is driving on a bumpy road; then the vehicle's own posture information is used to reversely eliminate the irregular displacement of the target in the inter-frame image caused by the jitter of the vehicle-mounted camera field of view and the vehicle's steering behavior through projection. Through the above method, no additional algorithm overhead is introduced to perform feature extraction operations, and only some existing information is used to effectively alleviate the impact of the irregular movement of the traffic light target in the vehicle-mounted narrow-angle camera, thereby improving the accuracy and robustness of tracking.

[0030] This application proposes a target tracking method, see Figure 1 , Figure 1 is a flowchart of a first embodiment of a target tracking method provided by the present application; Figure 2 It is a schematic diagram of the overall flow of the target tracking method provided by this application.

[0031] like Figure 1 As shown, the specific steps are as follows:

[0032] Step S11: Acquire two adjacent frames of traffic light images.

[0033] The two adjacent frames of traffic light images include a first traffic light image in front and a second traffic light image in the back.

[0034] Specifically, the target tracking device can obtain a continuous multi-frame video, and obtain two adjacent frames of traffic light images from the continuous multi-frame video. The continuous multi-frame video can be a real-time video or a stored video.

[0035] Furthermore, in an embodiment of the present application, the target tracking device obtains a target detection frame of the traffic light, and enlarges the first target detection frame, the second target detection frame, and / or the projection detection frame according to a preset ratio.

[0036] Among them, the target detection frame obtains the corner point coordinates of the first target detection frame, the second target detection frame, and / or the projection detection frame; based on the center point of the detection frame, the horizontal coordinates of the corner point coordinates are enlarged according to a first preset multiple, and the vertical coordinates of the corner point coordinates are enlarged according to a second preset multiple; wherein the second preset multiple is greater than the first preset multiple.

[0037] Specifically, based on the target detection result of the current frame, the target tracking device takes the center point of the original detection frame as the reference, and expands the detection frame by a certain proportion along the horizontal and vertical directions of the image. The target tracking device obtains the detection frame BBox of target i in frame t. t |{(x tl ,y tl ),(x br ,y br )}, where (x tl ,y tl ),(x br ,y br ) are the coordinates of the upper left corner and the lower right corner respectively. Then take the midpoint P of the detection box as t (cx t ,cy t ) as the reference, the image is expanded n times vertically and m times horizontally, and the expanded detection frame is BBox′ t |{(x′ tl ,y′ tl _,(x′ br ,y′ br )}, the specific formula is as follows:

[0038] x′ tl =cx t -m*(x br -x tl )

[0039] x′ br =cx t +m*(x br -x tl )

[0040] y′ tl =cy t -n*(y br -y tl )

[0041] y′ br =cy t +n*(y br -y tl )

[0042] like Figure 2 As shown, the present application first expands the traffic light detection frames of the current frame and the previous frame to a certain extent, especially in the vertical direction of the image. Then, based on the vehicle posture of the previous frame and the current frame, where the posture includes the deflection angles of the vehicle in three directions in the real three-dimensional coordinate system, the nonlinear offset of the traffic light target detection frame in the image caused by the vehicle's own shaking, turning and other operations is reversely eliminated, and the expanded detection frame of the previous frame is projected into the image of the current frame. Then match the projected expanded detection frame of the previous frame with the expanded detection frame of the current frame. And process them separately according to the matching results. Finally, repeat the above operations between frames to obtain the tracking trajectory of the traffic light target in multiple frames. For specific steps, please refer to the specific embodiments below.

[0043] Through the above method, the accuracy of traffic light tracking is further enhanced.

[0044] Step S12: Obtain an image transformation matrix of the first traffic light image and the second traffic light image.

[0045] Specifically, the present application proposes steps S121 to S123 as sub-steps of step S12, which are used to determine the transformation matrix of the first traffic light image and the second traffic light image.

[0046] For details, please see Figure 3 , Figure 3 This is the target tracking method provided by this application. Figure 1 Schematic diagram of the flow chart of the sub-steps of step S12.

[0047] like Figure 3 As shown, the specific steps are as follows:

[0048] Step S121: Acquire first vehicle posture information of the first traffic light image and second vehicle posture information of the second traffic light image.

[0049] The target tracking device obtains the vehicle posture information in two adjacent frames of traffic light images respectively, and obtains the first vehicle posture information imu_info of the first traffic light image t-1 and the second vehicle posture information imu_info of the second traffic light image t , where the attitude information includes the rotation angles roll (roll angle), pitch (pitch angle) and yaw (heading angle) in three directions in three-dimensional space.

[0050] Furthermore, the target calculates the transformation matrix H that projects the target detection frame of the previous frame to the current frame based on the posture information.

[0051] Step S122: converting the first vehicle posture information into a first rotation matrix; converting the second vehicle posture information into a second rotation matrix.

[0052] The target tracking device sends the first vehicle posture information imu_info t-1 and the second vehicle posture information imu_info t Converted into the first rotation matrix R t-1 and the second rotation matrix R t , where R t-1 and R t The acquisition process of is the same as R, and the conversion process is as follows:

[0053]

[0054] R=R yaw *R pitch *R roll

[0055] Step S123: Generate the image transformation matrix using the first rotation matrix and the second rotation matrix.

[0056] In one embodiment of the present application, the target tracking device obtains a vehicle transformation matrix based on the first rotation matrix, the second rotation matrix and the transformation matrix from the world coordinate system to the vehicle coordinate system; generates a camera transformation matrix based on the vehicle transformation matrix and the offset information between the camera and the vehicle; and generates the image transformation matrix based on the camera transformation matrix and the camera intrinsic parameters.

[0057] The camera transformation matrix is ​​calculated as follows: based on the vehicle transformation matrix, the rotation information of the three-dimensional space is obtained; the rotation information is fused with the offset information to obtain fused information; and the camera transformation matrix is ​​generated using the fused information.

[0058] Specifically, the target tracking device obtains the transformation matrix R_pre_cur_veh of the target from the previous frame to the current frame in the vehicle coordinate system:

[0059] R_pre_cur_veh=R_veh_imu*R t-1 -1 *R t *R_veh_imu -1

[0060] Among them, R_veh_imu is the transformation matrix from the real coordinate system to the vehicle coordinate system. Then the rotation matrix R_pre_cur_veh is restored to the rotation angles in three directions in the three-dimensional space, namely roll′, pitch′ and yaw′:

[0061] yaw′=arctan(R_pre_cur_veh[0,2],R_pre_cur_veh[2,2])

[0062]

[0063] roll′=arctan(sin(yaw′)*R_pre_cur_veh[2,1]-cos(yaw′)

[0064] *R_pre_cur_veh[0,1]),cos(yaw′)*R_pre_cur_veh[0,0]-sin(yaw′)

[0065] *R_pre_cur_veh[2,0]))

[0066] Add roll′, pitch′ and yaw′ to the offset angles roll0, pitch0 and yaw0 between the vehicle camera and the vehicle itself to restore the transformation matrix R_pre_cur_cam of the target in the camera coordinate system from the previous frame to the current frame, as follows:

[0067]

[0068] R_pre_cur_cam=R yaw *R pitch *R roll

[0069] Finally, multiply it by the camera intrinsic parameter and normalize it to get the final transformation matrix H from the previous frame to the current frame target in the image coordinate system:

[0070] H=K*R_pre_cur_cam*K -1

[0071] H[0]=H[0] / H[2]

[0072] H[1]=H[1] / H[2]

[0073] H[2]=H[2] / H[2]

[0074] Step S13: using the image transformation matrix to project the first target detection frame of the first traffic light image onto the second traffic light image to obtain a projected detection frame.

[0075] In one embodiment of the present application, the target tracking device uses the image transformation matrix to perform matrix transformation on the first target detection frame to obtain a transformed detection frame; and normalizes the transformed detection frame to generate the projected detection frame.

[0076] Expand the detection box BBox of the previous frame target contained in the tracking trajectory t ' -1 |{(x t ' l ,y t ' l ),(x′ br ,y b ' r )} Use the transformation matrix H to project into the current frame and get the BBox t ' - ′1|{(x t ' l ′,y t ′1′),(x′ br ′,y b ' r ′)}

[0077] Perform matrix transformation as follows:

[0078]

[0079] Normalize to get the position BBox of the target box of the previous frame in the current frame t ' - ′1|{(x t ′1′,y t ′1′),(x′ br ′,y b ' r ′)}, as follows:

[0080]

[0081] Step S14: determining the target detection values ​​of the two adjacent frames of traffic light images based on the projected detection frame and the second target detection frame of the second traffic light image.

[0082] Among them, in one embodiment of the present application, the target tracking device obtains the intersection-and-union ratio of the projected detection frame and the second target detection frame; in response to the intersection-and-union ratio being greater than a preset threshold, the second target detection frame is determined to be the target detection value of the two adjacent frames of traffic light images; in response to the intersection-and-union ratio being less than or equal to the preset threshold, the projected detection frame is determined to be the target detection value of the two adjacent frames of traffic light images.

[0083] Specifically, the target tracking device performs matching based on the projection frame of target i in frame t and the extended detection frames of all targets in frame t, and updates the trajectory of target i.

[0084] Furthermore, the target tracking device calculates the projection frame of target i in frame t And all target detection boxes in frame t The intersection-and-union ratio IOU is:

[0085]

[0086] When the IOU between the projection frame of target i and the detection frame j is greater than a specific threshold, the detection frame j is determined to be the detection frame of target i in frame t, and then the detection value is added to the trajectory of target i to update the trajectory. If no detection frame successfully matches the projection frame of target i, the projection frame is directly used as the detection value of the current frame target, added to the trajectory of target i, and the trajectory is updated.

[0087] The target tracking device repeats the operation of step S14 for each track in the track cluster and updates the status of each track. If there is a detection box that fails to match all tracks in the track cluster, it will be initialized as a new target as a new track; if there is a track that has not been successfully associated with any detection value for M consecutive frames, it will be deactivated and no longer output but still participate in matching. If its deactivation time exceeds K frames, it will be deleted.

[0088] Furthermore, the target tracking device repeatedly updates the state of each track between each adjacent frame to obtain the tracking tracks of all traffic light targets in the video sequence.

[0089] This application only uses the information obtained by traditional sensors and traditional methods such as image transformation, without introducing new deep learning tasks. It can reversely eliminate the irregular displacement of the target in the inter-frame image caused by the change of vehicle posture, improve the accuracy of inter-frame target association matching, and thus improve the tracking accuracy.

[0090] Without introducing new deep learning tasks, this application only uses target detection results, vehicle body posture information and traditional image transformation methods to eliminate the irregular jitter of the target in the field of view of the on-board camera, thereby achieving high-precision and stable traffic light target tracking.

[0091] In order to implement the target tracking method of the above embodiment, the present application also provides a target tracking device. Figure 4 , Figure 4 It is a structural schematic diagram of an embodiment of a target tracking device provided by the present application.

[0092] like Figure 4 As shown, the target tracking device 600 of this embodiment includes a processor 61 , a memory 62 , an input and output device 63 , and a bus 64 .

[0093] The processor 61 , the memory 62 , and the input / output device 63 are respectively connected to the bus 64 . The memory 62 stores a computer program, and the processor 61 is used to execute the computer program to implement the target tracking method of the above embodiment.

[0094] In this embodiment, the processor 61 may also be referred to as a CPU (Central Processing Unit). The processor 61 may be an integrated circuit chip having the ability to process signals. The processor 61 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The processor 61 may also be a GPU (Graphics Processing Unit), also known as a display core, a visual processor, or a display chip, which is a microprocessor that is specifically used for image computing on computers, workstations, game consoles, and some mobile devices (such as tablet computers, smart phones, etc.). The purpose of the GPU is to convert and drive the display information required by the computer system, and to provide a line scan signal to the display to control the correct display of the display. It is an important component that connects the display and the computer motherboard. As an important component of the computer host, the graphics card is responsible for outputting display graphics. The general-purpose processor may be a microprocessor or the processor 61 may also be any conventional processor, etc.

[0095] The present application also provides a computer storage medium, such as Figure 5 As shown, the computer storage medium 700 is used to store a computer program 71. When the computer program 71 is executed by a processor, it is used to implement the method described in the target tracking method embodiment of the present application.

[0096] The method involved in the target tracking method embodiment of the present application, when implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0097] The above description is only an implementation mode of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A target tracking method, characterized in that: The target tracking method comprises: Acquire two adjacent frames of traffic light images, wherein the two adjacent frames of traffic light images include a first traffic light image in front and a second traffic light image in the back; Obtaining an image transformation matrix of the first traffic light image and the second traffic light image; Projecting a first target detection frame of the first traffic light image onto the second traffic light image using the image transformation matrix to obtain a projected detection frame; Based on the projected detection frame and the second target detection frame of the second traffic light image, target detection values ​​of the two adjacent frames of traffic light images are determined.

2. The target tracking method according to claim 1, characterized in that: The target tracking method further includes: The first object detection frame, the second object detection frame, and / or the projection detection frame are enlarged according to a preset ratio.

3. The target tracking method according to claim 2, characterized in that: The step of enlarging the first target detection frame, the second target detection frame, and / or the projection detection frame according to a preset ratio includes: Obtaining coordinates of corner points of the first object detection frame, the second object detection frame, and / or the projected detection frame; Taking the center point of the detection frame as a reference, the horizontal coordinate of the corner point coordinate is enlarged by a first preset multiple, and the vertical coordinate of the corner point coordinate is enlarged by a second preset multiple; The second preset multiple is greater than the first preset multiple.

4. The target tracking method according to claim 1, characterized in that: The obtaining of the image transformation matrix of the first traffic light image and the second traffic light image includes: Acquire first vehicle posture information of the first traffic light image and second vehicle posture information of the second traffic light image; Converting the first vehicle posture information into a first rotation matrix; Converting the second vehicle posture information into a second rotation matrix; The image transformation matrix is ​​generated using the first rotation matrix and the second rotation matrix.

5. The target tracking method according to claim 4, characterized in that: The step of generating the image transformation matrix by using the first rotation matrix and the second rotation matrix includes: Acquire a vehicle transformation matrix based on the first rotation matrix, the second rotation matrix and a transformation matrix from a world coordinate system to a vehicle coordinate system; Generate a camera transformation matrix based on the vehicle transformation matrix and the offset information between the camera and the vehicle; The image transformation matrix is ​​generated based on the camera transformation matrix and camera intrinsic parameters.

6. The target tracking method according to claim 5, characterized in that: The generating a camera transformation matrix based on the vehicle transformation matrix and the offset information between the camera and the vehicle comprises: Based on the vehicle transformation matrix, obtaining rotation information of the three-dimensional space; Fusing the rotation information with the offset information to obtain fused information; The camera transformation matrix is ​​generated using the fused information.

7. The target tracking method according to claim 1, characterized in that: The step of projecting the first target detection frame of the first traffic light image onto the second traffic light image using the image transformation matrix to obtain the projected detection frame includes: Performing a matrix transformation on the first target detection frame using the image transformation matrix to obtain a transformed detection frame; The transformed detection frame is normalized to generate the projected detection frame.

8. The target tracking method according to claim 1 or 7, characterized in that: The determining the target detection values ​​of the two adjacent frames of traffic light images based on the projected detection frame and the second target detection frame of the second traffic light image includes: Obtaining an intersection-over-union ratio between the projected detection frame and the second target detection frame; In response to the intersection-over-union ratio being greater than a preset threshold, determining that the second target detection frame is a target detection value of the two adjacent frames of traffic light images; In response to the intersection-over-union ratio being less than or equal to the preset threshold, it is determined that the projected detection frame is a target detection value of the two adjacent frames of traffic light images.

9. A target tracking device, characterized in that: The target tracking device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the target tracking method as described in any one of claims 1 to 8.

10. A computer storage medium, characterized in that: The computer storage medium is used to store program data, and when the program data is executed by a computer, it is used to implement the target tracking method according to any one of claims 1 to 8.

Images