A target tracking system, method and storage medium for roadside unit collaboration

Abstract

The present invention seeks to protect a target tracking system, method and storage medium for collaborative roadside units. The system consists of a perception module, a fusion tracking module and a communication module. The perception module includes a variety of environmental perception sensors for detecting pedestrians, vehicles and other targets within a certain range around the roadside unit; the fusion tracking module performs data association, tracking filtering and new target extraction based on the road target measurement data provided by the perception module and the target trajectory information released by the adjacent roadside unit received by the communication module, thereby obtaining accurate motion state information of the road target; the communication module is used to release road target information to vehicles within the communication range, and to exchange target trajectory information between adjacent roadside units. This patent achieves continuous and stable tracking of road targets through the collaboration of adjacent roadside units, improves the accuracy of road target tracking, and enhances the safety of autonomous driving or assisted driving systems.

Description

Technical Field

[0001] The present invention belongs to computer and automation technology, mainly relates to the field of target tracking technology, and specifically relates to a target tracking system and method for roadside unit collaboration. Background Art

[0002] Target tracking technology plays an important role in the decision-making, control and path planning of intelligent vehicles, and is the technical basis for intelligent vehicle safety warning and autonomous driving.

[0003] A Chinese patent application, "A Method for Multi-Target Tracking in a Multi-Radar Cross-Region Network on a Highway" (Application Number: 202011481467.4), discloses a method for generating tracking information by performing coordinate transformation and data association on roadside radar data from an entire highway section through a control center. This method only breaks through the limitations of the detection range of a single radar, extending the radar's detection range infinitely, enabling target tracking across the entire road section. However, the radars do not utilize the collaborative observations of adjacent radars for tracking. A Chinese patent application, "A Distributed Target Tracking Method Based on Improved Joint Probabilistic Data Association" (Application Number: CN201610821318.5), discloses a target tracking method that improves on joint probabilistic data association. This method modifies the echo association probability calculated within each target tracking gate by the probabilistic data association method, calculates the corresponding state estimate, then spatially associates the state estimates of each sensor, and finally fuses the state estimates of the same target to obtain the final target state estimate. However, this method also fails to achieve coordination between distributed adjacent sensors.

[0004] To address the above problems, the present invention shares the tracked target trajectory through collaboration between adjacent RSUs, achieves continuous and stable tracking of road targets, improves road target tracking accuracy, and enhances the perception ability and safety of intelligent vehicles. Summary of the Invention

[0005] The present invention aims to address the above-mentioned problems in the prior art by proposing a target tracking system and method for roadside unit collaboration to reduce the number of new target tracking initiation judgments when tracking road targets across RSUs in the vehicle-road cooperative system, thereby obtaining a stable and continuous tracking trajectory, thereby effectively improving the target tracking performance of roadside sensors in the vehicle-road cooperative system. The technical solutions of the present invention are as follows:

[0006] A target tracking system coordinated with roadside units (RSUs) comprises a perception module, a communication module, and a fusion tracking module. The perception module, which includes perception hardware such as lidar, millimeter-wave radar, and cameras, detects road targets such as pedestrians and vehicles within a certain range around the RSU and obtains target measurement information. The fusion tracking module performs coordinate conversion, data association, and fusion filtering on the road target measurement data provided by the perception module to obtain accurate motion state information of the road targets. The communication module is used to distribute road target information to vehicles within the communication range and to exchange target trajectory information between adjacent RSUs.

[0007] Furthermore, on each RSU, the coordinate system used by the sensors of the perception module and the fusion tracking module is a local coordinate system fixed on the RSU, and the coordinate system used by the communication module when sending the target trajectory is the earth coordinate system, including but not limited to the WGS-84 coordinate system.

[0008] Furthermore, on each RSU, the perception module detects road targets such as pedestrians and vehicles and obtains a measurement set

[0009] Furthermore, on each RSU, when the communication module broadcasts the target status, it is necessary to convert the target trajectory in the local coordinate system into the earth coordinate system, and then broadcast it to the adjacent RSUs and vehicles. The broadcast information includes: the ID of the RSU and its coordinates in the earth coordinate system; the ID of each target, the target motion state and the tracking filter error covariance matrix.

[0010] Furthermore, on each RSU, when receiving the trajectory sent by the adjacent RSU, the communication module converts the received target motion state into the local coordinate system and constructs the potential trajectory set T in the local coordinate system of the RSU. p ;

[0011] Furthermore, the fusion tracking module establishes a target motion state model and a measurement model in the local coordinate system of the RSU to describe the target motion:

[0012] x k+1 =F k x k +Γ k ω k ,k∈N

[0013] z k =H k x k +υ k ,k∈N

[0014] in represents the x position, x speed, y position, y speed, width, height, z of the target at time kk =[x,y,w,h] T represents the measurement of the target at time k, N represents a natural number set, and F k is the state transfer matrix, Γ k is the noise matrix, H k is the target measurement matrix at time k, ω k and v k are process noise and measurement noise, which are independent of each other.

[0015] Furthermore, the fusion tracking module associates the measurement set with the target trajectory and performs the measurement set Z k Compared with the local target trajectory T c and potential target trajectory T p Perform association and assign corresponding measurements to successfully associated targets. For each target in the successfully associated target trajectory set, perform tracking filtering on the associated target state to obtain the optimal estimate of the target motion state at time k and the covariance matrix of the filtering error. Tracking filtering methods include but are not limited to Kalman filtering.

[0016] Furthermore, the method for associating the fusion tracking module measurement set with the target trajectory includes the following steps:

[0017] (1) Target motion state prediction: For the existing trajectory set T c Each goal in According to the target motion state equation and the state estimation of the target at the previous moment and the covariance matrix Calculate the one-step prediction value of the target state separately and the covariance matrix of the one-step forecast error

[0018] (2) Measurement set Z k With the existing trajectory set T c Target t i c One-step forecast value Perform association and measure the successful association Assigned to the corresponding target t i c , the measurement that is not successfully associated is recorded as

[0019] (3) Potential target motion state prediction: For the potential trajectory set T p Each target t in i p , according to the target motion state equation and the state estimation of the target at the previous moment and the covariance matrix Calculate the one-step prediction value of the target state separately and the covariance matrix of the one-step forecast error

[0020] (4) Measurement set Z′ k With the potential trajectory set T p Each goal in One-step forecast value Perform association and measure the successful association Assign to the corresponding target The measurement that failed to be associated is recorded as

[0021] (5) For the measurement set Z″ k , extract the new target trajectory through the tracking initiation method. The tracking initiation method includes but is not limited to the track initiation method based on Hough transform.

[0022] Furthermore, the fusion tracking module maintains the IDs of successfully associated target trajectories and potential targets unchanged, and assigns target IDs to new targets extracted by the tracking initiation method by the RSU.

[0023] Furthermore, the target ID consists of two parts: the ID of the RSU and the serial number of the new target extracted by the RSU. The serial number increases by one each time the RSU extracts a new target.

[0024] Furthermore, the data association method between the measurement set and the existing trajectory and potential trajectory includes but is not limited to the nearest neighbor method.

[0025] A target tracking method for roadside unit collaboration includes the following steps:

[0026] (1) Deploy and install several RSUs on the roadside. The RSUs have sensing, communication, and computing functions and each has its own ID. The detection areas of any two adjacent RSUs installed on the road section to be detected partially overlap.

[0027] (2) Assign a unique ID to each RSU, measure and record the coordinates of the RSU in the Earth coordinate system;

[0028] (3) RSU receives information broadcasted by neighboring RSUs;

[0029] (4) The RSU detects pedestrians, vehicles and other road targets in real time, associates and filters them with local target trajectories and target trajectories broadcast by adjacent RSUs, updates the existing target status, and continuously tracks them. For measurements that fail to be associated, new targets are extracted by tracking initiation.

[0030] (5) The RSU converts the information of continuously tracked targets and new targets into the earth coordinate system and broadcasts it to adjacent RSUs and road vehicles within the communication range.

[0031] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the target tracking method for coordinated roadside unit as described in any one of the above.

[0032] The advantages and beneficial effects of the present invention are as follows:

[0033] The present invention proposes a target tracking system and method for roadside unit collaboration. In this method, adjacent roadside units (RSUs) share target trajectory information via a communication module, which serves as the potential target trajectory of the RSU. When a target enters the detection range of the RSU from the detection range of an adjacent RSU sensor, the RSU's sensor measurements are first associated with the local target trajectory. Unassociated measurements continue to be associated with the potential target trajectory, and the remaining unassociated measurements are used to initiate tracking. Without collaboration between RSUs, the potential target trajectory information from adjacent RSUs will not be effectively utilized. Measurements from these targets will be treated as new targets and included in the tracking initiation. On the one hand, the tracking initiation requires multiple measurements to extract the new target, resulting in tracking delays. On the other hand, the tracking initiation error of the new target will always differ from the actual tracking error, and the filter will take a certain amount of time to converge to the appropriate accuracy, thus preventing continuous and stable target tracking, affecting the convergence speed and tracking accuracy of the target tracking filter. Through the collaboration of adjacent roadside units, this patent utilizes the target trajectory information of adjacent RSUs to associate with the RSU's measurements, achieving continuous and stable tracking of road targets, improving road target tracking accuracy, and enhancing the safety of autonomous or assisted driving systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a structural diagram of a target tracking system coordinated by a roadside unit according to a preferred embodiment of the present invention.

[0035] Figure 2 This is a flow chart of a target tracking method for roadside unit collaboration of the present invention

[0036] Figure 3 This is an application scenario diagram of a target tracking system coordinated by a roadside unit of the present invention.

[0037] Figure 4 This is a rendering of a target tracking method for roadside unit collaboration in the present invention. DETAILED DESCRIPTION

[0038] The following will describe the technical solutions in the embodiments of the present invention in detail with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present invention.

[0039] The technical solution of the present invention to solve the above technical problems is:

[0040] like Figure 1 The figure shows a structure diagram of a target tracking system coordinated by a roadside unit of the present invention. The RSU system consists of three parts: a perception module, a communication module, and a fusion tracking module.

[0041] The perception module includes perception hardware such as lidar, millimeter-wave radar, and cameras, which detect road targets such as pedestrians and vehicles within a certain range around the RSU and obtain target measurement information;

[0042] The fusion tracking module performs data association, tracking filtering, and new target extraction based on the road target measurement data provided by the perception module and the target trajectory information released by adjacent roadside units received by the communication module, thereby obtaining accurate motion status information of road targets.

[0043] The communication module is used to publish road target information to vehicles within the communication range and to exchange target trajectory information between adjacent roadside units.

[0044] like Figure 2 The flowchart of the target tracking method of the present invention is shown as follows:

[0045] (1) Sensor target detection: At time k, the target is detected by the perception module and the measurement set is obtained.

[0046] (2) Sensor measurement is associated with existing target trajectories: First, the existing target motion state is predicted, and the existing trajectory set T is c Each target t in i c , according to the target motion state equation and the state estimation of the target at the previous moment and the covariance matrix Calculate the one-step prediction value of the target state separately and the covariance matrix of the one-step forecast error Measurement set Z k With the existing trajectory set T c Target t i c One-step forecast value Perform association and measure the successful association Assigned to the corresponding target t i c , the measurement that is not successfully associated is recorded as

[0047] (3) Association between sensor measurements and potential trajectories: First, receive the potential trajectories from neighboring RSUs and construct the potential trajectory set T in the local coordinate system of the RSU. p; Then perform potential target motion state prediction, and calculate the potential trajectory set T p Each goal in According to the target motion state equation and the state estimation of the target at the previous moment and the covariance matrix Calculate the one-step prediction value of the target state separately and the covariance matrix of the one-step forecast error The final measurement set Z k ′ and the potential trajectory set T p Each goal in One-step forecast value Perform association and measure the successful association Assign to the corresponding target The measurement that failed to be associated is recorded as

[0048] (4) Extract new targets at the beginning of tracking: k ,By tracking initiation method, new target trajectory is extracted;

[0049] (5) Target state filtering: For each target in the successfully associated target trajectory set, the associated target state is tracked and filtered to obtain the optimal estimate of the target motion state at time k and the covariance matrix of the filtering error;

[0050] (6) Target state broadcast: The updated optimal estimate of the target motion state is converted to the earth coordinate system and broadcast to adjacent RSUs and vehicles through the communication module together with the covariance matrix of the filtering error.

[0051] like Figure 3 This is an application scenario diagram of a target tracking system with coordinated roadside units of the present invention. In this scenario, RSU1 detects CV1 and performs continuous and stable tracking, and RSU2 detects CV2 and performs continuous and stable tracking. At time k, CV2 is detected by RSU1 for the first time. When RSU1 and RSU2 are not coordinated, there is no trajectory information about CV2 on RSU1, and CV2 cannot be associated or filtered and tracked. The measurement of CV2 can only be used to start tracking. CV2 can only be extracted as a new target after several cycles, and the tracking process is interrupted. In the present invention, when RSU1 and RSU2 are coordinated, the tracking of CV2 can be migrated from RSU2 to RSU1 to achieve continuous and stable tracking, thereby improving the target tracking performance in the vehicle-road cooperative scenario.

[0052] like Figure 4 The figure shows the effect of the target tracking method of the roadside unit cooperation of the present invention. When there is cooperation between RSUs, the tracking error covariance P of the target can be obtained from the adjacent RSUs. k, and when there is no coordination between RSUs, P cannot be accurately obtained. k , the new target can only use the pre-given initial value of the filter error covariance P0. Figure a shows the tracking error when the target is moving at a low speed with and without RSU coordination, where the solid line is the continuous tracking error when there is coordination between RSUs, and the dotted line is P0 <P k The tracking error at the beginning of new target tracking is P0>P k Figure 2 shows the tracking error at the start of new target tracking. Clearly, the tracking accuracy with RSU coordination is much higher than without coordination. Figure 2 shows the tracking effect when the target is moving at high speed. The tracking accuracy with RSU coordination is also much higher than without coordination.

[0053] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0054] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0055] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0056] The above embodiments should be understood as merely illustrating the present invention and not as limiting the scope of protection of the present invention. After reading the contents of the present invention, technicians may make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A target tracking method for roadside unit collaboration, characterized in that: For each RSU, the following steps are involved: 2.1 System Modeling: Establish a local coordinate system on the RSU, and establish the target motion state model and measurement model in the local coordinate system: x k+1 =F k x k +C k oh k ,k∈N z k =H k x k +υ k ,k∈N in represents the x position, x speed, y position, y speed, width, height, z of the target at time k k =[x,y,w,h] T represents the measurement of the target at time k, N represents a natural number set, and F k is the state transfer matrix, Γ k is the noise matrix, H k is the target measurement matrix at time k, ω k and v k are process noise and measurement noise, which are independent of each other; 2.2 Potential trajectory construction: At time k, receive the trajectory sent by the adjacent RSU and construct the potential trajectory set T in the local coordinate system of this RSU p ; 2.3 Sensor target detection: At time k, the target is detected by the perception module and the measurement set is obtained 2.4 Measurement set and target trajectory association: Measurement set Z k Compared with the local target trajectory T c and potential target trajectory T p Perform association and assign corresponding measurements to the successfully associated targets; 2.5 Target state filtering: For each target in the successfully associated target trajectory set, track and filter the associated target state to obtain the optimal estimate of the target motion state at time k and the covariance matrix of the filtering error; 2.6 Target state broadcast: The updated optimal estimate of the target motion state is converted to the Earth coordinate system and broadcast to adjacent RSUs and vehicles through the communication module together with the covariance matrix of the filtering error; The method for associating a measurement set with a target trajectory comprises the following steps: 3.1 Target motion state prediction: For the existing trajectory set T c Each target t in i c , according to the target motion state equation described in step 2.1 and the state estimation of the target at the previous moment and the covariance matrix Calculate the one-step prediction value of the target state separately and the covariance matrix of the one-step forecast error 3.2 Measurement Set Z k With the existing trajectory set T c Target t i c One-step forecast value Perform association and measure the successful association Assigned to the corresponding target t i c , the measurement that is not successfully associated is recorded as 3.3 Potential target motion state prediction: For the potential trajectory set T p Each goal in According to the target motion state equation described in step 2.1 and the state estimation of the target at the previous moment and the covariance matrix Calculate the one-step prediction value of the target state separately and the covariance matrix of the one-step forecast error 3.4 Measurement Set Z k ′ and the potential trajectory set T p Each goal in One-step forecast value Perform association and measure the successful association Assign to the corresponding target The measurement that failed to be associated is recorded as 3.5 Pairs of measurement sets Z k ″, extract new target trajectories through the tracking initiation method.

2. A target tracking system for roadside unit collaboration using the method of claim 1, characterized in that: include: Perception module, communication module, and fusion tracking module. The perception module includes perception hardware such as lidar, millimeter-wave radar, and cameras to detect road targets including pedestrians and vehicles within a certain range around the RSU and obtain target measurement information; The fusion tracking module performs data association, tracking filtering, and new target extraction based on the road target measurement data provided by the perception module and the target trajectory information released by adjacent roadside units received by the communication module to obtain accurate motion state information of the road target; the communication module is used to publish road target information to vehicles within the communication range and exchange target trajectory information between adjacent roadside units.

3. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the target tracking method for roadside unit collaboration as claimed in claim 1.

Images