A stop line-based vehicle target fusion method and device

Abstract

The application relates to a stop line-based vehicle target fusion method and device, which comprises the following steps: dividing a lane in a field of view of a multi-view camera and an electric police camera based on a stop line to obtain a lane division result; tracking a target vehicle through the multi-view camera to obtain a multi-view camera data packet of the target vehicle in the lane; performing vehicle detection based on the multi-view camera data packet to obtain a vehicle detection result; continuing vehicle tracking based on the vehicle detection result to output an electric police camera data packet of the target vehicle; and matching data of the same target vehicle in the same lane in the multi-view camera data packet and the electric police camera data packet based on a time stamp to realize vehicle target fusion. Through data fusion of the multi-view camera and the electric police camera, the complete trajectory of the vehicle on the road can be reserved, and the vehicle flow and the vehicle speed of a road section can be more accurately counted.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation, and in particular relates to a vehicle target fusion method and device based on stop lines. Background Technology

[0002] In urban intelligent transportation systems, road traffic is highly complex. To monitor the real-time location and trajectory of vehicles on the road, filtering is needed, and traffic signals can be dynamically adjusted to optimize traffic flow. This requires tracking a vehicle's trajectory over a long distance. However, the coverage area of ​​each camera is limited, and it's crucial to maintain the uniqueness of the target ID across several consecutive cameras on the road. Current technology calculates the absolute position of the tracked target in each camera, then calculates the latitude and longitude of each target based on the absolute position, and finally uses an association algorithm to perform target fusion across cameras. The drawbacks of this existing technology are: the need to calibrate the intrinsic and extrinsic parameters of each camera beforehand, the need to set up several calibration points on the road, obtain the latitude and longitude of the calibration points using RTK equipment, and then transmit the calibration point information to the video equipment for coordinate calculation. RTK also refers to a type of measurement device that can receive satellite signals to achieve high-precision positioning, with accuracy down to the centimeter level. This calibration method is very complex and cumbersome; if there is inclement weather, equipment aging, or camera displacement, personnel need to recalibrate on-site. Summary of the Invention

[0003] This invention aims to overcome the shortcomings of existing technologies and proposes a vehicle target fusion method based on stop lines. This method allows vehicle target IDs to be relayed between electronic police cameras and multi-view cameras, enabling tracking at greater distances.

[0004] To achieve the above objectives, the present invention employs the following technical solution: a vehicle target fusion method based on stop lines, characterized in that it includes:

[0005] The lanes within the field of view of multi-view cameras and electronic police cameras are divided based on the stop line to obtain the lane division results;

[0006] The target vehicle is tracked by a multi-view camera, and the multi-view camera data packet of the target vehicle in the lane is obtained.

[0007] Vehicle detection is performed based on multi-view camera data packets to obtain vehicle detection results;

[0008] Based on the vehicle detection results, continue vehicle tracking and output the target vehicle's electronic police camera data packet.

[0009] Vehicle target fusion is achieved by matching data from multi-camera data packets and electronic police camera data packets containing the same target vehicle in the same lane based on timestamps.

[0010] As a further improvement of the present invention, the multi-view camera tracking of the target vehicle to obtain the multi-view camera data packet of the target vehicle within the lane specifically includes...

[0011] Obtain the latitude and longitude coordinates of the target vehicle and determine the target vehicle ID based on the tracking algorithm;

[0012] Acquire the frame of the target vehicle that enters the near-field view and determine the relative position of the target frame to the stop line.

[0013] The lane in which a vehicle is located is determined based on its relative position.

[0014] Multi-camera data packets are generated based on the target vehicle's latitude and longitude coordinates, target vehicle ID, the lane the vehicle is in, and the target vehicle's position relative to the stop line.

[0015] As a further improvement of the present invention, vehicle detection based on multi-view camera data packets, and the resulting vehicle detection results specifically include:

[0016] The target vehicle data in the multi-camera data packet is detected by a vehicle detection algorithm to obtain the vehicle frame information and the target vehicle ID.

[0017] As a further improvement of the present invention, the method further includes:

[0018] Data packets for the electronic police camera of the target vehicle are generated based on the vehicle frame information of the target vehicle, the target vehicle ID, and the position of the target vehicle relative to the stop line.

[0019] As a further improvement of the present invention, vehicle target fusion is achieved by matching data from multi-view camera data packets and electronic police camera data packets containing the same target vehicle in the same lane based on timestamps, including:

[0020] Match the data packets from multi-view cameras and electronic police cameras based on timestamps to find the frame data with the smallest time difference; traverse the two sets of data to find vehicles in the same lane that are simultaneously on the stop line in the two sets of data.

[0021] Using a HashMap with the target ID of the electronic police camera as the key and the target ID of the multi-camera as the value, the ID of the electronic police camera is replaced with the original target ID of the multi-camera until the corresponding target ID disappears, thus completing the target fusion.

[0022] The present invention also discloses a target fusion device based on stop lines, comprising:

[0023] The lane division module is used to divide the lanes within the field of view of multi-view cameras and electronic police cameras based on stop lines, and obtain the lane division results;

[0024] The vehicle tracking module is used to track target vehicles using multi-view cameras and obtain multi-view camera data packets of target vehicles within the lane.

[0025] The vehicle detection module is used to detect vehicles based on multi-view camera data packets and obtain vehicle detection results.

[0026] The vehicle tracking module is also used to continue vehicle tracking based on the vehicle detection results and output the target vehicle electronic police camera data packet.

[0027] The data fusion module is used to match data of the same target vehicle in the same lane from multi-camera data packets and electronic police camera data packets based on timestamps, thereby achieving vehicle target fusion.

[0028] As a further improvement of the present invention, the vehicle tracking module is also used for:

[0029] Obtain the latitude and longitude coordinates of the target vehicle and determine the target vehicle ID based on the tracking algorithm;

[0030] Acquire the frame of the target vehicle that enters the near-field view and determine the relative position of the target frame to the stop line.

[0031] The lane in which a vehicle is located is determined based on its relative position.

[0032] Multi-camera data packets are generated based on the target vehicle's latitude and longitude coordinates, target vehicle ID, the lane the vehicle is in, and the target vehicle's position relative to the stop line.

[0033] As a further improvement of the present invention, the vehicle detection module is also used for:

[0034] The target vehicle data in the multi-camera data packet is detected by a vehicle detection algorithm to obtain the vehicle frame information and the target vehicle ID.

[0035] As a further improvement of the present invention, the data fusion module is also used for:

[0036] Match the data packets from multi-view cameras and electronic police cameras based on timestamps to find the frame data with the smallest time difference; traverse the two sets of data to find vehicles in the same lane that are simultaneously on the stop line in the two sets of data.

[0037] Using a HashMap with the target ID of the electronic police camera as the key and the target ID of the multi-camera as the value, the ID of the electronic police camera is replaced with the original target ID of the multi-camera until the corresponding target ID disappears, thus completing the target fusion.

[0038] Compared with the prior art, the beneficial effects of the present invention are:

[0039] By fusing data from multi-camera and electronic police cameras, the complete trajectory of vehicles on the road can be preserved, allowing for more accurate statistics on traffic flow and speed on road sections. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the vehicle target fusion method based on stop lines of the present invention.

[0041] Figure 2 This is a schematic diagram of the vehicle target fusion device based on stop lines according to the present invention;

[0042] Figure 3 This is a scene diagram illustrating the vehicle target fusion method based on stop lines according to the present invention; Detailed Implementation

[0043] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0044] Example

[0045] like Figure 1 As shown, a vehicle target fusion method based on stop lines includes:

[0046] S110. Based on the stop line, divide the lanes within the field of view of the multi-view camera and the electronic police camera to obtain the lane division results. Specifically, set the first detection line and the second detection line based on the intersection stop line in the road; divide the road field of view into lanes; the first detection line refers to the detection line drawn in the road where vehicles are coming from, based on the intersection stop line; the second detection line is the detection line drawn in the entire intersection and the outgoing lanes, based on the intersection stop line.

[0047] S120. Track the target vehicle using a multi-view camera to obtain a multi-view camera data packet of the target vehicle in the lane. Specifically, track the target vehicle using a multi-view camera at the first detection line, determine the lane where the target vehicle is located according to the preset lane recognition area, and assemble each frame of the target vehicle image into a multi-view data packet, which is then sent to the electronic police camera via TCP network communication.

[0048] S130. Vehicle detection is performed based on multi-camera data packets to obtain vehicle detection results;

[0049] S140. Based on the vehicle detection results, continue vehicle tracking and output the target vehicle electronic police camera data packet; at the second detection line, use the results of the vehicle tracking algorithm to track the vehicle, obtain the target ID of the electronic police camera, and determine the target vehicle frame state and the lane it is in; use hash mapping to find the same target ID of each frame of tracking data after the stop line through the target ID of the electronic police camera, and track the target vehicle until it disappears.

[0050] S150, based on timestamp matching of data packets from multi-camera cameras and electronic police cameras, data of the same target vehicle in the same lane are used to achieve vehicle target fusion.

[0051] like Figure 2 As shown, lanes are divided according to the direction of vehicle travel, numbered from one side of the road to the other using the origin. The origin is 1. For example, if this road has 4 lanes, they are arranged from left to right as lane 1, lane 2, lane 3, and lane 4.

[0052] like Figure 3 As shown, the scene is equipped with multiple cameras, including a near-end camera, a mid-range camera, and a far-end camera. These cameras share overlapping fields of view, ensuring that the target ID of the same vehicle remains constant. The far-end camera captures the oncoming view of the vehicle, the mid-range camera captures the view below the multiple cameras, and the near-end camera shares the same field of view with the electronic police equipment. A tracking method is used to maintain the target ID of the same vehicle across the three cameras (near-end, mid-range, and far-end).

[0053] Each traffic enforcement camera contains a single camera that covers the entire intersection and oncoming lanes; multiple cameras and traffic enforcement cameras synchronize their time through the same NTP server.

[0054] NTP (Network Time Protocol) is a protocol used to synchronize computer time. It allows computers to synchronize with their servers or clock sources (such as quartz clocks, GPS, etc.), providing highly accurate time correction (less than 1 millisecond difference from the standard on LAN, and tens of milliseconds on WAN). It also uses encrypted authentication to prevent malicious protocol attacks. Time is propagated according to the tier of the NTP server. All servers are grouped into different Stratum tiers based on their distance from the external UTC source.

[0055] Optionally, the vehicle frame status of the target vehicle within the field of view is traversed. The vehicle frame status is divided into: before the stop line, on the stop line, and past the stop line. The determination method is as follows: before the stop line: the upper edge of the vehicle frame is below the stop line detection line; on the stop line: the upper edge of the vehicle frame is above the stop line, and the lower edge of the vehicle frame is below the stop line; past the stop line: the lower edge of the vehicle frame is above the stop line.

[0056] The vehicle frame information for each frame includes the pixel coordinates of the top left and bottom right corners of the vehicle. The vehicle frame state judgment result for each frame is stored in a cache, and the judgment result data is consistent with the multi-channel data packet structure.

[0057] Optionally, the multicast data packet includes: the timestamp of the current frame, the target ID of each vehicle, the lane number, the position status at the stop line, and the latitude and longitude coordinates;

[0058] The judgment result data includes: the timestamp of the current frame, the target ID of each vehicle, the lane number, the position status at the stop line, and the latitude and longitude coordinates. The timestamp is in milliseconds.

[0059] Optionally, the vehicle detection method involves matching the timestamp with the cached results of each frame of image data after receiving the data, and finding the frame with the smallest time difference; then, iterating through the two sets of data, it finds the vehicles in the same lane that are simultaneously on the stop line in both sets of data.

[0060] Optionally, the electronic police camera includes a TCP network service to receive each frame of data sent from the multi-camera system. It maintains a HashMap (a hash map, an implementation of the Map interface based on a hash table), where the key is the target ID of the electronic police camera and the value is the target ID of the multi-camera system.

[0061] The electronic traffic enforcement system captures real-time images from each frame of the camera and feeds them into a vehicle detection algorithm. The algorithm's result contains the frame outlines of all vehicles in that frame. This result is then used for vehicle tracking to obtain the target ID for each vehicle. Simultaneously, the system iterates through the frame outlines of each vehicle, classifying them into three states—before the stop line, on the stop line, and past the stop line—similar to the multi-view camera approach. The system also determines the lane the vehicle is in.

[0062] like Figure 3 As shown, this embodiment of the invention provides a target fusion device based on stop lines, comprising:

[0063] Lane division module 1 is used to divide the lanes within the field of view of multi-view cameras and electronic police cameras based on stop lines, and obtain lane division results;

[0064] Vehicle tracking module 2 is used to track target vehicles using multi-view cameras and obtain multi-view camera data packets of target vehicles within the lane.

[0065] Vehicle detection module 3 is used to perform vehicle detection based on multi-view camera data packets and obtain vehicle detection results;

[0066] The vehicle tracking module 2 is also used to continue vehicle tracking based on the vehicle detection results and output the target vehicle electronic police camera data packet.

[0067] Data fusion module 4 is used to match data of the same target vehicle in the same lane in multi-camera data packets and electronic police camera data packets based on timestamps, so as to achieve vehicle target fusion.

[0068] The vehicle tracking module 2 is also used for:

[0069] Obtain the latitude and longitude coordinates of the target vehicle and determine the target vehicle ID based on the tracking algorithm;

[0070] Acquire the frame of the target vehicle that enters the near-field view and determine the relative position of the target frame to the stop line.

[0071] The lane in which a vehicle is located is determined based on its relative position.

[0072] Multi-camera data packets are generated based on the target vehicle's latitude and longitude coordinates, target vehicle ID, the lane the vehicle is in, and the target vehicle's position relative to the stop line.

[0073] The vehicle detection module is also used for:

[0074] The target vehicle data in the multi-camera data packet is detected by a vehicle detection algorithm to obtain the vehicle frame information and the target vehicle ID.

[0075] The data fusion module 4 is also used for:

[0076] Match the data packets from multi-view cameras and electronic police cameras based on timestamps to find the frame data with the smallest time difference; traverse the two sets of data to find vehicles in the same lane that are simultaneously on the stop line in the two sets of data.

[0077] Using a HashMap with the target ID of the electronic police camera as the key and the target ID of the multi-camera as the value, the ID of the electronic police camera is replaced with the original target ID of the multi-camera until the corresponding target ID disappears, thus completing the target fusion.

[0078] Multi-camera scenarios include: multi-cameras suspended on traffic electronic monitoring poles at intersections, used to capture oncoming vehicles. Each multi-camera contains three cameras, covering a distance of 150 meters from the stop line at the intersection to the direction of oncoming vehicles. Electronic police camera scenarios include: electronic police cameras fixedly installed (standard mounting) on ​​traffic electronic monitoring poles at intersections, used to capture images in the direction of the intersection. Each camera contains one camera, covering a distance from the stop line at the intersection to the exit direction, and can cover the entire intersection and the outgoing lanes.

[0079] Multi-view cameras and electronic police cameras synchronize their time through the same NTP server, allowing vehicle target IDs to be relayed between the two devices, enabling tracking to a greater distance along the road.

[0080] In the description of this specification, references to terms such as "in one embodiment," "in yet another embodiment," "exemplary," or "in a particular embodiment," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0081] Although the present invention has been described in detail above with general descriptions, specific embodiments, and experiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A vehicle target fusion method based on stop lines, characterized in that, include: Based on the stop line, the lanes within the field of view of the multi-view camera and the electronic police camera are divided to obtain the lane division results; The target vehicle is tracked by a multi-view camera, and the multi-view camera data packet of the target vehicle in the lane is obtained. Vehicle detection is performed based on multi-view camera data packets to obtain vehicle detection results; Based on the vehicle detection results, continue vehicle tracking and output the target vehicle's electronic police camera data packet. Vehicle target fusion is achieved by matching data from multi-view camera data packets and electronic police camera data packets containing the same target vehicle in the same lane using timestamp matching, including: Based on timestamp matching of multi-camera data packets and electronic police camera data packets, find the frame data with the smallest time difference; traverse the two sets of data to find vehicles in the same lane that are simultaneously on the stop line in the two sets of data. Using a hash map with the target ID of the electronic police camera as the key and the target ID of the multi-camera as the value, the ID of the electronic police camera is replaced with the original target ID of the multi-camera until the corresponding target ID disappears, thus completing the target fusion.

2. The vehicle target fusion method based on stop lines according to claim 1, characterized in that, The target vehicle is tracked using multi-view cameras, and the resulting multi-view camera data packet for the target vehicle within the lane specifically includes... Obtain the latitude and longitude coordinates of the target vehicle and determine the target vehicle ID based on the tracking algorithm; Acquire the frame of the target vehicle that enters the near-field view and determine the relative position of the target frame to the stop line. The lane in which a vehicle is located is determined based on its relative position. Multi-camera data packets are generated based on the target vehicle's latitude and longitude coordinates, target vehicle ID, the lane the vehicle is in, and the target vehicle's position relative to the stop line.

3. The vehicle target fusion method based on stop lines according to claim 2, characterized in that, Vehicle detection based on multi-view camera data packets yields the following specific vehicle detection results: The target vehicle data in the multi-camera data packet is detected by a vehicle detection algorithm to obtain the vehicle frame information and the target vehicle ID.

4. The vehicle target fusion method based on stop lines according to claim 3, characterized in that, The method further includes: Data packets for the electronic police camera of the target vehicle are generated based on the vehicle frame information of the target vehicle, the target vehicle ID, and the position of the target vehicle relative to the stop line.

5. A target fusion device based on stop lines, comprising: The lane division module is used to divide the lanes within the field of view of multi-view cameras and electronic police cameras based on stop lines, and obtain the lane division results; The vehicle tracking module is used to track target vehicles using multi-view cameras and obtain multi-view camera data packets of target vehicles within the lane. The vehicle detection module is used to detect vehicles based on multi-view camera data packets and obtain vehicle detection results. The vehicle tracking module is also used to continue vehicle tracking based on the vehicle detection results and output the target vehicle electronic police camera data packet. The data fusion module is used to match data of the same target vehicle in the same lane from multi-camera data packets and electronic police camera data packets based on timestamps, thereby achieving vehicle target fusion; The data fusion module is also used to: match multi-camera data packets and electronic police camera data packets based on timestamps to find the frame data with the smallest time difference; traverse the two sets of data to find vehicles in the same lane that are simultaneously on the stop line in the two sets of data. Using a hash map with the target ID of the electronic police camera as the key and the target ID of the multi-camera as the value, the ID of the electronic police camera is replaced with the original target ID of the multi-camera until the corresponding target ID disappears, thus completing the target fusion.

6. The target fusion device based on stop line according to claim 5, characterized in that: The vehicle tracking module is also used for: Obtain the latitude and longitude coordinates of the target vehicle and determine the target vehicle ID based on the tracking algorithm; Acquire the frame of the target vehicle that enters the near-field view and determine the relative position of the target frame to the stop line. The lane in which a vehicle is located is determined based on its relative position. Multi-camera data packets are generated based on the target vehicle's latitude and longitude coordinates, target vehicle ID, the lane the vehicle is in, and the target vehicle's position relative to the stop line.

7. The target fusion device based on stop line according to claim 6, characterized in that, The vehicle detection module is also used for: The target vehicle data in the multi-camera data packet is detected by a vehicle detection algorithm to obtain the vehicle frame information and the target vehicle ID.

Images