An Intersection Twin Trajectory Fusion Method Based on Aerial Cameras

Abstract

The present invention discloses an intersection twin trajectory fusion method based on an aerial camera, which relates to the technical field of image processing. The method includes: configuring an unmanned aerial vehicle (UAV) and a bayonet camera to meet preset conditions, and using the configured UAV and the configured bayonet camera to capture the area to be detected, so as to obtain a first acquisition video from the perspective of the UAV and a second acquisition video from the perspective of the bayonet camera; parsing the first acquisition video and the second acquisition video in real time to generate a first target queue and a second target queue; and performing fusion processing on any target vehicle based on the first target queue and the second target queue. The present invention fuses the targets detected by the aerial hovering UAV and the targets detected by the bayonet camera, ensuring smooth display of the trajectory of each vehicle target on the map, solving the problem of trajectory disappearance caused by occlusion, reducing the complexity of twin trajectory fusion between intersection cameras, and improving the display effect.

Description

Background Art

[0002] With the improvement of people's living standards, the number of motor vehicles on the road is increasing. Traffic safety and congestion at urban intersections have always been one of the key concerns. The cameras installed at urban intersections are becoming denser. Driven by the rapid development of traffic intelligence, the target trajectory fusion between multiple cameras has been realized. Due to the limited installation angle of the equipment and the large number of vehicles at the intersection, occlusion phenomena are often unavoidable. When the fused target is displayed on the twin map, target loss phenomena usually occur, lacking a certain degree of smoothness.

[0003] Existing technologies usually install multiple cameras and radars at intersections. Algorithms detect vehicle targets in each direction separately. First, the vehicle targets in a single direction are fused through a fusion algorithm, and then the targets in multiple directions are filtered through target fusion. The fusion effect is usually better when the vehicle target goes straight. When fusing the turning vehicle, abnormal heading angle phenomena often occur. Especially when there is occlusion, it is difficult for the fused target to match the actual trajectory. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method for fusing twin trajectories at intersections based on an aerial camera in view of the deficiencies of the prior art, specifically as follows:

[0005] 1) In a first aspect, the present invention provides a method for fusing twin trajectories at intersections based on an aerial camera, and the specific technical solution is as follows:

[0006] Configure the unmanned aerial vehicle (UAV) and the bayonet camera to meet the preset conditions, and use the configured UAV and the configured bayonet camera to photograph the area to be detected, obtaining the first acquisition video from the perspective of the UAV and the second acquisition video from the perspective of the bayonet camera;

[0007] Parse the first acquisition video and the second acquisition video in real time, determine at least one target vehicle in the first acquisition video and at least one vehicle license plate information in the second acquisition video, generate a first target queue based on at least one target vehicle, and generate a second target queue based on at least one vehicle license plate information;

[0008] Based on the first target queue and the second target queue, perform fusion processing on any target vehicle.

[0009] The beneficial effects of a method for fusing twin trajectories at intersections based on an aerial camera provided by the present invention are as follows:

[0010] Fuse the targets detected by the aerial hovering UAV aerial photography and the targets detected by the bayonet camera to ensure that while the trajectory of each vehicle target is smooth in the map display, each vehicle target also carries information such as vehicle type and license plate attributes, solving the phenomenon of trajectory disappearance caused by occlusion, greatly reducing the complexity of fusing twin trajectories between intersection cameras, and improving the display effect.

[0011] Based on the above solution, the present invention can also be improved as follows.

[0012] Further, the preset conditions include: the first lane range and the first lane number in the shooting picture of the UAV correspond one-to-one with the second lane range and the second lane number in the shooting picture of the bayonet camera.

[0013] Further, the configuration process of the UAV also includes:

[0014] Determine at least one longitude and latitude coordinate base point, and at least one aerial photography image pixel base point.

[0015] Further, the process of fusing any target vehicle based on the first target queue and the second target queue is specifically as follows:

[0016] Determine the vehicle license plate information that first appears in the second target queue, and the lane number corresponding to the vehicle license plate information, and perform fusion processing on the target vehicle corresponding to the vehicle license plate information that first appears based on the image pixel position mapping relationship between the UAV and the bayonet camera.

[0017] 2) In the second aspect, the present invention also provides an intersection twin trajectory fusion system based on an aerial photography camera, and the specific technical solution is as follows:

[0018] The configuration module is used for: configuring the UAV and the bayonet camera to meet the preset conditions, and shooting the area to be detected through the configured UAV and the configured bayonet camera to obtain the first acquisition video from the perspective of the UAV and the second acquisition video from the perspective of the bayonet camera;

[0019] The determination module is used for: parsing the first acquisition video and the second acquisition video in real time, determining at least one target vehicle in the first acquisition video and at least one vehicle license plate information in the second acquisition video, generating a first target queue based on at least one target vehicle, and generating a second target queue based on at least one vehicle license plate information;

[0020] The fusion module is used for: fusing any target vehicle based on the first target queue and the second target queue.

[0021] Based on the above solution, the present invention can also be improved as follows.

[0022] Further, the preset conditions include that the first lane range and the first lane number in the captured image of the UAV correspond one-to-one with the second lane range and the second lane number in the captured image of the bayonet camera.

[0023] Further, the configuration process of the UAV further includes:

[0024] Determine at least one longitude and latitude coordinate base point and at least one aerial image pixel base point.

[0025] Further, the process of performing fusion processing on any target vehicle based on the first target queue and the second target queue is specifically as follows:

[0026] Determine the license plate information of the vehicle that first appears in the second target queue and the lane number corresponding to the license plate information of the vehicle, and perform fusion processing on the target vehicle corresponding to the license plate information of the vehicle that first appears based on the image pixel position mapping relationship between the UAV and the bayonet camera.

[0027] 3) In a third aspect, the present invention further provides an electronic device, which includes a processor. The processor is coupled to a memory, and at least one computer program is stored in the memory. The at least one computer program is loaded and executed by the processor so that the electronic device implements any one of the above methods.

[0028] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor so that a computer implements any one of the above methods.

[0029] It should be noted that for the beneficial effects obtained by the technical solutions of the second to fourth aspects of the present invention and the corresponding possible implementation manners, reference may be made to the technical effects of the first aspect and its corresponding possible implementation manners described above, and details are not described herein again. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objects, and advantages of the present invention will become more apparent:

[0031] Figure 1 It is a schematic flowchart of a method for fusing intersection twin trajectories based on an aerial camera according to an embodiment of the present invention;

[0032] Figure 2 It is a structural framework diagram of an electronic device. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below in conjunction with the accompanying drawings.

[0034] As Figure 1 shown, a method for fusing twin trajectories at intersections based on an aerial camera according to an embodiment of the present invention has the following specific technical solutions:

[0035] S1, Configure the drone and the bayonet camera to meet the preset conditions, and use the configured drone and the configured bayonet camera to photograph the area to be detected, obtaining a first acquisition video from the perspective of the drone and a second acquisition video from the perspective of the bayonet camera;

[0036] S2, Analyze the first acquisition video and the second acquisition video in real time, determine at least one target vehicle in the first acquisition video and at least one vehicle license plate information in the second acquisition video, generate a first target queue based on at least one target vehicle, and generate a second target queue based on at least one vehicle license plate information;

[0037] S3, Based on the first target queue and the second target queue, perform fusion processing on any target vehicle.

[0038] The beneficial effects of a method for fusing twin trajectories at intersections based on an aerial camera provided by the present invention are as follows:

[0039] Fuse the targets detected by the hovering aerial drone and the targets detected by the bayonet camera, ensuring that while the trajectory of each vehicle target is smooth in the map display, each vehicle target also carries information such as vehicle type and license plate attributes, solving the problem of trajectory disappearance caused by occlusion, greatly reducing the complexity of fusing twin trajectories between intersection cameras, and improving the display effect.

[0040] S1, Configure the drone and the bayonet camera to meet the preset conditions, and use the configured drone and the configured bayonet camera to photograph the area to be detected, obtaining a first acquisition video from the perspective of the drone and a second acquisition video from the perspective of the bayonet camera. Among them:

[0041] The area to be detected is any traffic intersection that needs to be monitored in real time.

[0042] The drone refers to a hovering aerial drone, which has the function of aerial photography detection and can send the captured image data to the ground receiving end through the transmission device carried on the drone for subsequent processing of the video data collected by the drone by the ground receiving end. At the same time, the drone is also equipped with a device for receiving signals sent by the ground sending end for receiving instructions sent by the ground sending end in real time.

[0043] The process of configuring the drone to meet the preset conditions is specifically as follows:

[0044] The view angle of the images captured by the drone covers the entire area to be detected. Each lane in the image is numbered to obtain a lane number. It should be noted that the intersection is defined as four intersections where two vehicles are moving towards each other. In any one intersection, the lanes moving in the same direction are numbered in sequence, and the lanes moving towards each other are re-numbered, that is, the northbound lane 1, northbound lane 2, southbound lane 1, and southbound lane 2 of intersection A are generated.

[0045] A certain number of longitude and latitude coordinate base points are selected in the image captured by the drone, and the pixel point coordinates corresponding to the above longitude and latitude coordinate base points are further determined. Combining the longitude and latitude coordinate base points and the pixel point coordinates, a transformation matrix is determined to facilitate subsequent determination of the longitude and latitude coordinates corresponding to the target vehicle through the pixel center point coordinates of the target vehicle detected within the field of view. Multiple aerial image pixel base points A are determined in the image captured by the drone.

[0046] In the image captured by the bayonet camera, a certain number of pixel base points a are selected, and the lane numbers corresponding to the image captured by the bayonet camera are marked in the same way as the lane number marking method in the image captured by the drone.

[0047] Meeting the preset conditions means that the lane numbers and the position coordinates corresponding to the lane ranges in the image captured by the drone should be consistent with the lane numbers and the position coordinates corresponding to the lane ranges in the image captured by the bayonet camera. In addition, the positional relationship between the aerial image pixel base point A and the pixel base point a corresponds one by one.

[0048] S2. Parse the first acquisition video and the second acquisition video in real time to determine at least one target vehicle in the first acquisition video and at least one vehicle license plate information in the second acquisition video. Generate a first target queue based on at least one target vehicle and a second target queue based on at least one vehicle license plate information. Among them:

[0049] Parse the first acquisition video in real time to determine the longitude and latitude coordinates, heading angle, and lane number corresponding to any target vehicle;

[0050] Parse the second acquisition video in real time to determine the license plate, vehicle type, and lane number corresponding to any target vehicle.

[0051] It should be noted that the parsing result of the first acquisition video is sent to the intersection fusion edge box by wireless transmission through the 5G module. The intersection fusion edge box is the processing device for parsing the second acquisition video captured by the bayonet camera. In addition, the second acquisition video captured by the bayonet camera is sent to the intersection fusion edge box corresponding to the bayonet camera by limited transmission through the internal network.

[0052] S3. Based on the first target queue and the second target queue, perform fusion processing on any target vehicle.

[0053] The intersection edge box fusion algorithm receives the target queues from the drone algorithm end and the intersection edge box algorithm end in real time, and performs fusion based on the mapping relationship between the lane number where the target first appears and the image pixel position. Since the lane numbers configured in the drone image and the bayonet camera image correspond one by one, it is more convenient to perform fusion determination. When the fusion is successful, the target attributes of the bayonet video are updated to the drone target (taking the vehicle target as an example, the target attributes are mainly information such as license plate number, vehicle type, and color; the vehicle target attribute information detected by the drone is updated). Since the time for detecting each frame of image target by the drone edge end and the intersection edge box detecting each frame of image target is inconsistent, target fusion matching is performed through a certain time difference method, such as 1 second. Usually, fusion is performed at a fixed frame rate of 10 frames per second. The fused and unfused drone targets are uniformly sent to the platform for display.

[0054] It should be further noted that the target queue includes a first target queue and a second target queue, where the first target queue is generated according to the first acquisition video, and the second target queue is generated according to the second acquisition video. The first target queue and the second target queue include but are not limited to: the unique identifier (license plate number) of the vehicle, vehicle type, vehicle color, the lane number where the vehicle is located, and the position information of the vehicle.

[0055] Based on the above solution, the present invention can also be improved as follows.

[0056] Further, the preset conditions include: the first lane range and the first lane number in the shooting picture of the drone correspond one by one to the second lane range and the second lane number in the shooting picture of the bayonet camera.

[0057] Further, the configuration process of the drone further includes:

[0058] Determine at least one longitude and latitude coordinate base point, and at least one aerial image pixel base point.

[0059] Further, the process of performing fusion processing on any target vehicle based on the first target queue and the second target queue is specifically as follows:

[0060] Determine the vehicle license plate information that first appears in the second target queue, and the lane number corresponding to the vehicle license plate information, and perform fusion processing on the target vehicle corresponding to the vehicle license plate information that first appears based on the mapping relationship between the image pixel positions of the drone and the bayonet camera.

[0061] In the above embodiments, although the steps are numbered as S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation, which is also within the protection scope of the present invention. It can be understood that in some embodiments, it may include some or all of the above embodiments.

[0062] The present invention also provides an intersection twin trajectory fusion system based on an aerial camera. The specific technical solution is as follows:

[0063] The configuration module is used for: configuring the drone and the bayonet camera to meet the preset conditions, and using the configured drone and the configured bayonet camera to capture the area to be detected, obtaining the first captured video from the perspective of the drone and the second captured video from the perspective of the bayonet camera;

[0064] The determination module is used for: parsing the first captured video and the second captured video in real time, determining at least one target vehicle in the first captured video and at least one vehicle license plate information in the second captured video, generating a first target queue based on at least one target vehicle, and generating a second target queue based on at least one vehicle license plate information;

[0065] The fusion module is used for: performing fusion processing on any target vehicle based on the first target queue and the second target queue.

[0066] Based on the above solution, the present invention can also be improved as follows.

[0067] Further, the preset conditions include: the first lane range and the first lane number in the captured image of the drone correspond one-to-one with the second lane range and the second lane number in the captured image of the bayonet camera.

[0068] Further, the configuration process of the drone further includes:

[0069] Determining at least one longitude and latitude coordinate base point and at least one aerial image pixel base point.

[0070] Further, the process of performing fusion processing on any target vehicle based on the first target queue and the second target queue is specifically:

[0071] Determining the vehicle license plate information that first appears in the second target queue and the lane number corresponding to the vehicle license plate information, and performing fusion processing on the target vehicle corresponding to the vehicle license plate information that first appears based on the image pixel position mapping relationship between the drone and the bayonet camera.

[0072] It should be noted that the beneficial effects of the intersection twin trajectory fusion system based on an aerial camera provided in the above embodiments are the same as those of the above intersection twin trajectory fusion method based on an aerial camera, and will not be elaborated here. In addition, when the system provided in the above embodiments realizes its functions, only the above division of each functional module is used as an example for illustration. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be seen in the method embodiments, which will not be elaborated here.

[0073] As Figure 2 shown, an electronic device 300 according to an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and at least one computer program 330 is stored in the memory 310. The at least one computer program 330 is loaded and executed by the processor 320 to enable the electronic device 300 to implement any of the above methods. Specifically:

[0074] The electronic device 300 may vary greatly due to configuration or performance differences, and may include one or more processors 320 (Central Processing Units, CPUs) and one or more memories 310. Among them, at least one computer program 330 is stored in the one or more memories 310, and the at least one computer program 330 is loaded and executed by the one or more processors 320 to enable the electronic device 300 to implement an intersection twin trajectory fusion method based on an aerial camera provided in the above embodiments. Of course, the electronic device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input / output interface for input and output. The electronic device 300 may also include other components for implementing the functions of the device, which will not be elaborated here.

[0075] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor to enable a computer to implement any of the above methods.

[0076] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0077] In an exemplary embodiment, a computer program product or a computer program is further provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes any one of the above methods.

[0078] It should be noted that the terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and do not represent a limitation on a specific order or sequence. In appropriate cases, the order of use of similar objects may be interchanged, so that the embodiments of the present application described herein can be implemented in an order other than the illustrated or described order.

[0079] Those skilled in the art know that the present invention can be implemented as a system, a method or a computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, that is: it can be completely hardware, or completely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. Generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable media contain computer-readable program code.

[0080] Any combination of one or more computer-readable media can be adopted. The computer-readable media can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, device or component.

[0081] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An intersection twin trajectory fusion method based on an aerial camera, characterized in that, Including: Configuring the drone and the bayonet camera to meet preset conditions, and using the configured drone and the configured bayonet camera to capture the area to be detected, obtaining a first captured video from the perspective of the drone and a second captured video from the perspective of the bayonet camera; Parsing the first captured video and the second captured video in real time, determining at least one target vehicle in the first captured video and at least one vehicle license plate information in the second captured video, generating a first target queue based on at least one target vehicle, and generating a second target queue based on at least one vehicle license plate information; Based on the first target queue and the second target queue, performing fusion processing on any target vehicle; The process of configuring the drone to meet preset conditions specifically is: Controlling the view angle of the image captured by the drone to cover the entire area to be detected, numbering each lane in the image to obtain lane numbers, defining the intersection as four intersections where two vehicles are moving towards each other, in any one intersection, the lanes moving in the same direction are numbered in sequence, and the lanes moving towards each other are renumbered; Selecting a certain number of longitude and latitude coordinate base points in the image captured by the drone, and further determining the pixel point coordinates corresponding to the longitude and latitude coordinate base points, combining the longitude and latitude coordinate base points and the pixel point coordinates to determine the transformation matrix, and combining the transformation matrix to determine the longitude and latitude coordinates corresponding to the target vehicle through the pixel center point coordinates of the target vehicle detected within the field of view; In the image captured by the bayonet camera, selecting multiple pixel base points a, and labeling each pixel base point a, and at the same time labeling the lane number corresponding to the image captured by the bayonet camera; Among them, meeting the preset conditions means that the lane numbers and the position coordinates corresponding to the lane ranges in the image captured by the drone should be consistent with the lane numbers and the position coordinates corresponding to the lane ranges in the image captured by the bayonet camera, and the position relationship between the aerial image pixel base points and the pixel base points a is one-to-one corresponding.

2. The method for fusing twin trajectories at intersections based on an aerial camera according to claim 1, wherein, The preset conditions include: the first lane range and the first lane number in the captured image of the drone correspond one-to-one with the second lane range and the second lane number in the captured image of the bayonet camera.

3. A method for fusing intersection twin trajectories based on an aerial camera according to claim 1, characterized in that, The configuration process of the drone further includes: Determining at least one longitude and latitude coordinate base point and at least one aerial image pixel base point.

4. A method for fusing intersection twin trajectories based on an aerial camera according to claim 1, characterized in that The process of performing fusion processing on any target vehicle based on the first target queue and the second target queue specifically is: Determining the vehicle license plate information that first appears in the second target queue and the lane number corresponding to the vehicle license plate information, and performing fusion processing on the target vehicle corresponding to the vehicle license plate information that first appears based on the image pixel position mapping relationship between the drone and the bayonet camera.

5. An intersection twin trajectory fusion system based on an aerial camera, which adopts an intersection twin trajectory fusion method based on an aerial camera as described in claim 1, is characterized in that, This system includes: The configuration module is used for: configuring the drone and the bayonet camera to meet preset conditions, and using the configured drone and the configured bayonet camera to capture the area to be detected, obtaining a first captured video from the perspective of the drone and a second captured video from the perspective of the bayonet camera; The determination module is configured to: parse the first captured video and the second captured video in real time, determine at least one target vehicle in the first captured video and at least one vehicle license plate information in the second captured video, generate a first target queue based on the at least one target vehicle, and generate a second target queue based on the at least one vehicle license plate information; The fusion module is configured to: perform fusion processing on any target vehicle based on the first target queue and the second target queue.

6. The intersection twin trajectory fusion system based on an aerial camera according to claim 5, wherein, The preset conditions include: the first lane range and the first lane number in the captured image of the drone correspond one-to-one with the second lane range and the second lane number in the captured image of the bayonet camera.

7. A road intersection twin trajectory fusion system based on an aerial camera according to claim 5, characterized in that, The configuration process of the drone further includes: Determine at least one longitude and latitude coordinate base point, and at least one aerial image pixel base point.

8. A twin trajectory fusion system for intersections based on an aerial camera according to claim 5, characterized in that, The process of performing fusion processing on any target vehicle based on the first target queue and the second target queue is specifically: Determine the vehicle license plate information that first appears in the second target queue, and the lane number corresponding to the vehicle license plate information, and perform fusion processing on the target vehicle corresponding to the vehicle license plate information that first appears based on the image pixel position mapping relationship between the drone and the bayonet camera.

9. An electronic device, characterized in that, The electronic device includes a processor, the processor is coupled to a memory, and at least one computer program is stored in the memory. The at least one computer program is loaded and executed by the processor so that the electronic device implements the method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, At least one computer program is stored in the computer-readable storage medium. The at least one computer program is loaded and executed by a processor so that a computer implements the method according to any one of claims 1 to 4.

Images