Holographic intersection traffic management system based on the combination of radar and multi-camera

Abstract

The invention discloses a holographic intersection traffic management system based on a combination of a radar-vision integrated machine and a multi-eye camera, and relates to the field of intelligent intersection traffic management. The system comprises: a radar-vision integrated machine, a main control machine, and a multi-eye camera. Through a combination configuration mode of multiple cameras of a camera group in the radar-vision integrated machine, a configuration method of near- and far-view image acquisition is adopted, target detection and recognition are performed from near to far, and then the target is subjected to multi-camera fusion processing, so that the accuracy of target detection can be improved and the tracking processing of the target is convenient. At the same time, when vehicles at an intersection are waiting in a queuing area, the multi-eye camera can be configured to clearly collect the image of the vehicle directly below, and the occlusion problem when the radar-vision integrated machine collects data can be effectively solved. Moreover, a radar sensor is used to obtain the motion information of each moving target from the stop line of the intersection to the end of the road, and the motion information is fused with the target data detected and recognized by the camera group and the multi-eye camera, so that the accuracy of target recognition and tracking can be further improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent traffic management, and in particular to a holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera. Background Art

[0002] Vehicle-road collaboration is an inevitable trend in the development of traffic control, and urban intersections are important nodes for vehicle-road collaboration. With the rise of the concept of holographic intersections, major software and hardware manufacturers have made plans. Holographic intersections can provide high-precision, real-time traffic data, and are an important means to promote the refined implementation of intelligent traffic management and fully empower urban traffic management. It uses a variety of sensors to perceive road conditions from multiple perspectives and modalities, accurately perceive traffic conditions, and finally digitally restore the complex state of intersections to achieve on-demand scheduling and intelligent management of road rights. In each module of the holographic intersection, perception is the first step, and the configuration of the sensor directly affects the subsequent data processing and algorithm flow.

[0003] At present, the main method of traffic management at holographic intersections is a multi-camera pure vision solution, which uses cameras of different focal lengths and resolutions and relevant algorithms in the field of computer vision to complete tasks such as target detection, classification, tracking, and re-identification. However, the existing multi-camera pure vision solution is strongly affected by lighting when the camera collects images. It is affected by factors such as night, strong light, rain, snow, and smoke. The visibility is poor when the environmental conditions are poor, and it does not have the characteristics of all-day and all-weather. As a result, the efficiency and accuracy of traffic management at existing holographic intersections are low. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera, so as to solve the problem of low efficiency and accuracy of traffic management in existing holographic intersections.

[0005] To achieve the above-mentioned purpose, the present invention provides a holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera, the system comprising: a radar-vision integrated machine, a multi-eye camera, and a main control machine; the radar-vision integrated machine is configured on an electric alarm pole; the multi-eye camera is configured below the radar-vision integrated machine and connected to the radar-vision integrated machine;

[0006] The multi-eye camera is used to collect images of the traffic light waiting area and the intersection queuing area corresponding to the electric police pole;

[0007] The radar-vision integrated machine includes a camera group and a radar sensor;

[0008] The camera group is used to collect images of the entire intersection area corresponding to the electric police pole, images of the vehicle waiting area on the opposite road, and images of the road area extending to the far end of the opposite road corresponding to the electric police pole;

[0009] The radar sensor is used to collect information about various moving targets from the stop line at the intersection to the end of the far road;

[0010] The main control machine is used to perform cross-lens tracking, recognition and fusion of images of the traffic light waiting area and the intersection queuing area corresponding to the electric police pole, and obtain identification information, feature information and pixel position information of each target in the image;

[0011] The main control machine is further used to perform target recognition on the images of different areas collected by the camera group, establish target track information, and fuse the track information of each target obtained by the camera group with the speed information and position information of each target obtained by the radar sensor, wherein the track information includes the identification information, feature information and real position information of the target, and the real position information of the target is obtained according to the pixel position information of the target in the picture;

[0012] The main control machine is also used to fuse the cross-lens tracking, identification and fusion data corresponding to the multi-eye camera with the fusion data corresponding to the radar vision integrated machine, and perform intersection traffic management based on the fused data.

[0013] Furthermore, the multi-camera includes: a front-view camera, a downward-view camera, and a rear-view camera;

[0014] The front-view camera is in the same direction as the road and the angle between the center of the field of view and the horizontal line is 10-40 degrees. The rear-view camera is in the opposite direction of the road and the angle between the center of the field of view and the horizontal line is 10-40 degrees. The center of the field of view of the downward camera is vertically downward, and is used to sequentially collect images of the traffic light waiting area and the intersection queuing area corresponding to the electric warning pole.

[0015] Furthermore, when the images of the traffic light waiting area and the intersection queuing area corresponding to the electric police pole collected by the front-view camera, the downward-view camera, and the rear-view camera are not sufficient to completely cover the areas on both sides of the road, the multi-eye camera further includes a left-view camera and a right-view camera;

[0016] The left-side camera is perpendicular to the road and faces left, and the angle between the center of the field of view and the horizontal line is 5-45 degrees. The right-side camera is perpendicular to the road and faces right, and the angle between the center of the field of view and the horizontal line is 5-45 degrees. The cameras are used to collect images of the waiting area for traffic lights and the queuing area at the intersection corresponding to the electric alarm pole.

[0017] The main control machine is also used to perform cross-lens tracking, recognition and fusion of images of the traffic light waiting area and the intersection queuing area corresponding to the electric police pole collected by the front-view camera, the downward-view camera, the rear-view camera, the left-view camera and the right-view camera, so as to obtain the identification information, feature information and pixel position information of each target in the picture.

[0018] Further, the system further comprises: a radar-visual integrated machine installed in the opposite direction to the radar-visual integrated machine;

[0019] The reversely installed radar and visual integrated machine is used to measure the speed of the target and collect the target image when the target is driving towards the electric alarm pole from a distance;

[0020] The main control machine is also used to obtain the identification information, feature information and pixel position information of the target in the image based on the recognition of the target image, and to perform data fusion and intersection traffic management based on the radar data and the identification information, feature information and pixel position information of the target in the image.

[0021] Further, the camera group includes: a global exposure camera, a near-view camera, and a far-view camera;

[0022] The global exposure camera is used to collect images of the entire intersection area corresponding to the electric police pole or the traffic light pole;

[0023] The close-range camera is used to collect images of the entire intersection area corresponding to the electric police pole or the traffic light pole and the vehicle waiting area on the opposite road;

[0024] The long-range camera is used to collect images of the vehicle waiting area on the opposite road corresponding to the electric alarm pole or the traffic light pole and images of the road area extending to the far end;

[0025] The main control machine is specifically used to perform target recognition on the image of the entire intersection area corresponding to the electric police pole or the traffic light pole collected by the global exposure camera, and obtain the identification information, feature information and pixel position information of the target in the image;

[0026] The main control machine is further used to perform target recognition on the images of the entire intersection area and the vehicle waiting area of ​​the opposite road corresponding to the electric police pole or the traffic light pole collected by the close-range camera, and match the target detected by the global exposure camera through a preset image stitching and re-recognition algorithm according to the recognition result to establish track information, wherein the track information includes the identification information, feature information and real position information of the target, and the real position information of the target is obtained according to the pixel position information of the target in the image;

[0027] The main control machine is also specifically used to detect and identify distant targets based on the vehicle waiting area images of the opposite road corresponding to the electric warning pole or traffic light pole and the road area images extending to the far end collected by the long-range camera, and match the recognition results with the target recognition results corresponding to the close-range camera through a preset image stitching algorithm to update the track information.

[0028] Furthermore, the main control machine is specifically used to obtain the distortion coefficient of each camera according to the compensation parameters of each camera in the camera group, and obtain the distortion coefficient of each camera according to the compensation parameters of each camera in the multi-eye camera;

[0029] The main control machine is further used to obtain the perspective transformation coefficient of each camera by selecting four non-collinear pixel points in the picture collected by each camera and the corresponding physical coordinates in the world coordinate system;

[0030] The main control machine is further used to obtain the real position information of the target according to the distortion coefficient, the perspective transformation coefficient and the pixel position information of the target in the picture.

[0031] Furthermore, the main control machine is also used to perform point cloud segmentation and clustering on the data of each moving target from the stop line at the intersection to the end of the road collected by the radar sensor to obtain speed information and position information of each target.

[0032] Furthermore, the installation sight angle of the integrated radar and vision device is 3-15 degrees downward from the horizontal direction.

[0033] Furthermore, the angle between the center of the field of view of the global exposure camera and the close-range camera and the horizontal line is in the range of 10-50 degrees.

[0034] Furthermore, the angle between the center of the vision camera's field of view and the horizontal line ranges from about 5 to 45 degrees.

[0035] The present invention provides a holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera. Through the combination configuration mode of multiple cameras of a camera group in the radar-vision integrated machine, a configuration method for collecting near and far view images is adopted. The target in the collected camera picture changes from large to small, and target detection and recognition are performed on the near and far view images respectively, and then the target is subjected to multi-camera fusion processing, which can improve the accuracy of target detection and facilitate the tracking of the target; at the same time, when vehicles at the intersection are waiting in a queuing area, small cars are easily blocked by large passenger cars and buses in front, and by configuring a multi-eye camera, the image of the vehicle directly below can be clearly collected, effectively solving the problem of blocking when the radar-vision integrated machine collects data; and a radar sensor is used to obtain the speed information and position information of each moving target from the intersection stop line to the end of the road, and the speed information and position information are fused with the target data detected and recognized by the camera group and the multi-eye camera, which can further improve the accuracy of target recognition and tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a schematic diagram of the scene acquisition by the integrated radar and multi-camera provided by the present invention Figure 1 ;

[0037] Figure 2 It is a structural schematic diagram of a holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera provided by the present invention;

[0038] Figure 3 This is a schematic diagram of the scene acquisition by the integrated radar and multi-camera provided by the present invention Figure 2 ;

[0039] Figure 4 This is a schematic diagram of the scene acquisition by the integrated radar and multi-camera provided by the present invention Figure 3 ;

[0040] Figure 5 This is a schematic diagram of the scene acquisition by the integrated radar and multi-camera provided by the present invention Figure 4 . DETAILED DESCRIPTION

[0041] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments.

[0042] The embodiment of the present invention provides a holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera, such as Figure 2 As shown, the system includes: a radar-vision integrated machine 21, a main control machine 22, and a multi-eye camera 23; the radar-vision integrated machine 21 is configured on an electric alarm pole; the multi-eye camera 23 is configured below the radar-vision integrated machine 21 and connected to the radar-vision integrated machine. The system connection scenario can be as follows Figure 1 As shown; the multi-eye camera 23 is used to collect images of the traffic light waiting area and the intersection queuing area corresponding to the electric police pole.

[0043] Among them, the multi-eye camera is hoisted directly below the integrated laser vision machine. The number of lenses, lens focal length, and installation angle of the multi-eye camera are related to the height from the ground, the field of view to be collected, the width and number of lanes, etc. For example, for a narrower total lane width, a three-eye camera configuration can be used; for a wider total lane width, a five-eye camera configuration can be used. The focal length of the front and rear lenses of the multi-eye camera is generally 6-20mm, the focal length of the side-view lens is generally 4-8mm, and the angle between the center of the field of view and the horizontal line is 5-45 degrees; the downward-looking camera is generally 2-6mm, and the center of the field of view is vertically downward. During the actual installation process, it can be adjusted according to the specific road conditions.

[0044] The radar and vision integrated machine 21 includes a camera group and a radar sensor.

[0045] Among them, the integrated radar and vision machine is usually installed on an electric police pole or traffic light pole at a height of about 5-7m. The electric police pole is a pole where electronic police or electronic eyes are installed, which is generally opposite to the traffic light pole. The angle between the line of sight of the integrated radar and the horizontal plane is usually 3-15 degrees, at which the intersection area can be better observed; the mainstream frequency of the radar sensor is 77G-80G, which meets the distance and distance measurement of motor vehicles in a scene with 8 lanes in both directions and a longitudinal distance of 350m. The focal length of the lens of the global exposure camera, the long-range camera and the close-range camera is generally 6-20mm. The angle between the center of the field of view of the global exposure camera and the close-range camera and the horizontal line is about 10-50 degrees; the angle between the center of the field of view of the long-range camera and the horizontal line is about 5-45 degrees. In terms of azimuth, the camera group can meet the coverage of 8 lanes in both directions. In terms of longitudinal distance, the effective range of the global exposure camera is 20-200m from the installation pole; the effective range of the near-field camera is about 20-300m from the installation pole; the effective range of the far-field camera is about 100-350m from the installation pole. Multiple cameras are used for motor vehicle information recognition and near-field and far-field target detection, among which: the global exposure camera is responsible for collecting targets in the intersection area, 15m-50m away from the installation position, the near-field camera is responsible for collecting targets in the vehicle waiting area of ​​the intersection and the opposite road, 20m-150m away from the installation position, and the far-field camera mainly collects vehicles from the vehicle waiting area to the end of the road, 150m-250m away from the installation position, thereby realizing the detection, recognition and tracking of targets from near to far.

[0046] The camera group is used to collect images of the entire intersection area corresponding to the electric police pole, images of the vehicle waiting area on the opposite road, and images of the road area extending to the far end of the opposite road corresponding to the electric police pole.

[0047] The radar sensor is used to collect speed information, acceleration information, position information, RCS information and other information of each moving target from the stop line at the intersection to the end of the far road.

[0048] The main control computer 22 is used to perform cross-lens tracking, identification and fusion of images of the traffic light waiting area and the intersection queuing area corresponding to the electric alarm pole, and obtain the identification information, feature information and pixel position information of each target in the image; it is also used to perform target identification on images of different areas collected by the camera group, establish the track information of the target, and fuse the track information of each target identified and obtained by the camera group with the speed information and position information of each target identified and obtained according to the radar sensor. The track information includes the identification information, feature information and real position information of the target. The real position information of the target is obtained according to the pixel position information of the target in the image; it is also used to fuse the cross-lens tracking, identification and fusion data corresponding to the multi-eye camera with the fusion data corresponding to the radar vision integrated machine, and perform intersection traffic management based on the fused data.

[0049] Furthermore, the multi-eye camera includes: a front-view camera, a downward-view camera, and a rear-view camera; the front-view camera is in the same direction as the road and the angle between the center of the field of view and the horizontal line is in the range of 5-45 degrees, the rear-view camera is in the opposite direction of the road and the angle between the center of the field of view and the horizontal line is in the range of 5-45 degrees, and the center of the field of view of the downward-view camera is vertically downward, and is used to sequentially collect images of the traffic light waiting area and the intersection queuing area corresponding to the electric alarm pole.

[0050] Furthermore, when the images of the traffic light waiting area and the queuing area at the intersection corresponding to the electric police pole collected by the front-view camera, the downward-view camera and the rear-view camera are not sufficient to completely cover the areas on both sides of the road, the multi-eye camera also includes a left-view camera and a right-view camera; the left-view camera is perpendicular to the road to the left and the angle between the center of the field of view and the horizontal line is 5-45 degrees, and the right-view camera is perpendicular to the road to the right and the angle between the center of the field of view and the horizontal line is 5-45 degrees, which are used to collect images of the traffic light waiting area and the queuing area at the intersection corresponding to the electric police pole; the main control computer is also used to perform cross-lens tracking, recognition and fusion of the images of the traffic light waiting area and the queuing area at the intersection corresponding to the electric police pole collected by the front-view camera, the downward-view camera, the rear-view camera, the left-view camera and the right-view camera to obtain the identification information, feature information and pixel position information of each target in the picture.

[0051] For example, Figure 3 As shown in the figure, in a common intersection scenario, a three-eye camera and a radar-vision integrated machine are placed at the electric warning light pole. When a target enters the illumination range of the sensor, it is first captured by the rear-view camera of the three-eye camera, then by the downward-looking camera, and finally by the front-view camera and the camera module of the radar-vision integrated machine. The images taken by the three-eye camera and the camera in the radar-vision integrated machine are used for target detection and fused with the targets captured by the radar sensor to obtain accurate target motion information.

[0052] For example, Figure 4 As shown in the figure, in the T-junction scenario, since there is only one-way road in the T-junction, when the integrated radar and vision device is placed at the electric warning pole at the intersection, it can only illuminate the center of the intersection, which will cause a large performance waste. Therefore, for the T-junction, the multi-eye camera is placed at the electric warning light point to obtain the queue information of vehicles on the road, and the radar and vision device is placed at the traffic light point. For other directions, both the integrated radar and vision device and the multi-eye camera are placed at the electric warning pole point.

[0053] Furthermore, in order to better observe the vehicle information in both directions, the vehicles are managed globally, such as Figure 5 As shown, the system also includes: a radar and video integrated machine installed in the opposite direction to the radar and video integrated machine; the radar and video integrated machine installed in the opposite direction is used to perform radar speed measurement and target image acquisition on the target when the target drives towards the electric alarm pole from a distance; the main control machine is also used to obtain the identification information, feature information and pixel position information of the target in the image based on the recognition of the target image, and perform data fusion and intersection traffic management based on the radar data and the identification information, feature information and pixel position information of the target in the image.

[0054] Furthermore, the camera group includes: a global exposure camera, a close-range camera, and a long-range camera; the global exposure camera is used to capture images of the entire intersection area corresponding to the electric warning pole or the traffic light pole; the close-range camera is used to capture images of the entire intersection area corresponding to the electric warning pole or the traffic light pole and the vehicle waiting area on the opposite road; the long-range camera is used to capture images of the vehicle waiting area on the opposite road corresponding to the electric warning pole or the traffic light pole and the road area extending to the far end.

[0055] The main control computer 22 is specifically used to perform target recognition on the image of the entire intersection area corresponding to the electric police pole or the traffic light pole collected by the global exposure camera, and obtain the identification information, feature information and pixel position information of the target in the image; it is also specifically used to perform target recognition on the image of the entire intersection area and the vehicle waiting area of ​​the opposite road corresponding to the electric police pole or the traffic light pole collected by the close-range camera, and match the recognition result with the target detected by the global exposure camera through a preset image stitching and re-recognition algorithm to establish track information, wherein the track information includes the identification information, feature information and real position information of the target, and the real position information of the target is obtained according to the pixel position information of the target in the image; it is also specifically used to detect and recognize distant targets on the image of the vehicle waiting area of ​​the opposite road corresponding to the electric police pole or the traffic light pole and the image of the road area extending to the far end collected by the distant camera, and match the recognition result with the target recognition result corresponding to the close-range camera through a preset image stitching algorithm to update the track information.

[0056] Specifically, for example, first identify the target information of the picture captured by the global exposure camera. Such as license plate information, body color and model, etc.; then detect the target in the picture captured by the close-range camera and match it with the target detected by the global exposure camera. Track the target, assign the target ID and establish track information. Use image stitching and re-identification algorithms to match the vehicle ID detected by the close-range camera with the target detected by the global exposure camera, and assign relevant information such as the license plate, color and model of the corresponding ID vehicle. Finally, use the long-range camera to detect distant targets and match them with the targets detected by the close-range camera. Generally, the picture captured by the long-range camera is a local enlargement of the far area of ​​the picture captured by the close-range camera. Therefore, the target frame captured in the long-range camera can be transformed into the close-range camera using the image stitching method.

[0057] Furthermore, the main control computer 22 is specifically used to obtain the distortion coefficient of each camera according to the compensation parameters of each camera in the camera group, and obtain the distortion coefficient of each camera according to the compensation parameters of each camera in the multi-eye camera; obtain the perspective transformation coefficient of each camera by selecting four non-collinear pixel points in the picture collected by each camera and the corresponding physical coordinates in the world coordinate system; obtain the real position information of the target according to the distortion coefficient, the perspective transformation coefficient and the pixel position information of the target in the picture.

[0058] Specifically, for example, the camera distortion correction parameters can be solved by the Zhang Zhengyou calibration method or the compensation parameters given by the manufacturer. The perspective transformation coefficient is solved by selecting four non-collinear pixel points in the captured image and the corresponding physical coordinates in the world coordinate system. The real position of the target is obtained using the distortion coefficient, perspective transformation coefficient and detection pixels. In general, the detection algorithm based on deep learning is performed in the original image captured by the camera without distortion correction, and the pixel position of the target in the image is obtained. Therefore, after the distortion position compensation is completed using the correction coefficient, the perspective transformation coefficient is used to map the target pixel to the bird's-eye view.

[0059] Furthermore, the radar-vision integrated machine also includes: a configuration unit; the configuration unit is used to time-calibrate the image data collected by each camera and synchronize the timestamps corresponding to the image data collected by different cameras; the installation position of the radar sensor is spatially calibrated, and aligned with the lane information of the monitoring area according to the correction position error coefficient. It should be noted that due to the encoding and decoding of the video and the transmission delay, the timestamps of each camera in the camera group are generally not synchronized. First, the camera is timed, and after the collected picture is transmitted to the edge calculator, the timestamps of the collected data are time-aligned to ensure that the data to be processed by each camera at each moment is the same frame of data.

[0060] Furthermore, the main control computer 22 is also used to perform point cloud segmentation and clustering on the data of each moving target from the stop line at the intersection to the end of the road collected by the radar sensor to obtain speed information and position information of each target.

[0061] The embodiment of the present invention provides a holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera. Through the combination configuration mode of multiple cameras of a camera group in the radar-vision integrated machine, a configuration method of near- and far-view image acquisition is adopted. The target in the acquired camera picture changes from large to small, and target detection and recognition are performed on the near- and far-view images respectively, and then the target is subjected to multi-camera fusion processing, which can improve the accuracy of target detection and facilitate the tracking of the target; at the same time, when vehicles at the intersection are waiting in the queuing area, small cars are easily blocked by large passenger cars and buses in front, and the blockage problem when the radar-vision integrated machine collects data can be effectively solved by configuring a multi-eye camera; and a radar sensor is used to obtain the speed information and position information of each moving target from the stop line of the intersection to the end of the road, and the speed information and position information are fused with the target data detected and recognized by the camera group and the multi-eye camera, which can further improve the accuracy of target recognition and tracking.

[0062] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0063] The disclosed embodiments are described above to enable any person skilled in the art to implement or use the present invention. Various modifications of these embodiments are obvious to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0064] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including", just as "including," is explained as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".

[0065] Those skilled in the art may also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0066] The various illustrative logic blocks or units described in the embodiments of the present invention can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of the above. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0067] The steps of the method or algorithm described in the embodiments of the present invention can be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC, and the ASIC can be arranged in a user terminal. Optionally, the processor and the storage medium can also be arranged in different components in the user terminal.

[0068] In one or more exemplary designs, the above functions described in the embodiments of the present invention can be implemented in hardware, software, firmware or any combination of the three. If implemented in software, these functions can be stored on a computer-readable medium, or transmitted in the form of one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. The storage medium can be any available medium that can be accessed by any general or special computer. For example, such computer-readable media can include but are not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program codes in the form of instructions or data structures and other forms that can be read by general or special computers, or general or special processors. In addition, any connection can be appropriately defined as a computer-readable medium, for example, if the software is transmitted from a website site, server or other remote resource through a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disk and disc include compact disk, laser disk, optical disk, DVD, floppy disk and blue-ray disk. Disks usually copy data magnetically, while discs usually copy data optically with lasers. The above combination can also be included in computer readable media.

[0069] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera, characterized in that: The system comprises: a radar-vision integrated machine, a multi-eye camera, and a main control machine; the radar-vision integrated machine is arranged on an electric alarm pole; the multi-eye camera is arranged below the radar-vision integrated machine and connected to the radar-vision integrated machine; The multi-eye camera is used to collect images of the traffic light waiting area and the intersection queuing area corresponding to the electric police pole; The radar-vision integrated machine includes a camera group and a radar sensor; The camera group is used to collect images of the entire intersection area corresponding to the electric police pole, images of the vehicle waiting area on the opposite road, and images of the road area extending to the far end of the opposite road corresponding to the electric police pole; The radar sensor is used to collect information about various moving targets from the stop line at the intersection to the end of the far road; The main control machine is used to perform cross-lens tracking, recognition and fusion of images of the traffic light waiting area and the intersection queuing area corresponding to the electric police pole, and obtain identification information, feature information and pixel position information of each target in the image; The main control machine is further used to perform target recognition on the images of different areas collected by the camera group, establish target track information, and fuse the track information of each target obtained by the camera group with the speed information and position information of each target obtained by the radar sensor, wherein the track information includes the identification information, feature information and real position information of the target, and the real position information of the target is obtained according to the pixel position information of the target in the picture; The main control machine is also used to fuse the cross-lens tracking, identification and fusion data corresponding to the multi-camera with the fusion data corresponding to the integrated radar and vision machine, and perform intersection traffic management based on the fused data; The multi-camera includes: a front-view camera, a downward-view camera, and a rear-view camera; The front-view camera is in the same direction as the road and the angle between the center of the field of view and the horizontal line is 5-45 degrees. The rear-view camera is in the opposite direction of the road and the angle between the center of the field of view and the horizontal line is 5-45 degrees. The center of the field of view of the downward-looking camera is vertically downward, and is used to sequentially collect images of the traffic light waiting area and the intersection queuing area corresponding to the electric alarm pole; The camera group includes: a global exposure camera, a close-up camera, and a long-range camera; The global exposure camera is used to collect images of the entire intersection area corresponding to the electric police pole or the traffic light pole; The close-range camera is used to collect images of the entire intersection area corresponding to the electric police pole or the traffic light pole and the vehicle waiting area on the opposite road; The long-range camera is used to collect images of the vehicle waiting area on the opposite road corresponding to the electric alarm pole or the traffic light pole and images of the road area extending to the far end; The main control machine is specifically used to perform target recognition on the image of the entire intersection area corresponding to the electric police pole or the traffic light pole collected by the global exposure camera, and obtain the identification information, feature information and pixel position information of the target in the image; The main control machine is further used to perform target recognition on the images of the entire intersection area and the vehicle waiting area of ​​the opposite road corresponding to the electric police pole or the traffic light pole collected by the close-range camera, and match the target detected by the global exposure camera through a preset image stitching and re-recognition algorithm according to the recognition result to establish track information, wherein the track information includes the identification information, feature information and real position information of the target, and the real position information of the target is obtained according to the pixel position information of the target in the image; The main control machine is also specifically used to detect and identify distant targets based on the vehicle waiting area images of the opposite road corresponding to the electric warning pole or traffic light pole and the road area images extending to the far end collected by the long-range camera, and match the recognition results with the target recognition results corresponding to the close-range camera through a preset image stitching algorithm to update the track information.

2. According to claim 1, a holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera is characterized in that: When the road is wide, the images of the traffic light waiting area and the intersection queuing area corresponding to the electric police pole collected by the front-view camera, the downward-view camera, and the rear-view camera are not enough to completely cover the areas on both sides of the road, the multi-view camera also includes a left-view camera and a right-view camera; The left-side camera is perpendicular to the road and faces left, and the angle between the center of the field of view and the horizontal line is 5-45 degrees. The right-side camera is perpendicular to the road and faces right, and the angle between the center of the field of view and the horizontal line is 5-45 degrees. It is used to collect images on both sides of the traffic light waiting area and the intersection queuing area corresponding to the electric alarm pole; The main control machine is also used to perform cross-lens tracking, recognition and fusion of images of the traffic light waiting area and the intersection queuing area corresponding to the electric police pole collected by the front-view camera, the downward-view camera, the rear-view camera, the left-view camera and the right-view camera, so as to obtain the identification information, feature information and pixel position information of each target in the picture.

3. According to claim 1, a holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera is characterized in that: The system further includes: a radar-visual integrated machine installed in the opposite direction to the radar-visual integrated machine; The reversely installed radar and visual integrated machine is used to measure the speed of the target and collect the target image when the target is driving towards the electric alarm pole from a distance; The main control machine is also used to identify the target image to obtain the target's identification information, feature information and pixel position information of the target in the image, and perform data fusion and intersection traffic management based on the radar data and the target's identification information, feature information and pixel position information of the target in the image.

4. A holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera according to any one of claims 1 to 3, characterized in that: The main control machine is specifically used to obtain the distortion coefficient of each camera according to the compensation parameters of each camera in the camera group, and obtain the distortion coefficient of each camera according to the compensation parameters of each camera in the multi-eye camera; The main control machine is further used to obtain the perspective transformation coefficient of each camera by selecting four non-collinear pixel points in the picture collected by each camera and the corresponding physical coordinates in the world coordinate system; The main control machine is further used to obtain the real position information of the target according to the distortion coefficient, the perspective transformation coefficient and the pixel position information of the target in the picture.

5. According to claim 1, a holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera is characterized in that: The main control machine is also specifically used to perform point cloud segmentation and clustering on the data of each moving target from the stop line at the intersection to the end of the road collected by the radar sensor, so as to obtain the speed information and position information of each target.

6. A holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera according to any one of claims 1 to 3, characterized in that: The installation angle of the integrated radar and vision machine is 3-15 degrees downward from the horizontal direction.

7. A holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera according to any one of claims 1 to 3, characterized in that: The angle between the center of the field of view of the global exposure camera and the close-range camera and the horizontal line is in the range of 10-50 degrees.

8. A holographic intersection traffic management system based on the combination of a radar-vision integrated machine and a multi-eye camera according to any one of claims 1 to 3, characterized in that: The angle between the center of the vision camera's field of view and the horizontal line ranges from 5 to 45 degrees.

Images