Holographic intersection fusion method and system for multi-view geometry suitable for traffic scenarios

Abstract

The embodiment of the application provides a holographic intersection fusion method and system suitable for multi-view geometry of a traffic scene, the fusion method comprising: step S10, acquiring images of different directions of a road surface by a plurality of cameras respectively; step S20, performing vehicle key point detection and lane line detection in each frame of image, and outputting positions of vehicles and lane lines in a camera coordinate system in each frame of image; step S30, mapping the positions of the vehicles and the lane lines in the camera coordinate system in each frame of image into a top view coordinate system of a high-definition map respectively; step S40, calculating the similarity of a coordinate sequence based on the coordinates of the vehicles in different frames of different cameras under the high-definition map, and determining frame offset; and step S50, constructing a fused top view of the traffic scene under the plurality of cameras based on the top view coordinates of the vehicles and the lane lines under each camera and the frame offset; the method has the beneficial effects of simple deployment and low maintenance cost, and is suitable for the technical field of computer vision and machine learning.

Description

Technical Field

[0001] This application relates to the technical fields of computer vision and machine learning, and more specifically, to a holographic intersection fusion method and system for multi-view geometry applicable to traffic scenarios. Background Technology

[0002] In the field of urban road traffic, when faced with complex intersections, the usual method is to use front-end cameras to capture images and videos, and then process the data in the cloud and send it back to the control center. This process is lengthy. At the same time, traffic accidents are still mainly reported manually, which can easily lead to "minor accidents causing major congestion".

[0003] With the diversified development of detection technologies, making full use of existing detection methods and collaborative detection technologies that leverage the advantages of multi-sensor resources and information to improve urban traffic capacity, especially road perception and detection at intersections, is an important development direction for intelligent transportation.

[0004] Holographic intersections, as a development direction of intelligent transportation, adopt multi-directional radar-visual fitting technology and combine high-precision maps to present a holographic view of the intersection. Through automatic perception of traffic accidents and events, accurate data assists in determining responsibility, which can reduce the occurrence of accidents, provide accurate lane-level traffic flow data, and support adaptive timing of intersection signal control.

[0005] Among them, the radar-visual fitting technology can perform structured analysis of data from connected radar, video and other devices, and present a digital perspective by combining high-precision maps, so as to accurately depict the movement of motor vehicles, non-motor vehicles and pedestrians.

[0006] However, the technologies used in the front-end equipment of holographic intersections mainly rely on ranging sensors such as millimeter-wave radar and lidar, which result in high equipment costs. Furthermore, the joint calibration process of these sensors and ordinary industrial cameras in the front-end equipment is cumbersome and has poor robustness. These reasons lead to the high cost of holographic intersection technology, which is not conducive to the application and popularization of this technology. Summary of the Invention

[0007] This application provides a holographic intersection fusion method and system with multi-view geometry suitable for traffic scenarios, which is simple to deploy and has low maintenance costs.

[0008] According to a first aspect of the embodiments of this application, a holographic intersection fusion method for multi-view geometry applicable to traffic scenarios is provided, including:

[0009] Step S10: Acquire images of the road surface from different directions using multiple cameras;

[0010] Step S20: Perform vehicle key point detection and lane line detection in each frame image, and output the position of the vehicle and lane line in the camera coordinate system in each frame image;

[0011] Step S30: Map the positions of vehicles and lane lines in each frame of the image in the camera coordinate system to the top-view coordinate system of the high-precision map.

[0012] Step S40: Based on the coordinates of vehicles in different frames from different cameras under the high-precision map, calculate the similarity of the coordinate sequences and determine the frame offset;

[0013] Step S50: Based on the top-view coordinates of vehicles and lane lines under each camera and the frame offset, construct a fused top-view view of the traffic scene under multiple cameras.

[0014] Preferably, step S20 involves detecting vehicle key points and lane lines in each image, and outputting the positions of the vehicle and lane lines in the camera coordinate system in each image; specifically including:

[0015] Step S201: Construct the geometric constraint relationship between the vehicle chassis and the wheels;

[0016] Step S202: Detect vehicle key points in each frame of the image using a deep network model and geometric constraints, and output the position of the vehicle key points in each frame of the image in the camera coordinate system.

[0017] Step S203: Lane lines in each frame of the image are detected using a deep network model;

[0018] Step S204: Output the position of the lane line in the camera coordinate system in each frame image.

[0019] Preferably, step S202 involves detecting vehicle key points in each frame of the image using a deep network model and geometric constraints, and outputting the positions of the vehicle key points in each frame of the image in the camera coordinate system; specifically including:

[0020] Step S2021: Input each frame image into the YOLOV5 model and output the vehicle detection image corresponding to each frame image.

[0021] Step S2022: Input each vehicle detection image into the HRNet model and output the vehicle key points corresponding to each vehicle detection image.

[0022] Step S2023: Based on the vehicle key points and geometric constraints, adjust the position of the vehicle key points in the camera coordinate system.

[0023] Preferably, step S203 involves detecting lane lines in each frame of the image using a deep network model. The lane line detection for each frame specifically includes:

[0024] Step S2031: Extract lane line features by segmenting the lane lines;

[0025] Step S2032: By clustering instances, the discrete lane line coordinate information in the lane line features is connected to output a feature point set representing lane instances.

[0026] Step S2033: Perform curve fitting on the feature point set of lane line instances through lane line fitting to output smooth lane lines.

[0027] Preferably, the vehicle has eight key points, namely: four key points for identifying the vehicle chassis and four key points for identifying the four wheels.

[0028] According to a second aspect of the embodiments of this application, a holographic intersection fusion system with multi-view geometry suitable for traffic scenarios is provided, including:

[0029] The image acquisition module is used to acquire images of the road surface from different directions using multiple cameras;

[0030] The vehicle and lane detection module is used to detect vehicle key points and lane lines in each frame of the image, and outputs the position of the vehicle and lane lines in the camera coordinate system in each frame of the image.

[0031] The coordinate mapping module is used to map the positions of vehicles and lane lines in each frame of the image from the camera coordinate system to the top-view coordinate system of the high-precision map.

[0032] The time synchronization module is used to calculate the similarity of coordinate sequences and determine frame offset based on the coordinates of vehicles in different frames from different cameras on a high-precision map.

[0033] The fusion module is used to construct a fused top-down view of the traffic scene from multiple cameras based on the top-down coordinates of vehicles and lane lines from each camera and the frame offset.

[0034] Preferably, the vehicle and lane detection module includes:

[0035] The building block is used to construct the geometric constraints between the vehicle chassis and wheels;

[0036] The vehicle detection module is used to detect vehicle key points in each frame of the image through a deep network model and geometric constraints, and outputs the position of the vehicle key points in the camera coordinate system in each frame of the image.

[0037] The lane detection module is used to detect lane lines in each frame of the image using a deep network model and output the position of the lane lines in the camera coordinate system in each frame of the image.

[0038] Preferably, in the vehicle detection module, the detection of vehicle key points in each frame of image includes:

[0039] The first detection module is used to input each frame of image into the YOLOV5 model and output the vehicle detection image corresponding to each frame of image.

[0040] The second detection module is used to input the detection images of each vehicle into the HRNet model and output the key points of each vehicle detection image.

[0041] The key point position adjustment and output module is used to adjust the position of the vehicle key points in the camera coordinate system based on the vehicle key points and geometric constraints.

[0042] Preferably, in the lane detection module, lane line detection for each frame of image includes:

[0043] The lane line segmentation module is used to extract lane line features by segmenting lane lines;

[0044] The lane line instance clustering module is used to connect the discrete lane line coordinate information in the lane line features through instance clustering, and output a feature point set representing the lane instance;

[0045] The lane line fitting module performs curve fitting on the feature point set of lane line instances to output smooth lane lines.

[0046] The embodiments of this application, by adopting the above technical solutions, have the following technical effects:

[0047] In this invention, road feature point recognition is achieved through vehicle key point detection and lane line detection, thus realizing the structuring of traffic information. At the same time, the frame offset is determined by calculating the similarity of coordinate sequences, and a fused top-view view of the traffic scene under multiple cameras is constructed based on the top-view coordinates of vehicles and lane lines under each camera and the frame offset. This invention has the advantages of full-process automation, strong stability, simple deployment, and low maintenance cost, making it highly practical. Attached Figure Description

[0048] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0049] Figure 1 This is a flowchart illustrating the holographic intersection fusion method for multi-view geometry applicable to traffic scenarios, as described in this application embodiment.

[0050] Figure 2 This is a flowchart illustrating step S20 of the holographic intersection fusion method for multi-view geometry applicable to traffic scenarios in this application embodiment.

[0051] Figure 3This is a flowchart illustrating step S202 in the holographic intersection fusion method for multi-view geometry applicable to traffic scenarios in this application embodiment;

[0052] Figure 4 This is a flowchart illustrating step S203 in the holographic intersection fusion method for multi-view geometry applicable to traffic scenarios in this application embodiment;

[0053] Figure 5 This is a schematic diagram of the structure of a holographic intersection fusion system with multi-view geometry applicable to traffic scenarios, as described in this application embodiment.

[0054] Figure label:

[0055] 10 is the image acquisition module, 20 is the vehicle and lane detection module, 30 is the coordinate mapping module, 40 is the time synchronization module, and 50 is the fusion module;

[0056] 201 is the construction module, 202 is the vehicle detection module, and 203 is the lane detection module;

[0057] 2021 is the first detection module, 2022 is the second detection module, and 2023 is the key point position adjustment and output module;

[0058] 2031 is the lane line segmentation module, 2032 is the lane line instance clustering module, and 2033 is the lane line fitting module. Detailed Implementation

[0059] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0060] Example 1

[0061] like Figure 1 As shown, the embodiments of this application apply to a holographic intersection fusion method based on multi-view geometry in traffic scenarios, including:

[0062] Step S10: Acquire images of the road surface from different directions using multiple cameras;

[0063] Step S20: Perform vehicle key point detection and lane line detection in each frame image, and output the position of the vehicle and lane line in the camera coordinate system in each frame image;

[0064] Step S30: Map the positions of vehicles and lane lines in each frame of the image in the camera coordinate system to the top-view coordinate system of the high-precision map.

[0065] Step S40: Based on the coordinates of vehicles in different frames from different cameras under the high-precision map, calculate the similarity of the coordinate sequences and determine the frame offset;

[0066] Step S50: Based on the top-view coordinates of vehicles and lane lines under each camera and the frame offset, construct a fused top-view view of the traffic scene under multiple cameras.

[0067] In this embodiment, vehicle key point detection and lane line detection are performed in each frame of the image to structure the traffic information in each frame and output the positions of vehicles and lane lines in the camera coordinate system in each frame. Then, the obtained road sign key points are matched with road sign key points in the high-precision map. Through the transformation matrix, the coordinates of vehicles in different frames of different cameras under the high-precision map are obtained. Then, the similarity of these coordinate sequences is calculated to determine the frame offset. Finally, the single-lens tracking of different cameras is fused to construct a fused top-down view of the traffic scene under multiple cameras.

[0068] Specifically, after step S30, the vehicle positions in top-down coordinates of multiple frames are generated. The purpose of step S40 is to use time synchronization technology to find the frame offset with the minimum cost within a certain range. The key to time synchronization is the cost function and the search strategy. In this embodiment, the cost function can be the Euclidean distance in the same coordinate system, and the search strategy can be dtw.

[0069] Furthermore, since the images from different cameras have overlapping areas, the same vehicle may be captured by multiple cameras. In this case, projecting the vehicle from different cameras onto the top view will result in a single vehicle having multiple coordinate points. To solve this problem, this application uses tracking and legal region segmentation after fusing multiple camera images. That is, only vehicles appearing in the legal region in a camera will be projected onto the top view. To determine the same vehicle, vehicle tracking needs to be added. Vehicle tracking and legal region segmentation jointly determine which vehicles a camera will project onto the top view, thereby avoiding the problem of a single vehicle having multiple coordinates in the top view.

[0070] This application presents a holographic intersection fusion method for multi-view geometry applicable to traffic scenarios. It identifies road feature points through vehicle key point detection and lane line detection, thus structuring traffic information. Furthermore, this application employs multi-camera automatic joint calibration and multi-camera timestamp synchronization technologies. By calculating the similarity of coordinate sequences, it determines frame offsets and constructs a fused top-view view of the traffic scene under multiple cameras based on the vehicle and lane line top-view coordinates and frame offsets from each camera. This invention boasts advantages such as full-process automation, high stability, simple deployment, and low maintenance costs, making it highly practical.

[0071] Example 2

[0072] like Figure 2 As shown, based on Embodiment 1, a holographic intersection fusion method for multi-view geometry applicable to traffic scenarios is described. Step S20 involves detecting vehicle key points and lane lines in each image, and outputting the positions of vehicles and lane lines in the camera coordinate system for each image. Specifically, this includes:

[0073] Step S201: Construct the geometric constraint relationship between the vehicle chassis and the wheels;

[0074] Step S202: Detect vehicle key points in each frame of the image using a deep network model and geometric constraints, and output the position of the vehicle key points in each frame of the image in the camera coordinate system.

[0075] Step S203: Lane lines in each frame of the image are detected using a deep network model;

[0076] Step S204: Output the position of the lane line in the camera coordinate system in each frame image.

[0077] In this embodiment, vehicle recognition and lane line key point detection mainly use deep learning technology to structure the traffic information of each frame of image (which can be RGB image) and output the position of vehicles and road signs in the camera coordinate system.

[0078] Specifically, such as Figure 3 As shown, step S202 involves detecting vehicle key points in each frame of the image using a deep network model and geometric constraints, and outputting the positions of the vehicle key points in each frame of the image within the camera coordinate system; specifically, it includes:

[0079] Step S2021: Input each frame image into the YOLOv5 model and output the vehicle detection image corresponding to each frame image.

[0080] Step S2022: Input each vehicle detection image into the HRNet model and output the vehicle key points corresponding to each vehicle detection image.

[0081] Step S2023: Based on the vehicle key points and geometric constraints, adjust the position of the vehicle key points in the camera coordinate system.

[0082] Specifically, there are 8 key points for the vehicle: 4 key points for identifying the vehicle chassis and 4 key points for identifying the wheels.

[0083] In this embodiment, the core of vehicle key point detection is to find the center of the vehicle's projection on the road surface. Compared with mainstream 3D detection technologies that require camera intrinsic and extrinsic parameters or point cloud information, the vehicle center detection method used in this application only uses RGB images and mainly consists of two steps:

[0084] First, use a deep learning model to detect key points of the vehicle in the camera coordinate system;

[0085] In this embodiment, the HRNet model can be used. The HRNet model is a top-down key point detection model. First, the entire image is used as the input for vehicle detection, and the vehicle detection result is output. Second, the output of a single vehicle is used as the input for the HRNet model, and the key point result of the vehicle is output. The vehicle key points are defined as an 8-point model, including 4 points of the chassis and 4 points of the wheels.

[0086] Second, by utilizing geometric constraints such as the vehicle's wheelbase, the projection center of the vehicle on the road surface is optimized.

[0087] Because there are errors in the marking of the contact points between the wheel and the ground, and the wheel may have occluded points, invisible points in the key point detection usually have large errors.

[0088] In this application, the quadrilateral formed by the four wheels has a geometric constraint relationship, and the lines connecting the four points of the chassis and the wheels are parallel and intersect at a point in the image coordinate system. These geometric constraints can assist in the detection of key points of the wheels and improve the accuracy of key point detection.

[0089] Furthermore, such as Figure 4 As shown, step S203 involves detecting lane lines in each frame of the image using a deep network model. The lane line detection for each frame specifically includes:

[0090] Step S2031: Extract lane line features by segmenting the lane lines;

[0091] Step S2032: By clustering instances, the discrete lane line coordinate information in the lane line features is connected to output a feature point set representing lane instances.

[0092] Step S2033: Perform curve fitting on the feature point set of lane line instances through lane line fitting to output smooth lane lines.

[0093] In this embodiment, deep learning is used to extract road surface location information such as road signs. The lane line detection technology is mainly divided into two parts: deep network and post-processing. The deep network model converts the RGB image into a lane line vector field feature map, a lane line segmentation feature map, and a lane line category feature map. Post-processing is to fuse the information from the three feature maps to generate lane lines.

[0094] The lane line detection algorithm mainly includes the following steps: lane line segmentation, lane line instance clustering, and lane line fitting. Among them, the lane line segmentation module is a binary classification module, mainly used to distinguish whether a point is a lane line or not. Lane line instance clustering is implemented through post-processing, predicting vectors of point connection relationships in the horizontal and vertical directions, and connecting the vectors output by the model to form different lane instances. Lane line fitting uses a multinomial to perform curve fitting on the segmented lane line instance point set, which can filter out outliers and make the lane lines smoother.

[0095] A second aspect of this application also provides a holographic intersection fusion system with multi-view geometry suitable for traffic scenarios.

[0096] like Figure 5 As shown, a holographic intersection fusion system with multi-view geometry suitable for traffic scenarios includes:

[0097] Image acquisition module 10 is used to acquire images of the road surface from different directions using multiple cameras;

[0098] The vehicle and lane detection module 20 is used to detect vehicle key points and lane lines in each frame of the image, and output the position of the vehicle and lane lines in the camera coordinate system in each frame of the image.

[0099] The coordinate mapping module 30 is used to map the positions of vehicles and lane lines in each frame of the image to the top-view coordinate system of the high-precision map.

[0100] The time synchronization module 40 is used to calculate the similarity of coordinate sequences and determine frame offset based on the coordinates of vehicles in different frames from different cameras on a high-precision map.

[0101] The fusion module 50 is used to construct a fused top-down view of the traffic scene under multiple cameras based on the top-down coordinates of vehicles and lane lines under each camera and the frame offset.

[0102] Specifically, the vehicle and lane detection module 20 includes:

[0103] Module 201 is used to construct the geometric constraint relationship between the vehicle chassis and the wheels;

[0104] The vehicle detection module 202 is used to detect vehicle key points in each frame of the image through a deep network model and geometric constraints, and output the position of the vehicle key points in each frame of the image in the camera coordinate system.

[0105] The lane detection module 203 is used to detect lane lines in each frame of the image through a deep network model and output the position of the lane lines in the camera coordinate system in each frame of the image.

[0106] Furthermore, in the vehicle detection module 202, the detection of vehicle key points in each frame of image includes:

[0107] The first detection module 2021 is used to input each frame of image into the Yolov5 model and output the vehicle detection image corresponding to each frame of image.

[0108] The second detection module 2022 is used to input each vehicle detection image into the HRNet model and output the vehicle key points corresponding to each vehicle detection image.

[0109] The Key Point Position Adjustment and Output Module 2023 is used to adjust the position of vehicle key points in the camera coordinate system based on vehicle key points and geometric constraints.

[0110] Furthermore, in the lane detection module 203, lane line detection for each frame of image includes:

[0111] Lane segmentation module 2031 is used to extract lane features by segmenting lane lines;

[0112] Lane line instance clustering module 2032 is used to connect discrete lane line coordinate information in lane line features through instance clustering, and output a feature point set representing lane instances;

[0113] The lane line fitting module 2033 performs curve fitting on the feature point set of lane line instances and outputs smooth lane lines.

[0114] A third aspect of this application also provides a holographic intersection fusion device with multi-view geometry suitable for traffic scenarios.

[0115] A holographic intersection fusion device with multi-view geometry suitable for traffic scenarios includes:

[0116] Memory, used to store computer programs;

[0117] A controller is used to implement the holographic intersection fusion method for multi-view geometry applicable to traffic scenarios as described above when executing the computer program.

[0118] The memory may be a computer-readable storage medium, including ROM, RAM, disk, or optical disk, etc.

[0119] The device can be any device capable of replacing advertisements. The device can be various terminal devices, such as desktop computers, laptops, etc., and can be implemented through software and / or hardware.

[0120] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0122] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0123] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0124] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0125] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A holographic intersection fusion method based on multi-view geometry suitable for traffic scenarios, characterized by: include: Step S10: Acquire images of the road surface from different directions using multiple cameras; Step S20: Perform vehicle key point detection and lane line detection in each frame image, and output the position of the vehicle and lane line in the camera coordinate system in each frame image; Step S30: Map the positions of vehicles and lane lines in each frame of the image in the camera coordinate system to the top-view coordinate system of the high-precision map. Step S40: Based on the coordinates of vehicles in different frames from different cameras under the high-precision map, calculate the similarity of the coordinate sequences and determine the frame offset; Step S50: Based on the top-view coordinates of vehicles and lane lines under each camera and the frame offset, construct a fused top-view view of the traffic scene under multiple cameras; Step S20 involves detecting vehicle key points and lane lines in each image, and outputting the positions of the vehicle and lane lines in the camera coordinate system in each image; specifically including: Step S201: Construct the geometric constraint relationship between the vehicle chassis and the wheels; Step S202: Detect vehicle key points in each frame of the image using a deep network model and geometric constraints, and output the position of the vehicle key points in each frame of the image in the camera coordinate system. Step S203: Lane lines in each frame of the image are detected using a deep network model; Step S204: Output the position of the lane lines in the camera coordinate system in each frame image; Step S202 involves detecting vehicle key points in each frame of the image using a deep network model and geometric constraints, and outputting the positions of the vehicle key points in each frame of the image in the camera coordinate system; specifically including: Step S2021: Input each frame image into the YOLOv5 model and output the vehicle detection image corresponding to each frame image. Step S2022: Input each vehicle detection image into the HRNet model and output the vehicle key points corresponding to each vehicle detection image. Step S2023: Based on the vehicle key points and geometric constraints, adjust the position of the vehicle key points in the camera coordinate system; Step S203 involves detecting lane lines in each frame of the image using a deep network model. The lane line detection for each frame specifically includes: Step S2031: Extract lane line features by segmenting the lane lines; Step S2032: By clustering instances, the discrete lane line coordinate information in the lane line features is connected to output a feature point set representing lane instances. Step S2033: Perform curve fitting on the feature point set of lane line instances through lane line fitting to output smooth lane lines.

2. The holographic intersection fusion method for multi-view geometry applicable to traffic scenarios according to claim 1, characterized in that: The vehicle has eight key points: four key points for identifying the vehicle chassis and four key points for identifying the four wheels.

3. A holographic intersection fusion system with multi-view geometry suitable for traffic scenarios, characterized in that: include: The image acquisition module (10) is used to acquire images of the road surface from different directions using multiple cameras; The vehicle and lane detection module (20) is used to detect vehicle key points and lane lines in each frame of the image and output the position of the vehicle and lane lines in the camera coordinate system in each frame of the image. The coordinate mapping module (30) is used to map the positions of vehicles and lane lines in the camera coordinate system to the top-view coordinate system of the high-precision map in each frame image. The time synchronization module (40) is used to calculate the similarity of coordinate sequences and determine the frame offset based on the coordinates of vehicles in different frames from different cameras under the high-precision map. The fusion module (50) is used to construct a fused top-view view of the traffic scene under multiple cameras based on the top-view coordinates of vehicles and lane lines under each camera and the frame offset. The vehicle and lane detection module (20) includes: Module (201) is used to construct the geometric constraints between the vehicle chassis and the wheels; The vehicle detection module (202) is used to detect vehicle key points in each frame image through a deep network model and geometric constraints, and output the position of vehicle key points in each frame image in the camera coordinate system. The lane detection module (203) is used to detect lane lines in each frame of the image through a deep network model and output the position of the lane lines in the camera coordinate system in each frame of the image. The vehicle detection module (202) performs vehicle key point detection in each frame of image, including: The first detection module (2021) is used to input each frame of image into the YOLOV5 model and output the vehicle detection image corresponding to each frame of image. The second detection module (2022) is used to input each vehicle detection image into the HRNet model and output the vehicle key points corresponding to each vehicle detection image. Key point position adjustment and output module (2023), used to adjust the position of vehicle key points in the camera coordinate system based on vehicle key points and geometric constraints; In the lane detection module (203), lane line detection for each frame of image includes: The lane line segmentation module (2031) is used to extract lane line features by segmenting lane lines; The lane line instance clustering module (2032) is used to connect the discrete lane line coordinate information in the lane line features through instance clustering, and output the feature point set representing the lane instance; The lane line fitting module (2033) performs curve fitting on the feature point set of lane line instances and outputs smooth lane lines.

Images