Vehicle-road cooperation roadside signal processing device and method

Abstract

The invention provides a vehicle-road cooperation roadside signal processing device. The vehicle-road cooperation roadside signal processing device comprises a high-definition camera, a laser radar, a millimeter-wave radar and a signal lamp collector which are arranged on the roadside. The RSU module is connected with the intelligent network connection vehicle and the edge computing node and is used for transmitting the vehicle position, the driving intention, the early warning information and the control instruction to the intelligent network connection vehicle and the edge computing node; the edge computing node comprises a data management module, a fusion sensing module, an application function module and a system management module, and the system management module performs monitoring, log level-to-level management, crash restart strategy and OTA updating functions on equipment states. The invention also provides a vehicle-road cooperation roadside signal processing method, which comprises the steps of designing parameter configuration, collecting data, generating traffic participant data based on a multi-sensor data fusion algorithm, and carrying out ROI partitioning and marking to generate an event detection analysis result and a traffic flow analysis result.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic management technology, and more specifically, to a vehicle-road cooperative roadside signal processing device and method. Background Technology

[0002] With the rapid development of intelligent transportation systems, vehicle-road cooperative technology has become a core means to improve road safety and traffic efficiency. Traditional vehicle-road cooperative systems mostly rely on single sensors (such as cameras or radar) to achieve local environmental perception, which has problems such as blind spots, insufficient data fusion capabilities, and limited real-time performance. In addition, the high-latency communication between roadside equipment and the cloud cannot meet the real-time decision-making and control requirements of intelligent connected vehicles, and there is a lack of unified standards for the collaborative deployment and standardized data processing of multiple types of sensors (such as LiDAR, millimeter-wave radar, and video equipment). The computing power allocation, equipment deployment strategies, and communication network architecture of existing edge computing nodes also have efficiency bottlenecks and are difficult to adapt to the dynamic needs of complex traffic scenarios. Summary of the Invention

[0003] This invention provides a vehicle-road cooperative roadside signal processing device and method, aiming to solve the problems of existing vehicle-road cooperative systems relying on a single sensor (such as a camera or radar) to achieve local environmental perception, which have problems such as blind spots, insufficient data fusion capabilities, and limited real-time performance, and improve the accuracy and reliability of vehicle-road cooperative roadside signal processing.

[0004] To achieve the above objectives, the present invention provides a vehicle-road cooperative roadside signal processing device, comprising:

[0005] The sensing equipment group includes high-definition cameras, lidar, millimeter-wave radar, and traffic light collectors deployed on the roadside to collect video streams, radar point cloud data, structured traffic data, and traffic light status data.

[0006] The communication equipment group includes an RSU module, which is connected to the intelligent connected vehicle and the edge computing node, and transmits vehicle location, driving intention, warning information and control commands to the intelligent connected vehicle and the edge computing node.

[0007] Edge computing nodes include data management modules, fusion sensing modules, application function modules, and system management modules.

[0008] The data management module receives video streams, radar point cloud data, structured traffic data, and traffic light status data transmitted by the sensing device group, and processes the video streams, radar point cloud data, traffic light status data, and structured traffic data to achieve data interaction.

[0009] The fusion perception module generates traffic participant data based on the video stream, radar point cloud data, structured traffic data, and traffic light status data processed by the data management module, and on a multi-sensor data fusion algorithm. The traffic participant data includes traffic participants and their trajectories.

[0010] The application function module performs ROI partitioning and labeling on the traffic participant data generated by the fusion perception module, and generates event detection analysis results and traffic flow analysis results based on the traffic participant data and the set event judgment conditions.

[0011] The system management module monitors device status, manages log hierarchy, implements crash restart strategies, and provides OTA update functionality.

[0012] In one embodiment, the high-definition camera is installed on the boom of a composite pole 20-30 meters away from the stop line, with a detection range covering the entrance and exit lanes of the intersection. The installation height is 6 meters, and the lens is focused and adjusted to the blind zone of 15-20 meters under the pole.

[0013] The lidar includes a solid-state lidar and a blind spot radar. The blind spot radar covers an area with a radius of 30 meters, and the radar point cloud data frame rate is ≥10Hz.

[0014] The millimeter-wave radar is vertically mounted and outputs target type, velocity, and structured data at a frame rate of ≥10Hz.

[0015] In one embodiment, generating event detection and analysis results based on traffic participant data and set event determination conditions specifically includes:

[0016] Conditions for determining an intersection congestion event: When the queue of vehicles is no less than 3 vehicles during two red lights in a lane phase, and the exit lane overflows, and the average speed of vehicles on the road segment is less than 20% of the speed limit, the event detection and analysis result is a current road congestion event;

[0017] Conditions for determining a pedestrian or non-motorized vehicle intrusion event: When it is determined that a pedestrian or non-motorized vehicle has entered the intersection or motor vehicle lane and the dwell time is not less than 3 seconds, the event detection and analysis result is a pedestrian or non-motorized vehicle intrusion event.

[0018] Conditions for determining a solid line crossing event: When it is determined that a vehicle has been driving over the solid line for no less than 3 seconds or has changed lanes by crossing the solid line, the event detection and analysis result is a solid line crossing event.

[0019] Conditions for identifying abnormal parking events: When it is determined that parking in a no-parking zone for no less than 30 seconds constitutes an abnormal parking event;

[0020] Conditions for judging abnormal low-speed events: When the number of vehicles judged is no more than 3 and the vehicle speed is less than 40% of the current road speed limit for no less than 10 seconds, or when the number of vehicles judged is greater than 3 and the vehicle speed is less than 40% of the current traffic flow average speed for no less than 10 seconds, the event detection and analysis result is an abnormal low-speed event.

[0021] Conditions for determining a wrong-way driving event: When the vehicle trajectory deviates from the lane direction by more than 90° and lasts for no less than 3 seconds, the event detection and analysis result is a wrong-way driving event;

[0022] Conditions for determining a speeding event: When the vehicle speed exceeds the current road speed limit by more than 10% and lasts for no less than 3 seconds, the event detection and analysis result is a speeding event.

[0023] In one embodiment, the traffic flow analysis results specifically include:

[0024] The number of vehicles passing through a designated section of a road per unit of time;

[0025] The distance a vehicle travels in a unit of time;

[0026] The number of vehicles per unit length of road;

[0027] The difference between the actual travel time required for a vehicle to pass through an intersection under obstructed conditions and the time required to travel the same distance normally;

[0028] The length of a vehicle queue from the stop line at the intersection or the start of the queue to the end of the queue;

[0029] The vehicle's current traffic situation;

[0030] Based on the average delay of vehicles passing through the intersection, traffic conditions are divided into four levels: severe congestion, moderate congestion, light congestion, and smooth traffic, to evaluate the current traffic situation of vehicles.

[0031] A vehicle-road cooperative roadside signal processing method, implemented by the aforementioned vehicle-road cooperative roadside signal processing device, includes:

[0032] Step S101: Design parameter configuration, and install the corresponding sensing device according to the parameter configuration;

[0033] Step S102: Collect video streams, radar point cloud data, structured traffic data and traffic light status data detected by the sensing device, and perform data interaction on the video streams, radar point cloud data, structured traffic data and traffic light status data.

[0034] Step S103: Generate traffic participant data based on a multi-sensor data fusion algorithm. The traffic participant data includes traffic participants and their trajectories.

[0035] Step S104: Perform ROI partitioning and labeling on the traffic participant data, and generate event detection analysis results and traffic flow analysis results based on the traffic participant data and event judgment conditions;

[0036] Among them, based on edge computing algorithms, parameter configuration, sensing devices, ROI labeling, and fusion algorithms are iteratively optimized.

[0037] In one embodiment, iterative optimization of parameter configuration, sensing devices, ROI labeling, and fusion algorithms based on edge computing algorithms specifically includes:

[0038] Step S201, parameter configuration and verification: Configure relevant parameters in the edge computing node, such as network connection parameters, routing table settings, firewall settings, time settings, device information and deployment settings, and complete the verification.

[0039] Step S202: Install and debug the equipment, and confirm whether the required roadside equipment is complete. The roadside equipment includes high-definition cameras, lidar, millimeter-wave radar, traffic light collectors, brackets, and cables.

[0040] Step S203: Calibrate the high-definition camera, millimeter-wave radar, and lidar. The calibration of the high-definition camera includes the calibration of the internal parameters of the collaborative intelligent high-definition camera, the calibration of the external parameters of the collaborative intelligent high-definition camera, and the calibration of the auxiliary intelligent high-definition camera. The calibration of the lidar includes the calibration of the external parameters of the lidar calibration. The external parameter calibration of the collaborative intelligent high-definition camera and the external parameter calibration of the lidar calibration are performed simultaneously.

[0041] Step S204: Divide the ROI region into a region of interest and a shielded region. The region of interest includes the region of interest of the high-definition camera and the region of interest of the millimeter radar. Use ROI annotation to perform lane annotation, lane line annotation, traffic participant detection area annotation, traffic event annotation, and traffic flow analysis annotation on the region of interest of the high-definition camera, the region of interest of the millimeter radar, and the shielded region.

[0042] Step S205 involves optimizing the target object fusion algorithm, target object perception performance, ROI area, sensitivity, event detection performance, and accuracy of traffic statistics parameters.

[0043] In one embodiment, the calibration steps for the external parameters of the collaborative intelligent high-definition camera and the external parameters of the lidar calibration specifically include: selection of acquisition points, RTK point acquisition, acquisition of calibration images, point calibration, latitude and longitude fusion, and calibration completion.

[0044] The auxiliary intelligent high-definition camera calibration adopts a multimodal data mapping calibration method based on static feature points to achieve high-precision alignment between the image pixel coordinate system and the geographic coordinate system;

[0045] The millimeter-wave radar calibration specifically includes: acquiring the coordinate system of the millimeter-wave radar and the world coordinate system for multiple identical targets, and calculating and establishing the mapping relationship between the world coordinate system and the millimeter-wave radar coordinate system through these data, so as to determine the latitude and longitude and the north deflection angle of the world coordinate system of the millimeter-wave radar.

[0046] In one embodiment, dividing the ROI region into a region of interest and a masked region specifically includes:

[0047] Step S2041: Divide the ROI area into the region of interest for high-definition cameras, the region of interest for millimeter radar, and the shielded area, so that the sensing range of the sensing devices overlaps in the center of the intersection or the critical area.

[0048] Among them, when the perception accuracy of the high-definition camera is lower than that of the radar, causing one or more of the following situations in the fusion result: jump, back and forth jitter, or splitting, the option is to shrink the far-end detection range of the high-definition camera or adopt a partitioned selection method.

[0049] Step S2042: Draw the Region of Interest (ROI) of the high-definition camera, the millimeter radar, and the shielded area based on images or maps to limit the effective area of ​​the data.

[0050] Step S2043: Mark traffic participants within the defined effective area; wherein the rules for marking traffic participants within the defined effective area specifically include:

[0051] Priority is set for effective regions. Regions of interest drawn on the same sensing device can overlap. If they overlap, the priority of the effective regions is used to determine the region.

[0052] When a sensing device does not define a region of interest, traffic participants that do not fall within any shielded area will be included in the fusion by default;

[0053] When a sensing device defines a region of interest, traffic participants that do not fall within any shielded area will be discarded by default.

[0054] On the same sensing device, ROI regions drawn based on maps and ROI regions drawn based on images will be mixed together for judgment according to priority.

[0055] In one embodiment, lane labeling specifically includes: drawing along the boundary of a single lane within the region of interest to form a closed polygon, and labeling the lane as a motor vehicle lane or a non-motor vehicle lane;

[0056] Lane markings specifically include: using polylines to draw solid lines in the aforementioned motor vehicle lanes or non-motor vehicle lanes;

[0057] The specific steps of traffic participant detection area labeling include: drawing a closed polygon in the labeling image to indicate that traffic participants within this area will be identified, thus labeling it as a region of interest;

[0058] Among them, when there are obstacles in the traffic participant detection area that are prone to false detection, a shielding area is superimposed on the traffic participant detection area to avoid false detection;

[0059] Traffic incident labeling specifically includes: labeling traffic incidents within the area of ​​interest. Traffic incidents include intersection congestion incidents, pedestrian or non-motorized vehicle crossing the intersection incidents, solid line crossing incidents, abnormal parking incidents, abnormal low speed incidents, wrong-way driving incidents, and speeding incidents.

[0060] The specific parameters for determining traffic incident labels include:

[0061] The parameters for labeling intersection congestion events are set as follows: the lower limit of vehicle speed is set to 20 km / h, and the alarm threshold is set to: the number of vehicles with a speed lower than 20 km / h accounts for 20%.

[0062] The event labeling parameter for pedestrians or non-motorized vehicles entering the intersection is set to a time threshold of 3 seconds.

[0063] The event annotation parameter for the solid line is set to a time threshold of 3 seconds.

[0064] The abnormal parking event labeling parameter is set to a time threshold of 30 seconds.

[0065] The abnormal low-speed event labeling parameters are set as follows: the lower speed limit is set to 10km / h, the time threshold is set to 10s, and the alarm threshold is set to: the number of vehicles with a speed lower than 10km / h accounts for 40%.

[0066] The parameters for labeling retrograde events are set to a time threshold of 3 seconds.

[0067] The parameters for labeling speeding events are set as follows: the lower speed limit is 40 km / h, the time threshold is 3 seconds, and the alarm threshold is set to 10% of the vehicles exceeding 40 km / h.

[0068] Traffic flow analysis annotations specifically include: extracting average vehicle speed, density, delay, congestion status, and queue length within the region of interest.

[0069] In one embodiment, optimizing the fusion algorithm for the target objects specifically includes:

[0070] The raw data output by the sensing devices are distinguished and observed using different colors. An overlap threshold is set to determine whether the overlap of the detection bounding boxes of the same traffic participant output by multiple sensing devices is within the overlap threshold. If it is within the overlap threshold, the data processed by the fusion algorithm is then judged.

[0071] Based on the data processed by the fusion algorithm, observe the process of a vehicle waiting at the red light before the stop line and entering the fusion zone in the center of the intersection, or observe the traffic flow, observe the handover between the output data of the high-definition camera and the output data of the radar, and observe whether there are vehicle splitting and identity switching errors. If there are no errors, end the target object fusion algorithm optimization.

[0072] The target perception performance is optimized, including the target positioning accuracy, detection rate, accuracy, target size accuracy, system latency, target frame loss rate, target ID loss and discontinuity, number of abrupt changes in positioning and heading angle, queue detection, parking detection, lane ID accuracy, target splitting, false targets, target position jitter, inter-frame time variation, and perception range.

[0073] Optimization of ROI region adjustment, sensitivity adjustment, and event detection performance includes:

[0074] Adjust the ROI (Region of Interest) range for image or video processing to exclude interference from irrelevant traffic participants;

[0075] Set a sensitivity level of 5, and adjust the sensitivity level according to the false alarm or missed alarm ratio;

[0076] In simulated or real-world scenarios, traffic events are triggered via OBU, MEC, or cloud platform. Expected detection and accuracy values ​​are set according to requirements. The detection and accuracy values ​​in the triggered traffic events are calculated and compared with the set expected detection and accuracy values. If they match, the optimization ends.

[0077] The detection rate is calculated as follows: (Number of successfully detected events / Number of actual triggered events) × 100%.

[0078] Accuracy = (Number of correctly detected events / Total number of detected events) × 100%

[0079] Optimizing traffic flow detection specifically includes: collecting MEC structured traffic flow data during peak hours, outputting the actual traffic flow and the traffic flow output by MEC for the same time period, and calculating the traffic flow accuracy.

[0080] Wherein, traffic flow accuracy = |actual traffic volume - MEC output traffic volume| / actual traffic volume

[0081] The calculated accuracy is compared with the set expected value for traffic flow accuracy. If it meets the set expected value, the traffic flow detection optimization ends.

[0082] The present invention has the following beneficial effects:

[0083] Holographic perception and blind-spot-free coverage: Multi-sensor collaboration reduces the detection blind zone to less than 20 meters and extends the long-distance detection range to 250 meters, supporting centimeter-level positioning and continuous tracking of complex traffic targets.

[0084] Low-latency decision-making and control: Edge-side data fusion and hierarchical computing power configuration compress data processing latency to the millisecond level, and the transmission frequency of key information (such as early warning commands) reaches 10Hz, meeting the real-time response requirements of the vehicle.

[0085] System reliability and scalability: Standardized deployment and modular network design support rapid device adaptation, while dedicated network communication and redundant power supply ensure system stability in complex environments. Attached Figure Description

[0086] Figure 1a A block diagram of a vehicle-road cooperative roadside signal processing device according to an embodiment of the present invention;

[0087] Figure 1b This is a schematic diagram illustrating the specific implementation of a vehicle-road cooperative roadside signal processing device according to an embodiment of the present invention;

[0088] Figure 2a and Figure 2b This is a cooperative intelligent location topology diagram and an auxiliary intelligent location topology diagram of a vehicle-road cooperative roadside signal processing device according to an embodiment of the present invention;

[0089] Figure 3 This is a schematic flowchart of a vehicle-road cooperative roadside signal processing method according to an embodiment of the present invention.

[0090] Among them, 100 is the sensing device group; 110 is the high-definition camera; 120 is the lidar; 130 is the millimeter-wave radar; 140 is the traffic light collector; 200 is the communication device group; 300 is the edge computing node; 310 is the data management module; 320 is the fusion sensing module; 330 is the application function module; and 340 is the system management module. Detailed Implementation

[0091] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. The described embodiments are some embodiments of this application, but not all embodiments. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0092] Figure 1a This is a block diagram of a vehicle-road cooperative roadside signal processing device according to an embodiment of the present invention, comprising:

[0093] The sensing device group 100 includes a high-definition camera 110, a lidar 120, a millimeter-wave radar 130, and a traffic light collector 140 deployed on the roadside to collect video streams, radar point cloud data, structured traffic data, and traffic light status data.

[0094] Communication equipment group 200 includes an RSU (Roadside Unit) module, which is connected to intelligent connected vehicles and edge computing nodes 300, such as... Figure 1b As shown, the system transmits vehicle location, driving intent, warning information, and control commands to the intelligent connected vehicle and the edge computing node 300.

[0095] The edge computing node 300 includes a data management module 310, a fusion sensing module 320, an application function module 330, and a system management module 340.

[0096] The data management module 310 receives video streams, radar point cloud data, structured traffic data, and traffic light status data transmitted by the sensing device group 100, and processes the video streams, radar point cloud data, traffic light status data, and structured traffic data to achieve data interaction.

[0097] The fusion perception module 320 generates traffic participant data based on the video stream, radar point cloud data, structured traffic data and traffic light status data processed by the data management module 310, and on a multi-sensor data fusion algorithm. The traffic participant data includes traffic participants and traffic participant trajectories.

[0098] Application function module 330 performs ROI (region of interest) partitioning and labeling on the traffic participant data generated by fusion perception module 320, and generates event detection analysis results and traffic flow analysis results based on the traffic participant data and the set event judgment conditions;

[0099] The system management module 340 monitors the device status, manages log hierarchically, implements crash restart strategies, and provides OTA (over-the-air) update functionality.

[0100] Specifically, in one embodiment, such as Figure 1b As shown, the construction of vehicle-road cooperative intelligence and auxiliary intelligence sensing and communication equipment in the field, as well as the provision of the basic environment for edge computing applications, specifically includes:

[0101] The deployment of 47 collaborative intelligent high-performance edge computing nodes, 110 high-definition cameras, 96 solid-state LiDARs, and 26 blind spot LiDARs for collaborative intelligent intersections and road sections has been completed.

[0102] The deployment of 103 intelligent low-computing-power edge computing nodes, 337 high-definition cameras, and 337 millimeter-wave radars for intelligent intersections and ramps has been completed.

[0103] The deployment of 145 collaborative and auxiliary intelligent regional vehicle-road communication devices and 119 traffic light data collectors has been completed.

[0104] Complete the construction of supporting private network communication equipment, pole boxes, pipelines, power supply, and other necessary components required for the deployment of field sensing and communication equipment;

[0105] Complete the construction of the basic environment for roadside vehicle-road cooperative edge computing applications.

[0106] The communication equipment group 200 transmits vehicle location, driving intention, early warning information, and decision / control information to the edge computing node 300.

[0107] The traffic light collector 140 transmits real-time signal timing information to the edge computing node 300 and the communication equipment group 200.

[0108] The communication equipment group 200 receives the vehicle location and driving intention of the intelligent connected vehicle, and broadcasts warning information and decision / control information to the intelligent connected vehicle via the RSU.

[0109] Edge computing node 300 transmits traffic participant data, traffic event data, and traffic flow data to the collaborative sensing subsystem.

[0110] Specifically, when installing the HD camera 110, the height is approximately 6 meters. At intersections, prioritize the second integrated pole arm, 20-30 meters from the stop line at the entrance lane (if none is available, use the first integrated pole at the entrance / exit lane). For road sections, install directly on the integrated pole arm. The arm position should be close to the center of the road. At four-lane intersections, the farthest end of the arm should extend to the boundary between the third and fourth lanes, with the equipment placed in the middle of the third lane. The detection range must meet the following requirements: the intersection camera faces the center of the intersection, simultaneously covering the entire panorama of the entrance / exit lanes. When detecting two directions on a road section, install two cameras facing the same direction. The bottom of the image should include the turn arrow of the entrance lane, and the opposite entrance / exit lanes should be located in the upper 1 / 3 area of ​​the image. The central divider should be aligned with the left and right 2 / 5 of the image.

[0111] The principles of installing and adjusting the angle of high-definition cameras (110) using traffic camera poles at intersections as an example:

[0112] The HD camera 110's field of view is generally directly facing the intersection, covering the area from the pole down to the opposite side of the intersection, making the road surface fill the field of view as much as possible, aligning the horizontal field of view with the horizontal zebra crossing, and being symmetrical from left to right, including the zebra crossings on both sides to the center area.

[0113] The installation conditions (height, location, boom and detection range) of solid-state lidar are the same as those of HD camera 110. The equipment angle requirements are: the near end of the point cloud should cover the turn arrow of the entrance road and ensure a blind zone of 15-20 meters below the boom, and the far end should extend to a range of 250 meters; the near point of the ROI area should be set at the center point of the intersection about 50 meters away from the equipment.

[0114] Solid-state lidar angle adjustment: Align the center point of the radar field of view with the center point of the road area to be covered to ensure effective monitoring range; place the radar device's ROI in the exact center of the road area to be covered to ensure that the radar's optimal coverage area can monitor more targets.

[0115] The blind spot lidar is installed on the cantilever arm of the roadside utility pole, located in the middle of the entrance / exit lane, at a height of approximately 6 meters. The equipment must be installed perpendicular to the ground. Additional blind spot lidars are required at locations where two solid-state lidars have already been deployed, specifically to detect the area within a 30-meter radius below the utility pole. The equipment must be kept perpendicular to ensure coverage of the target blind spot.

[0116] Blind spot lidar angle adjustment: The blind spot lidar is installed using a custom U-shaped bracket, with the coverage direction perpendicular to the road surface downwards. The lidar is fixed to the vertically downward-facing base universal joint with screws, and a level is used to ensure that the installation surface of the blind spot lidar is level with the ground.

[0117] The millimeter-wave radar 130 is installed at a height of approximately 6 meters. At intersections, it should be installed on the second integrated pole arm, 20-30 meters from the stop line at the entrance lane (if none is available, use the first integrated pole, electronic police pole, or traffic light pole at the entrance / exit lanes). The arm position should be consistent with that of the high-definition camera 110. The detection range must face the center of the intersection, simultaneously covering the entrance / exit lanes. The equipment angle must be strictly perpendicular to the ground; upward or downward angles are prohibited. Adjust the orientation according to different intersection types: at perpendicular intersections, the front edge of the equipment should be parallel to the stop line; at angled intersections, the front should be perpendicular to the centerline of the road surface; for sloped roads, the front of the equipment should be perpendicular to the centerline projection.

[0118] Millimeter-wave radar 130 installation and angle adjustment: Adjust the millimeter-wave radar 130 so that its side is perpendicular to the horizontal plane (this can be done with the help of a plumb line); adjust the horizontal offset angle so that it faces the area to be detected; adjust the vertical offset angle so that the vertical line of its front surface falls within the detection area.

[0119] The communication equipment is installed on the first integrated pole / electronic police pole / traffic light pole arm closest to the integrated equipment box at the intersection. The position of the integrated pole arm in the road section is the same as the position of the high-definition camera 110 arm. The height can be set at about 6m. The communication range is within a radius of 300 meters from the center point of the equipment. The equipment angle is such that the high-precision positioning antenna side needs to face the sky.

[0120] The traffic light data acquisition unit 140 and the edge computer node are installed in the intersection integrated equipment box.

[0121] After the equipment is installed, such as Figure 2a , Figure 2b As shown, communication establishment: After the collaborative intelligent intersection / segment equipment and the auxiliary intelligent intersection / ramp equipment complete communication aggregation at the intersection aggregation switch, they are connected to the leased equipment room's field access switch through the existing communication optical cable.

[0122] High-definition cameras 110 and solid-state LiDAR within the collaborative intelligent intersection area are connected to the cross-section switch via Cat5e network cables. The cross-section switch is connected to the intersection aggregation switch via a 4-core optical fiber cable. Edge computing nodes MEC, RSU, and traffic light collectors 140 are connected to the intersection aggregation switch via Cat5e network cables.

[0123] The collaborative intelligent road section high-definition camera 110, solid-state lidar, blind spot lidar, edge computing node MEC, and RSU are connected to the section switch (road section) via Cat5e network cable. The section switch (road section) is connected to the nearest intersection aggregation switch via 4-core optical cable.

[0124] The high-definition camera 110 and millimeter-wave radar 130 within the range of the auxiliary intelligent intersection are connected to the cross-section switch via Cat5e network cable. The cross-section switch is connected to the intersection aggregation switch via a 4-core optical cable. The edge computing node MEC, RSU, and traffic light collector 140 are connected to the intersection aggregation switch via Cat5e network cable.

[0125] The auxiliary intelligent on-ramp is equipped with 110 high-definition cameras, 130 millimeter-wave radars, edge computing nodes (MEC), and RSUs connected to the cross-section switch (on-ramp) via Cat5e network cables. The cross-section switch (on-ramp) is connected to the nearest intersection aggregation switch via 4-core optical fiber.

[0126] In addition, the signal light acquisition unit 140 uses a multi-core cable to connect to the SCATS (Sydney Coordinated Adaptive Traffic System) signal controller to acquire signal light phase information.

[0127] This application has multiple interfaces, including an external interface, a lidar interface, a video stream interface, a traffic light collector 140 interface, an RSU interface, an infrastructure private network interface, and a cloud control platform interface, and performs judgment and real-time alarms on the data from multiple interfaces.

[0128] The system management module 340 monitors the device status, including device status monitoring and application status monitoring.

[0129] Device Status Monitoring: An operation and maintenance interface is established between the edge computing node and external connected devices such as RSUs, cameras, and millimeter-wave radar 130. The operating status of these external devices is monitored through this interface. Specifically, the operating status of the external devices includes:

[0130] Equipment information, such as equipment location latitude and longitude, configuration parameters, software and hardware version numbers, and operational information; equipment heartbeat information, indicating whether its own operating status is normal; equipment operating status, including equipment operating status, CPU, memory, disk, and network usage information; equipment logs, log information is mainly used for remote diagnosis and debugging; operation and maintenance management configuration and query, realizing daily operation and maintenance management. The edge computing node 300 periodically reports operation and maintenance management-related data to the cloud control platform, including RSU data, camera data, and millimeter-wave radar 130 data.

[0131] Application status monitoring: During operation, the application status should be reported regularly, alarms and error information should be reported in a timely manner, and the application algorithm should have the ability to handle anomalies to improve the stability of the application algorithm.

[0132] OTA update functions mainly include installing, uninstalling, and upgrading existing applications, as well as adding, removing, starting, and stopping instances of installed applications.

[0133] For installing new applications or updating program files of existing applications, different update processes are used depending on the application deployment type:

[0134] SO type (traditional deployment based on physical servers, requiring dedicated hardware): relies on two cloud resources: OTA service running on the platform and OTA resource package file server.

[0135] Docker type (containerized deployment, supporting cloud-edge collaboration and rapid upgrades): relies on running DockerRegistry (container image repository) on the platform.

[0136] Other types: Applications that only implement the OTA settings and response information of the operation and management platform protocol. If either of the above two types of OTA operations fails midway, the failure information will eventually be returned to the operation and management platform in the OTA settings response message.

[0137] Log Management: In the edge computing node 300, the device's basic software programs, application algorithms, and interface modules are inspected, and five levels of program warning and error message logs are established: 1. Debug; 2. Information; 3. Warning; 4. Error; and 5. Fatal Error. Different log levels are selected for output based on different operating conditions.

[0138] 1. Debugging: Prints fine-grained information and event information, mainly used to print some runtime information during the development process.

[0139] 2. Information: Print information that highlights the application's operation process at a coarse-grained level.

[0140] 3. Warning: Printing information may result in potential error conditions.

[0141] 4. Error: Prints a message indicating that an error event has occurred, but the system continues to operate normally.

[0142] 5. Fatal Errors: Prints information about each serious error event that will cause the application to crash.

[0143] In addition, for the algorithm applications in edge computing node 300, a custom log is output to record the algorithm's running status.

[0144] Crash management: In the edge computing node 300, the running status of the device's basic software programs, application algorithms and interface modules is monitored in real time, and the corresponding business operations are guaranteed when the above programs crash.

[0145] Crash Alerts: Upon a crash, the above program must immediately generate an alert, report it to the platform, and write it to the log for auditing purposes. Crash Restart Policy: The program must restart immediately after a crash to minimize the impact on the corresponding business operations. A crash restart policy should be configurable, including whether to restart indefinitely and the maximum number of restarts. Crash Reports: For SO-type application modules, a crash report (core dump files, etc.) should be generated upon a crash and uploaded to the platform periodically for operations and development personnel to locate problems and improve program stability. The system should also have the function of periodically clearing accumulated crash reports.

[0146] The data interaction in the data management module 310 includes: acquiring the original video of the high-definition camera 110 at intersections and road sections through the RTSP (Real-Time Streaming Protocol) protocol, and receiving and parsing the real-time video data of the camera.

[0147] This enables the reception and parsing of raw structured data from the millimeter-wave radar 130 at intersections and road sections for identifying traffic participants.

[0148] Acquire raw radar point cloud data from LiDAR 120 (solid-state LiDAR and blind spot LiDAR), and parse, process, and analyze the radar point cloud data packets to extract data such as traffic object type and 3D size.

[0149] Acquire signal light status and timing information from signal light collector 140.

[0150] It enables data interaction with the RSU module, including: BSM (Basic Safety Message), VIR (Vehicle Identification Message), RSM (Road Safety Message), RSC (Roadside Coordination Message), and Dynamic RSI (Dynamic Roadside Information).

[0151] Enables data interaction with the cloud control platform, including: participants, traffic lights, and traffic events.

[0152] Enables data interaction with the entire operation and management system, including: status, alarm data, OTA upgrades, operation and maintenance configuration, and parameter reporting.

[0153] Enables data interaction with infrastructure private networks and store-and-forward systems.

[0154] The installation of the collaborative intelligent infrastructure environment for edge computing node 300 mainly includes the installation of k8s (Kubernetes) and the installation of k8s dependencies.

[0155] In one embodiment, the high-definition camera 110 is installed on the boom of a composite pole 20-30 meters away from the stop line, with a detection range covering the entrance and exit lanes of the intersection. The installation height is 6 meters, and the lens is focused and adjusted to the blind zone of 15-20 meters under the pole.

[0156] The lidar 120 includes a solid-state radar and a blind spot radar. The blind spot radar covers an area with a radius of 30 meters, and the radar point cloud data frame rate is ≥10Hz.

[0157] The millimeter-wave radar 130 is vertically mounted and outputs target type, velocity, and structured data at a frame rate ≥10Hz.

[0158] In one embodiment, generating event detection and analysis results based on traffic participant data and set event determination conditions specifically includes:

[0159] Conditions for determining an intersection congestion event: When the queue of at least 3 vehicles is between two red lights in a lane phase, and the exit lane overflows, preventing vehicles from passing, and the average speed of the road segment is 20% lower than the speed limit, the event detection and analysis result is a current road congestion event;

[0160] Conditions for determining a pedestrian or non-motorized vehicle intrusion event: When it is determined that a pedestrian or non-motorized vehicle has entered the intersection or motor vehicle lane and the dwell time is not less than 3 seconds, the event detection and analysis result is a pedestrian or non-motorized vehicle intrusion event.

[0161] Conditions for determining a solid line crossing event: When it is determined that a vehicle has been driving over the solid line for no less than 3 seconds or has changed lanes by crossing the solid line, the event detection and analysis result is a solid line crossing event.

[0162] Conditions for identifying abnormal parking events: When it is determined that parking in a no-parking zone for no less than 30 seconds constitutes an abnormal parking event;

[0163] Conditions for judging abnormal low-speed events: When the number of vehicles judged is no more than 3 and the vehicle speed is less than 40% of the current road speed limit for no less than 10 seconds, or when the number of vehicles judged is greater than 3 and the vehicle speed is less than 40% of the current traffic flow average speed for no less than 10 seconds, the event detection and analysis result is an abnormal low-speed event.

[0164] Conditions for determining a wrong-way driving event: When the vehicle trajectory deviates from the lane direction by more than 90° and lasts for no less than 3 seconds, the event detection and analysis result is a wrong-way driving event;

[0165] Conditions for determining a speeding event: When the vehicle speed exceeds the current road speed limit by more than 10% and lasts for no less than 3 seconds, the event detection and analysis result is a speeding event.

[0166] In one embodiment, the traffic flow analysis results specifically include:

[0167] The number of vehicles passing through a designated section of a road per unit of time;

[0168] The distance a vehicle travels in a unit of time;

[0169] The number of vehicles per unit length of road;

[0170] The difference between the actual travel time required for a vehicle to pass through an intersection under obstructed conditions and the time required to travel the same distance normally;

[0171] The length of a vehicle queue from the stop line at the intersection or the start of the queue to the end of the queue;

[0172] The vehicle's current traffic situation;

[0173] Based on the average delay of vehicles passing through the intersection, traffic conditions are divided into four levels: severe congestion, moderate congestion, light congestion, and smooth traffic, to evaluate the current traffic situation of vehicles.

[0174] A vehicle-road cooperative roadside signal processing method, implemented by the aforementioned vehicle-road cooperative roadside signal processing device, includes:

[0175] Step S101: Design parameter configuration, and install the corresponding sensing device according to the parameter configuration;

[0176] Step S102: Collect video streams, radar point cloud data, structured traffic data and traffic light status data detected by the sensing device, and perform data interaction on the video streams, radar point cloud data, structured traffic data and traffic light status data.

[0177] Step S103: Generate traffic participant data based on a multi-sensor data fusion algorithm. The traffic participant data includes traffic participants and their trajectories.

[0178] Step S104: Perform ROI partitioning and labeling on the traffic participant data, and generate event detection analysis results and traffic flow analysis results based on the traffic participant data and event judgment conditions;

[0179] Among them, based on edge computing algorithms, parameter configuration, sensing devices, ROI labeling, and fusion algorithms are iteratively optimized.

[0180] Specifically, during step S101, it is necessary to debug the MEC network and the basic environment, including:

[0181] Step S1011 requires confirming that the required services have been deployed on the edge computing node 300. This involves checking the status of each service to confirm that it has started normally, and confirming that all sensor devices and data debugging tools can be accessed via the network. Use PING to confirm whether a device can be accessed. If the accessed device acts as a server during data transmission, check whether the TCP server is running using tools such as nc or telnet. Configure PTP time synchronization by entering the IP address and port number of the PTP time synchronization server through the manual input interface of the MEC device management system to complete PTP time synchronization.

[0182] Step S1012: After the MEC software deployment is complete, further software configuration is required, including data acquisition, intersection configuration, and sensor access. Data acquisition refers to retrieving data from the imported roadside sensing devices. Intersection configuration involves importing the map into the MEC for intersection configuration, including information such as intersection name, shape, intersection center point latitude and longitude coordinates, city code, and area code. Collaborative intelligence also supports automatic parsing of road segment names, lane areas (direction, entrances and exits, numbering from lanes near the median strip to lanes away from the median strip), lane / lane line type, number, lane driving direction, and lane speed limit information from the imported high-precision map. Sensor access allows viewing information such as the name, number, type, manufacturer, model, IP, installation location, detection direction, latitude and longitude location, operating status, and remarks of all currently connected high-definition cameras 110, millimeter-wave radars, or lidars 120 via the MEC device management list.

[0183] Step S1013, perform data push check, including:

[0184] Collaborative Intelligence: Use `kubectl logs -f cloud-control-xxxx | grep -i “tcp”` to query the MEC data push status. The following values ​​indicate success:

[0185] Traffic incident data: event

[0186] Traffic participant data: participation

[0187] Traffic light data: light

[0188] Assisted intelligence: By using tcpdump in MEC to check if the algorithm is pushing data, an example command is as follows:

[0189] sudo tcpdump -i any udp port 50813 -nn -X

[0190] In one embodiment, iterative optimization of parameter configuration, sensing devices, ROI labeling, and fusion algorithms based on edge computing algorithms specifically includes:

[0191] Step S201, parameter configuration and verification: Configure the relevant parameters in the edge computing node 300, including network connection parameters, routing table settings, firewall settings, time settings, device information and deployment settings, and complete the verification.

[0192] Step S202: Install and debug the equipment, and confirm that the required roadside equipment is complete. The roadside equipment includes a high-definition camera 110, a lidar 120, a millimeter-wave radar and a traffic light collector 140, a bracket, and cables.

[0193] Step S203: Calibrate the HD camera 110, millimeter-wave radar, and lidar 120. The calibration of the HD camera 110 includes the internal parameter calibration of the collaborative intelligent HD camera 110, the external parameter calibration of the collaborative intelligent HD camera 110, and the auxiliary intelligent HD camera 110 calibration. The calibration of the lidar 120 includes the external parameter calibration of the lidar 120 calibration. The external parameter calibration of the collaborative intelligent HD camera 110 and the external parameter calibration of the lidar 120 are performed simultaneously.

[0194] Step S204: Divide the ROI region into a region of interest and a shielded region. The region of interest includes the region of interest of the high-definition camera 110 and the region of interest of the millimeter radar. Use ROI annotation to perform lane annotation, lane line annotation, traffic participant detection area annotation, traffic event annotation, and traffic flow analysis annotation on the region of interest of the high-definition camera 110, the region of interest of the millimeter radar, and the shielded region.

[0195] Step S205 involves optimizing the target object fusion algorithm, target object perception performance, ROI area, sensitivity, event detection performance, and accuracy of traffic statistics parameters.

[0196] Specifically, the internal parameter calibration steps for the HD camera 110 are as follows:

[0197] Connect the HD camera 110 to a power source and connect it to the computer via network cable; the HD camera 110 has a default IP address and does not need to be connected to an external network.

[0198] Configuration: bit rate 4M, frame rate 25, video encoded with H.264, resolution 1920×1080P;

[0199] Verify the pulling stream before recording the video to check whether the firmware contains SEI timestamps;

[0200] Enter the internal parameter calibration tool, fill in the configuration items, and the video stream can be obtained. When filling in the configuration items, the serial number of the high-definition camera 110 needs to be consistent with that in the subsequent MEC device management page; the resolution of the high-definition camera 110 needs to be consistent with the configuration content in the manufacturer's device management page; currently, a specified 12×9 checkerboard calibration board is used;

[0201] A real-time video stream appears in the real-time video frame. One person holds the checkerboard in front of the high-definition camera 110, and the other person guides the placement of the checkerboard according to the real-time video. The frame selection module on the right will automatically generate picture frames synchronously. Recording for 2-3 minutes at various angles can meet the calibration requirements;

[0202] Check the generated picture frames, requiring at least 12 pictures, and try to select pictures with diverse angles;

[0203] After the internal parameter calibration result is prompted, it is necessary to confirm that there is no picture deformation in the "undistortion" module. If rmsd_world < 0.3 in the generated json file of the internal parameter calibration result (the reprojection position error after calibration < 0.3 meters), the internal parameter calibration is completed. If not up to the standard, pictures need to be selected or even re-recorded and calibrated again.

[0204] The external parameter calibration of the high-definition camera 110 must be carried out simultaneously with the external parameter calibration of the lidar 120. The calibration steps include:

[0205] Select the acquisition points. During a period with low traffic flow, preselect the points based on the calibration images provided by the MEC to ensure the uniqueness of the point IDs within the intersection. The point selection should follow the following principles:

[0206] The points must be evenly distributed, covering the entire area, and can be corresponding one by one in the image, point cloud and real scene (with the help of traffic cones for positioning);

[0207] The points need to fall on the same plane road surface, avoiding the curb and surrounding obstacles (such as lamp posts, trees, etc.), and maintaining a distance of at least 0.5 meters;

[0208] The layout is in the shape of "eye" or "field", and the distance meets the product requirements. Priority should be given to selecting the corner points of the traffic cones and ground dotted lines, avoiding the center position of the figure;

[0209] The distal points should be selected with distinct features to reduce the error between the image and the actual RTK coordinates.

[0210] RTK point acquisition: An RTK positioning terminal, a measuring rod, and a smart Pad terminal work together. The measuring rod is erected vertically at the selected location, set to a height of 1.8 meters, and the bubble level is adjusted to the center. The acquisition is confirmed via the Pad terminal, and the point is marked after maintaining the position for 3 seconds.

[0211] Image Acquisition for Calibration: Acquire the images to be calibrated from the HD camera 110 and the LiDAR 120 through the device management interface. The following conditions must be met:

[0212] Collect no fewer than 11 points in a single direction, including at least 9 matching points and 2 remote endpoints;

[0213] Matching points: located within the clear perception range of the intersection of the image and the point cloud, covering near / far zebra crossings and areas within intersections, with at least 3 points selected for each category, and priority given to ground reflection signs or traffic cones;

[0214] Distant endpoints: distributed at the far ends of the left, center, and right regions of the camera's field of view, beyond the point cloud's perception range.

[0215] Point marking:

[0216] Import the image to be calibrated into the external parameter calibration interface, and display it in groups according to the sensor installation orientation. Interactive functions are supported.

[0217] Adjust point cloud block size, scale and pan images, set calibration point ID / color / size, and support clearing or moving calibration points;

[0218] Each image is independently calibrated to ensure that the location corresponds precisely to the image or radar point cloud data.

[0219] Latitude and longitude fusion: Supports two coordinate association methods: obtaining GPS coordinate files collected offline by RTK (which need to be imported in the correct format) or directly rendering online high-precision maps;

[0220] If using coordinate files, ensure that the point IDs are consistent with the calibration data, and verify the format and accuracy through the platform.

[0221] Calibration completion and result verification: The system automatically generates calibration results, and the manually calibrated points (red) are compared with the system's reflected points (green);

[0222] Failure handling: If calibration fails, the cause must be traced:

[0223] Coordinate error: Return to the "Latitude and Longitude Integration" step to correct the coordinates and recalculate;

[0224] Calibration error: Return to the "Point Calibration" step to readjust the calibration points.

[0225] After the internal and external parameters were calibrated, the calibration results are as follows:

[0226] img*.jpg: A set of undistorted images;

[0227] wgs84.csv: The calibrated gps and pixel corresponding point file;

[0228] camera_intrinsic.json: The internal parameters of the high-definition camera 110;

[0229] center.location.json: The origin of the intersection.

[0230] In one embodiment, the steps for calibrating the external parameters of the collaborative intelligent high-definition camera 110 and the external parameters calibrated by the lidar 120 specifically include: collection point selection, RTK point collection, calibration image acquisition, point calibration, longitude and latitude and fusion, and calibration completion;

[0231] The calibration of the auxiliary intelligent high-definition camera 110 adopts a multi-modal data mapping calibration method based on static feature points to achieve high-precision alignment between the image pixel coordinate system and the geographic coordinate system;

[0232] The calibration of the millimeter-wave radar specifically includes: obtaining the coordinate system of the millimeter-wave radar and the world coordinate system for multiple identical targets, and calculating and establishing the mapping relationship between the world coordinate system and the millimeter-wave radar coordinate system through these data to determine the longitude, latitude and north deflection angle of the world coordinate system of the millimeter-wave radar.

[0233] The calibration operation process of the multi-modal data mapping calibration method based on static feature points:

[0234] Feature point selection and image acquisition: Obtain the real scene image covered by the target camera's field of view, and select no less than 8 calibration points with clear geometric features.

[0235] The layout principle of calibration points: Uniformly distributed in a "mesh" or "field" shaped matrix, the nearest end is 10 meters behind the stop line on the same side of the high-definition camera 110, and the farthest end is 150 meters away from the sensor;

[0236] Priority is given to selecting easily recognizable features such as the corners of road signs and the intersections of traffic markings.

[0237] RTK geographic coordinate acquisition: Use the triangulation method to determine the position of the calibration points, start the Qianxun RTK positioning system to enter the fixed solution state after laying the ground marks;

[0238] Traverse each image feature point in sequence, and use the Qianxun RTK positioning device to collect coordinates:

[0239] Ensure that the bottom of the positioning rod coincides precisely with the feature point and maintain a vertical state;

[0240] Record the WGS-84 geographic coordinates of each feature point one by one to form a coordinate sequence file.

[0241] Verification of calibration results: Import the image coordinates (u, v) of the calibration points and the corresponding RTK coordinates (longitude, latitude, elevation) into the MEC calibration management system to construct a coordinate system mapping model;

[0242] Calibration and verification test: Assign personnel to carry RTK equipment to obtain fixed solution coordinates at random test points within the intersection;

[0243] Simultaneously extract the mapping position of the test point from the MEC output and calculate the plane projection deviation between the two.

[0244] Acceptance criteria: A deviation of less than 1 meter is considered acceptable; otherwise, the calibration process must be repeated until the standard is met.

[0245] Calibration deliverables: After calibration, the output file includes digital images corresponding to each calibrated feature point;

[0246] Annotation file: A matrix data table containing image pixel coordinates (u, v) and WGS-84 geographic coordinates;

[0247] Verify test records and deviation analysis reports.

[0248] The specific calibration of millimeter-wave radar is as follows:

[0249] For point selection, the standard requirement is 10 points: 1 radar origin and 9 target acquisition points. The radar origin collects GPS geographic coordinates (Lat, Lng), and the target acquisition points collect radar detection coordinates (x, y) and GPS coordinates (Lng, Lat). Latitude and longitude are recorded in degrees.

[0250] Location information acquisition process: Upgrade the radar firmware to dedicated calibration firmware; select acquisition points on the road according to the layout, place cones at the points, check the detection status of the cone targets at the points, identify and locate the targets, record the (x, y) coordinates of the targets in the calibration information; after collecting the (x, y) coordinates of the point with cones, place RTK at the point to collect latitude and longitude coordinates; place the RTK device at the point and record the latitude and longitude coordinates of the point; collect and record the latitude and longitude coordinates of the radar origin; record and organize the (x, y) and corresponding (Lng, Lat) coordinates of each point, including id (acquisition point type), radar (millimeter-wave radar), target (acquisition point); x and y represent the position of the point in the millimeter-wave radar coordinate system; Lng and Lat represent the latitude and longitude information collected by RTK, recorded in "degrees".

[0251] Calibration Verification: The radar coordinates and RTK point coordinates of the calibration points are entered into the millimeter-wave radar's calibration management page. A fixed solution is obtained manually at any point within the inner edge using a cone and the RTK device, and the GPS position is recorded. Simultaneously, the position of the corresponding pedestrian is compared with the structured data output from the millimeter-wave radar. If the deviation is no greater than 0.8 meters, the calibration result is considered to meet expectations; otherwise, the calibration result is considered not to meet expectations, and the point information collection process is repeated until the result meets expectations.

[0252] After calibration, the output results are the radar coordinates and GPS coordinates of the recorded collection points that meet the expectations.

[0253] The LiDAR calibration results are as follows: pc.pcd: original point cloud; annot_3d.csv: calibrated points (including center point, ground point and matching point); coor_mapping_tudatong.json: coordinate mapping relationship; wgs84.csv: calibrated GPS and corresponding point file; center.location.json: intersection origin.

[0254] In one embodiment, dividing the ROI region into a region of interest and a masked region specifically includes:

[0255] Step S2041: Divide the ROI area into the region of interest of the high-definition camera 110, the region of interest of the millimeter radar, and the shielded area, so that the sensing range of the sensing device has an overlap in the center of the intersection or the critical area.

[0256] Among them, when the perception accuracy of the high-definition camera is lower than that of the radar, causing one or more of the following situations in the fusion result: jump, back and forth jitter, or split, the remote detection range of the high-definition camera 110 is reduced or a partition selection method is adopted.

[0257] Step S2042: Draw the Region of Interest (ROI) of the high-definition camera 110, the millimeter radar ROI, and the shielded area based on images or maps to limit the effective area of ​​the data.

[0258] Step S2043: Mark the traffic participants within the defined effective area;

[0259] The specific rules for marking traffic participants within the defined effective area include:

[0260] Priority is set for effective regions. Regions of interest drawn on the same sensing device can overlap. If they overlap, the priority of the effective regions is used to determine the region.

[0261] When a sensing device does not define a region of interest, traffic participants that do not fall within any shielded area will be included in the fusion by default;

[0262] When a sensing device defines a region of interest, traffic participants that do not fall within any shielded area will be discarded by default.

[0263] On the same sensing device, ROI regions drawn based on maps and ROI regions drawn based on images will be mixed together for judgment according to priority.

[0264] Specifically, the goal of avoiding false detections is achieved by overlaying high-priority shielding zones on grassy areas or roadside poles where false detections are more likely to occur.

[0265] Image-based rendering uses the actual footage captured by the HD camera 110 to draw pixel-based regions on the image, thereby defining the effective area for data reporting by the HD camera 110.

[0266] Map-based plotting methods define the effective area of ​​data by drawing regions on a map in units of latitude and longitude.

[0267] In one embodiment, ROI annotation is primarily done visually on the ROI annotation page of MEC device management.

[0268] In one embodiment, lane labeling specifically includes: drawing along the boundary of a single lane within the region of interest to form a closed polygon, and labeling the lane as a motor vehicle lane or a non-motor vehicle lane;

[0269] Lane markings specifically include: using polylines to draw solid lines in the aforementioned motor vehicle lanes or non-motor vehicle lanes;

[0270] The specific steps of traffic participant detection area labeling include: drawing a closed polygon in the labeling image to indicate that traffic participants within this area will be identified, thus labeling it as a region of interest;

[0271] Among them, when there are obstacles in the traffic participant detection area that are prone to false detection, a shielding area is superimposed on the traffic participant detection area to avoid false detection;

[0272] Traffic incident labeling specifically includes: labeling traffic incidents within the area of ​​interest. Traffic incidents include intersection congestion incidents, pedestrian or non-motorized vehicle crossing the intersection incidents, solid line crossing incidents, abnormal parking incidents, abnormal low speed incidents, wrong-way driving incidents, and speeding incidents.

[0273] The specific parameters for determining traffic incident labels include:

[0274] The parameters for labeling intersection congestion events are set as follows: the lower limit of vehicle speed is 20 km / h, and the alarm threshold is 20%, meaning that within the monitoring area, the number of vehicles with a speed lower than 20 km / h accounts for 20% of the total number of vehicles.

[0275] The event labeling parameter for pedestrians or non-motorized vehicles entering the intersection is set to a time threshold of 3 seconds.

[0276] The event annotation parameter for the solid line is set to a time threshold of 3 seconds.

[0277] The abnormal parking event labeling parameter is set to a time threshold of 30 seconds.

[0278] The abnormal low-speed event labeling parameters are set as follows: the lower speed limit is 10km / h, the time threshold is 10s, and the alarm threshold is 40%. That is, within the monitoring area, the number of vehicles with a speed lower than 10km / h for 10 seconds accounts for 40% of the total number of vehicles.

[0279] The parameters for labeling retrograde events are set to a time threshold of 3 seconds.

[0280] The parameters for labeling speeding events are set as follows: the lower speed limit is 40 km / h, the time threshold is 3 seconds, and the alarm threshold is 10%. That is, within the monitoring area, the number of vehicles with a speed exceeding 40 km / h accounts for 10% of the total number of vehicles.

[0281] Traffic flow analysis annotations specifically include: extracting average vehicle speed, density, delay, congestion status, and queue length within the region of interest.

[0282] In one embodiment, optimizing the fusion algorithm for the target objects specifically includes:

[0283] The raw data output by the sensing devices are distinguished and observed using different colors. An overlap threshold is set to determine whether the overlap of the detection bounding boxes of the same traffic participant output by multiple sensing devices is within the overlap threshold. If it is within the overlap threshold, the data processed by the fusion algorithm is then judged.

[0284] Based on the data processed by the fusion algorithm, observe the process of a vehicle waiting at the red light before the stop line and entering the fusion zone in the center of the intersection, or observe the traffic flow, observe the handover between the output data of the HD camera 110 and the output data of the radar, and observe whether there are vehicle splitting and identity switching errors. If there are none, end the target object fusion algorithm optimization.

[0285] The target perception performance is optimized, including the target positioning accuracy, detection rate, accuracy, target size accuracy, system latency, target frame loss rate, target ID loss and discontinuity, number of abrupt changes in positioning and heading angle, queue detection, parking detection, lane ID accuracy, target splitting, false targets, target position jitter, inter-frame time variation, and perception range.

[0286] Optimization of ROI region adjustment, sensitivity adjustment, and event detection performance includes:

[0287] Adjust the ROI (Region of Interest) range for image or video processing to exclude interference from irrelevant traffic participants;

[0288] Set a sensitivity level of 5, and adjust the sensitivity level according to the false alarm or missed alarm ratio;

[0289] In simulated or real-world scenarios, traffic events are triggered via OBU, MEC, or cloud platform. Expected detection and accuracy values ​​are set according to requirements. The detection and accuracy values ​​in the triggered traffic events are calculated and compared with the set expected detection and accuracy values. If they match, the optimization ends.

[0290] The detection rate is calculated as follows: (Number of successfully detected events / Number of actual triggered events) × 100%.

[0291] Accuracy = (Number of correctly detected events / Total number of detected events) × 100%

[0292] Optimizing traffic flow detection specifically includes: collecting MEC structured traffic flow data during peak hours, outputting the actual traffic flow and the traffic flow output by MEC for the same time period, and calculating the traffic flow accuracy.

[0293] Wherein, traffic flow accuracy = |actual traffic volume - MEC output traffic volume| / actual traffic volume

[0294] The calculated accuracy is compared with the set expected value for traffic flow accuracy. If it meets the set expected value, the traffic flow detection optimization ends.

[0295] In one embodiment, it is observed that the bounding boxes (BEVs) of the same traffic participant output by different sensors almost completely overlap. This phenomenon indicates that the calibration effect of these sensors is very ideal. When the BEVs of the same traffic participant given by multiple sensors are highly overlapping, it shows that the position, size, and other information of the target object have formed a good consistency among different sensors, providing a good data foundation for multi-sensor fusion algorithms.

[0296] If the overlap is abnormal, the equipment can be recalibrated or the ROI settings can be adjusted.

[0297] Furthermore, the target positioning accuracy optimization includes: based on the MEC output structured target data and RTK vehicle ground truth data, finding the matching trajectory between the MEC and ground truth data; finding the Euclidean distance between similar time points based on the matching trajectory; calculating the average Euclidean distance for different distance intervals; reducing the 110 ROI area of ​​the high-definition camera to improve accuracy; comparing the target position information data output by the fusion sensing algorithm with the ground truth data to determine whether the overall position of the output data points lags behind or ahead of the ground truth system, and the average value of the overall offset data; calculating the average value and compensating the target position information output by the fusion sensing algorithm to be closer to the ground truth data based on the average value.

[0298] Target detection rate optimization includes: recording 10 minutes of structured data during peak and off-peak hours of the MEC; and locally streaming and storing 10 minutes of ground truth data from all HD cameras at the intersection (storage time is the same as the MEC recording time). Ground truth video data is aligned with MEC structured screen recording data. A list is compiled to statistically analyze the recognition and ground truth status of cars, vehicles, non-motorized vehicles, and pedestrians within the ROI range. The detection rate is calculated as: Detection rate = Number of correctly detected targets / Actual number of targets. The collaborative intelligence component employs optimized filtering and fusion algorithms, adjusting the measurement noise covariance in the filtering parameters to find a suitable value for signal fluctuation and faster convergence. For occluded targets, the confidence level of the LiDAR data is increased, enhancing the weight of LiDAR data in the fusion algorithm. The auxiliary intelligence component continuously collects and annotates on-site image data while simultaneously training and iterating the model to improve the target recognition detection rate.

[0299] Target accuracy optimization: Record 10 minutes of structured data during peak and off-peak hours using MEC; locally stream and store 10 minutes of ground truth data from all HD cameras at the intersection (storage time is the same as MEC recording time). Align the ground truth video data with the MEC structured screen recording data. List and statistically analyze the recognition and ground truth status of cars, large vehicles, non-motorized vehicles, and pedestrians within the ROI range. Calculate accuracy: Accuracy = Number of correctly classified targets / Number of actually detected targets. Common issues with target accuracy include false detections of large vehicles and mutual false detections of pedestrians and non-motorized vehicles. The optimization method for these is the same as the target detection rate optimization method.

[0300] Target size accuracy optimization: The true vehicle size is measured manually, and structured data output by the MEC (Multi-access Edge Computing) is recorded. Based on trajectory matching in positioning accuracy, the true vehicle size output by the MEC is confirmed, and then the difference is calculated with the manually measured true vehicle size. Target size accuracy is related to the performance of the algorithm model itself, and its optimization method is consistent with the target detection rate optimization method.

[0301] System latency optimization: The data acquisition method is the same as that used for common positioning accuracy optimization. Target structured data and video ground truth data are output through the MEC. Based on trajectory matching in positioning accuracy, the time deviation between two points with similar locations is identified and averaged. Internal processing logic is optimized on the high-definition camera side, and the radar and video fusion algorithm fusion strategy is optimized on the MEC side to reduce data processing and fusion time. The impact of holographic intersection visualization and other business services, as well as disk I / O, on CPU utilization is reduced.

[0302] Target frame drop rate optimization: By recording 5-10 minutes of structured data output from MEC during peak hours, the appearance and disappearance timestamps of each ptcId were statistically analyzed. The theoretical frame count for each ptcId was calculated based on a 10Hz refresh rate, and the actual frame count for that ptcId was also recorded. The frame drop rate was then calculated based on all ptcIds. During data recording, excessive CPU load caused frame drops. By optimizing the device access module and middleware, the CPU load was reduced, thus preventing frame drops.

[0303] Optimization of Target ID Loss and Discontinuity: Structured data from MEC output is recorded for 5-10 minutes during peak hours. The intersection is divided into an inner zone (stop line area) and an outer zone (extending 10 meters outwards). Under normal vehicle traffic conditions, if the target quality is stable, theoretically, the target must first appear outside the outer zone edge, then enter the inner zone, then exit the inner zone, and finally disappear after reaching the outer zone edge. Calculations are as follows: target disappears within the inner zone; target appears within the inner zone; target moves from the outer zone to the inner zone and back to the outer zone, repeating this process more than twice. Optimization is primarily achieved through a trajectory extrapolation algorithm. Specifically, when the fusion perception algorithm stably tracks a target within the ROI range for 5 consecutive frames without output, trajectory extrapolation is performed to extend its lifecycle. However, the number of extrapolated frames is limited to exclude situations where the target leaves the detection area, resulting in substandard target data quality. Therefore, the maximum number of frames is set to 50. The optimization process mainly involves adjusting the number of extrapolated target frames (default 30 frames, dynamically adjusted based on actual conditions for different intersections, ranging from 10-50 frames).

[0304] Another scenario involves a target object being obscured, causing a section of its trajectory to disappear. In this case, the re-identified target object is matched with the trajectory extrapolation data at the disappearance point. The matching is based on a positional deviation threshold (default 2m, dynamically adjusted for different intersections based on actual conditions, ranging from 1-3 meters). Target objects that meet the matching criteria are identified as the same target, and the missing trajectory due to obstruction is completed and an ID is assigned, thereby optimizing the continuity of the ID.

[0305] Optimization of position and heading angle abrupt changes: Structured data from MEC output was recorded for 5-10 minutes during peak hours. Data where the distance between two consecutive frames exceeds 5 meters for position abrupt changes, and where the difference in heading angle between two consecutive frames exceeds 30 degrees for heading angle abrupt changes, were selected. The calculation formula is: Abnormal rate = Abnormal data / Total data volume. During optimization, the measurement noise covariance in the filter parameters was adjusted. A larger measurement noise covariance value indicates less sensitivity to noise, meaning smaller fluctuations in the filtered data, but slower convergence. A suitable measurement noise covariance value was found through ground truth data feedback to achieve optimal filter performance. The fusion sensing algorithm uses a compensation strategy based on abrupt changes in the position and heading angle data of the target in two consecutive frames. The mean value is calculated and compensated for in the next frame to ensure that the abrupt changes in the output data are controlled within the expected range.

[0306] Queue detection optimization: Data and video streams from fused sensing systems during peak hours are recorded. Statistics are compiled for queues of 5 vehicles in a single lane, counting the number of vehicles detected in that lane. Statistics are also compiled for each left-turn and execution lane with 5 vehicles stopped. During optimization, in the target detection phase, the deep learning algorithm is optimized to perform target recognition and localization on high-definition camera images, identifying queued vehicles. Extensive real-world data is used for model training to improve the algorithm's generalization ability and adaptability. Specific improvements and optimizations are made to the algorithm model to address the specific needs of queue detection. Static holding strategy optimization: By determining the vehicle's speed and whether it is decelerating, if the speed is below 5 km / h (at which point millimeter-wave radar has difficulty detecting vehicles and blind spot detection by oblique high-definition cameras is challenging), the vehicle is determined to be about to stop. Based on its historical motion state, the deceleration is calculated to determine its stopping point, completing the target's trajectory and implementing static holding, thereby improving the overall queue detection effect. When the target accelerates again and the speed exceeds 5 km / h, the static holding strategy is canceled. Optimize target recognition detection rate: The method is consistent with the target object perception performance detection rate tuning method.

[0307] Parking detection optimization: By recording fused sensing data and video streams during peak hours, the system confirms whether there is fused sensing data at parking points during the positioning accuracy statistics process. Statistics are collected for each parking instance in each direction. The optimization method is consistent with the target object perception performance detection rate optimization method.

[0308] Lane ID accuracy optimization: Based on 5-10 minutes of structured data output from MEC during peak recording times. Each participant determines their lane ID based on the summarized lane range on the map, comparing it with the lane ID after fusion. During optimization, the loss function parameters are tuned, specifically the internal parameters of the loss function for the lane ID recognition task, to improve recognition accuracy. High-definition cameras and radar are calibrated and verified to ensure the synchronization and accuracy of their perceived data with the actual lanes.

[0309] Target splitting and false target optimization: Based on 10-minute structured data output from MEC peak and off-peak periods, 10-minute ground truth data from all HD cameras at the intersection are locally streamed and stored (storage time is the same as MEC recording time). The number of target splitting and false targets is counted, distinguishing between intersections and lanes for statistical analysis. Trajectory visualization results and video streams can also be displayed simultaneously based on timestamps. During optimization, the parameters of model training are adjusted: By adjusting the model's hyperparameters (such as learning rate, batch size, number of iterations, etc.) and post-processing parameters (such as non-maximum suppression threshold, IOU threshold, etc.), the occurrence of target splitting and false target phenomena can be reduced. ROI shielding: By comparing the output position of false targets from fused perception targets with the HD camera's field of view map, the ROI shielding area of ​​signs and markings is drawn to solve the false target problem caused by false detection. Expanding the ROI of HD cameras: The main area of ​​fusion is typically large vehicles on roads. Millimeter-wave radar itself has this problem with large vehicle detection. By expanding the ROI of HD cameras, the system can detect vehicles within the road segment and assist in determining the type and quantity of targets, improving the confidence level of HD cameras in identifying target types and quantities. Targets that cannot be matched by radar will have their lifespan reduced and their data discarded, thus solving the road segment fusion problem. Trajectory extrapolation by binding with map information: Fragmented areas also exist in right-turn areas within intersections. This is because right-turn areas are coverage blind spots or the location information of targets detected by the equipment is too large, leading to fusion failures and fragmentation. Trajectory extrapolation by binding with map information fills in the gaps in coverage blind spots and addresses the problem of insufficient accuracy in target location information identified by sensors. Specifically, a Kalman filter algorithm is used to smoothly output the target's trajectory based on the heading angle and velocity of the target's movement, confined to the lane centerline on the map. This improves the success rate of target fusion and solves the fusion problem in right-turn areas within intersections.

[0310] Target position jitter optimization: Based on 5-10 minutes of structured data output from MEC during peak recording times, stopped vehicles are selected, and the changes in latitude and longitude displacement of the target during the parking process are statistically analyzed. During optimization, the parameters of the model training are adjusted: by adjusting the model's hyperparameters (such as learning rate, batch size, number of iterations, etc.) and post-processing parameters (such as non-maximum suppression threshold, IOU threshold, etc.). This can reduce target splitting and false target phenomena. The method is consistent with the optimization methods for localization and heading angle abrupt changes. Inter-frame time variation optimization: Based on 5-10 minutes of structured data output from MEC during peak recording times, calculations are performed on an intersection-by-intersection basis, calculating the difference between the timestamps of the data emitted by MEC and the timestamps of the data emitted between two frames. During optimization, it is checked whether the frequency configuration parameter of the data output section is a fixed 10Hz; if a deviation exists, the program logic is reviewed and checked.

[0311] Perception Range Optimization: Based on 5-10 minutes of structured data output from the MEC during peak recording times. The intersection center point is selected using a high-precision map, and the boundaries of fused perception data for various participants in each direction are statistically analyzed to distinguish between entrances and exits and calculate the perception range. During optimization, the installation positions and angles of the high-definition cameras and radar are adjusted: based on on-site measurement data, the installation positions and angles of the high-definition cameras and radar are adjusted to meet the perception range requirements. After all target perception performance indicators are optimized, the data collection and calculation process for each performance indicator is repeated to verify that the optimized output data meets expectations.

[0312] In the description of this application, it should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. For ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0313] The above embodiments are provided for those skilled in the art to implement or use this application. Those skilled in the art can make various modifications or changes to the above embodiments without departing from the spirit of this application. Therefore, the scope of protection of this application is not limited to the above embodiments, but should be the maximum scope that conforms to the innovative features mentioned in the claims.

Claims

1. A vehicle-road cooperative roadside signal processing device, characterized in that, include: The sensing device group includes high-definition cameras, lidar, millimeter-wave radar and traffic light collectors deployed on the roadside. The sensing device group collects video streams, radar point cloud data, structured traffic data and traffic light status data. The communication equipment group includes an RSU module, which is connected to the intelligent connected vehicle and the edge computing node, and transmits vehicle location, driving intention, warning information and control commands to the intelligent connected vehicle and the edge computing node. Edge computing nodes include data management modules, fusion sensing modules, application function modules, and system management modules. The data management module receives video streams, radar point cloud data, structured traffic data, and traffic light status data transmitted by the sensing device group, and processes the video streams, radar point cloud data, traffic light status data, and structured traffic data to achieve data interaction. The fusion perception module generates traffic participant data based on the video stream, radar point cloud data, structured traffic data, and traffic light status data processed by the data management module, and on a multi-sensor data fusion algorithm. The traffic participant data includes traffic participants and their trajectories. The application function module performs ROI partitioning and labeling on the traffic participant data generated by the fusion perception module, and generates event detection analysis results and traffic flow analysis results based on the traffic participant data and the set event judgment conditions. The system management module monitors device status, manages log hierarchy, implements crash restart strategies, and provides OTA update functionality.

2. The vehicle-road cooperative roadside signal processing device according to claim 1, characterized in that, The high-definition camera is installed on the boom of a composite pole 20-30 meters away from the stop line. The detection range covers the entrance and exit lanes of the intersection. The installation height is 6 meters, and the lens is focused and adjusted to the blind zone of 15-20 meters under the pole. The lidar includes a solid-state lidar and a blind spot radar. The blind spot radar covers an area with a radius of 30 meters, and the radar point cloud data frame rate is ≥10Hz. The millimeter-wave radar is vertically mounted and outputs target type, velocity, and structured data at a frame rate of ≥10Hz.

3. The vehicle-road cooperative roadside signal processing device according to claim 1, characterized in that, Based on traffic participant data and set event determination criteria, the event detection and analysis results are generated, including: Conditions for determining an intersection congestion event: When the queue of vehicles is no less than 3 vehicles during two red lights in a lane phase, and the exit lane overflows, and the average speed of vehicles on the road segment is less than 20% of the speed limit, the event detection and analysis result is a current road congestion event; Conditions for determining a pedestrian or non-motorized vehicle intrusion event: When it is determined that a pedestrian or non-motorized vehicle has entered the intersection or motor vehicle lane and the dwell time is not less than 3 seconds, the event detection and analysis result is a pedestrian or non-motorized vehicle intrusion event. Conditions for determining a solid line crossing event: When it is determined that a vehicle has been driving over the solid line for no less than 3 seconds or has changed lanes by crossing the solid line, the event detection and analysis result is a solid line crossing event. Conditions for identifying abnormal parking events: When it is determined that parking in a no-parking zone for no less than 30 seconds constitutes an abnormal parking event; Conditions for judging abnormal low-speed events: When the number of vehicles judged is no more than 3 and the vehicle speed is less than 40% of the current road speed limit for no less than 10 seconds, or when the number of vehicles judged is greater than 3 and the vehicle speed is less than 40% of the current traffic flow average speed for no less than 10 seconds, the event detection and analysis result is an abnormal low-speed event. Conditions for determining a wrong-way driving event: When the vehicle trajectory deviates from the lane direction by more than 90° and lasts for no less than 3 seconds, the event detection and analysis result is a wrong-way driving event; Conditions for determining a speeding event: When the vehicle speed exceeds the current road speed limit by more than 10% and lasts for no less than 3 seconds, the event detection and analysis result is a speeding event.

4. The vehicle-road cooperative roadside signal processing device according to claim 1, characterized in that, Traffic flow analysis results include: The number of vehicles passing through a designated section of a road per unit of time; The distance a vehicle travels in a unit of time; The number of vehicles per unit length of road; The difference between the actual travel time required for a vehicle to pass through an intersection under obstructed conditions and the time required to travel the same distance normally; The length of a vehicle queue from the stop line at the intersection or the start of the queue to the end of the queue; The vehicle's current traffic situation; Based on the average delay of vehicles passing through the intersection, traffic conditions are divided into four levels: severe congestion, moderate congestion, light congestion, and smooth traffic, to evaluate the current traffic situation of vehicles.

5. A vehicle-road cooperative roadside signal processing method, implemented by the vehicle-road cooperative roadside signal processing device as described in any one of claims 1-4, comprising: Step S101: Design parameter configuration, and install the corresponding sensing device according to the parameter configuration; Step S102: Collect video streams, radar point cloud data, structured traffic data and traffic light status data detected by the sensing device, and perform data interaction on the video streams, radar point cloud data, structured traffic data and traffic light status data. Step S103: Generate traffic participant data based on a multi-sensor data fusion algorithm. The traffic participant data includes traffic participants and their trajectories. Step S104: Perform ROI partitioning and labeling on the traffic participant data, and generate event detection analysis results and traffic flow analysis results based on the traffic participant data and event judgment conditions; Among them, based on edge computing algorithms, parameter configuration, sensing devices, ROI labeling, and fusion algorithms are iteratively optimized.

6. The vehicle-road cooperative roadside signal processing method according to claim 5, characterized in that, Based on edge computing algorithms, iterative optimizations were performed on parameter configuration, sensing devices, ROI labeling, and fusion algorithms, including: Step S201, parameter configuration and verification: Configure the relevant parameters in the edge computing node, including network connection parameters, routing table settings, firewall settings, time settings, device information and deployment settings, and complete the verification. Step S202: Install and debug the equipment, and confirm whether the required roadside equipment is complete. The roadside equipment includes high-definition cameras, lidar, millimeter-wave radar, traffic light collectors, brackets, and cables. Step S203: Calibrate the high-definition camera, millimeter-wave radar, and lidar. The calibration of the high-definition camera includes the calibration of the internal parameters of the collaborative intelligent high-definition camera, the calibration of the external parameters of the collaborative intelligent high-definition camera, and the calibration of the auxiliary intelligent high-definition camera. The calibration of the lidar includes the calibration of the external parameters of the lidar calibration. The external parameter calibration of the collaborative intelligent high-definition camera and the external parameter calibration of the lidar calibration are performed simultaneously. Step S204: Divide the ROI region into a region of interest and a shielded region. The region of interest includes the region of interest of the high-definition camera and the region of interest of the millimeter radar. Use ROI annotation to perform lane annotation, lane line annotation, traffic participant detection area annotation, traffic event annotation, and traffic flow analysis annotation on the region of interest of the high-definition camera, the region of interest of the millimeter radar, and the shielded region. Step S205 involves optimizing the target object fusion algorithm, target object perception performance, ROI area, sensitivity, event detection performance, and accuracy of traffic statistics parameters.

7. The vehicle-road cooperative roadside signal processing method according to claim 6, characterized in that, The calibration steps for the external parameters of the collaborative intelligent high-definition camera and the external parameters of the lidar calibration specifically include: selection of acquisition points, RTK point acquisition, acquisition of calibration images, point calibration, latitude and longitude fusion, and completion of calibration. The auxiliary intelligent high-definition camera calibration adopts a multimodal data mapping calibration method based on static feature points to achieve high-precision alignment between the image pixel coordinate system and the geographic coordinate system; The millimeter-wave radar calibration specifically includes: acquiring the coordinate system of the millimeter-wave radar and the world coordinate system for multiple identical targets, and calculating and establishing the mapping relationship between the world coordinate system and the millimeter-wave radar coordinate system through these data, so as to determine the latitude and longitude and the north deflection angle of the world coordinate system of the millimeter-wave radar.

8. The vehicle-road cooperative roadside signal processing method according to claim 6, characterized in that, The ROI region is divided into the region of interest and the masked region, specifically including: Step S2041: Divide the ROI region into the region of interest of the high-definition camera, the region of interest of the millimeter radar, and the shielded area, so that the sensing range of the sensing devices overlaps in the center of the intersection or the critical area; wherein, when the sensing accuracy of the high-definition camera is lower than that of the radar, causing one or more of the following situations in the fusion result: jump, back and forth jitter, splitting, the far-end detection range of the high-definition camera is reduced or a partition selection method is adopted. Step S2042: Draw the Region of Interest (ROI) of the high-definition camera, the millimeter radar, and the shielded area based on the image or map to limit the effective area of ​​the data. Step S2043: Mark traffic participants within the defined effective area; wherein the rules for marking traffic participants within the defined effective area specifically include: Priority is set for effective regions. Regions of interest drawn on the same sensing device can overlap. If they overlap, the priority of the effective regions is used to determine the region. When a sensing device does not define a region of interest, traffic participants who do not fall within any shielded area will be included in the fusion by default; When a sensing device defines a region of interest, traffic participants that do not fall within any shielded area will be discarded by default. On the same sensing device, ROI regions drawn based on maps and ROI regions drawn based on images will be mixed together for judgment according to priority.

9. The vehicle-road cooperative roadside signal processing method according to claim 6, characterized in that, Lane labeling includes: drawing along the boundary of a single lane within the region of interest to form a closed polygon, and labeling the lane as a motor vehicle lane or a non-motor vehicle lane; Lane markings include: solid lines drawn using polylines in the aforementioned motor vehicle lanes or non-motor vehicle lanes; The annotation of the traffic participant detection area includes: drawing a closed polygon in the annotation screen to indicate that traffic participants in this area will be identified, which is annotated as the region of interest; when there are obstacles in the traffic participant detection area that are prone to false detection, a shielding area is superimposed on the traffic participant detection area to avoid false detection. Traffic event labeling includes: labeling traffic events within the region of interest. Traffic events include intersection congestion events, pedestrian or non-motorized vehicle crossing the intersection, crossing solid lines, abnormal parking, abnormally slow speeding, driving against traffic, and speeding. The judgment parameters for traffic event labeling include: The parameters for labeling intersection congestion events are set as follows: the lower limit of vehicle speed is set to 20 km / h, and the alarm threshold is set to: the number of vehicles with a speed lower than 20 km / h accounts for 20%. The event labeling parameter for pedestrians or non-motorized vehicles entering the intersection is set to a time threshold of 3 seconds. The event annotation parameter for the solid line is set to a time threshold of 3 seconds. The abnormal parking event labeling parameter is set to a time threshold of 30 seconds. The abnormal low-speed event labeling parameters are set as follows: the lower speed limit is set to 10km / h, the time threshold is set to 10s, and the alarm threshold is set to: the number of vehicles with a speed lower than 10km / h accounts for 40%. The parameters for labeling retrograde events are set to a time threshold of 3 seconds. The parameters for labeling speeding events are set as follows: the lower speed limit is 40 km / h, the time threshold is 3 seconds, and the alarm threshold is set to 10% of the vehicles exceeding 40 km / h. Traffic flow analysis annotations include: extracting average vehicle speed, density, delay, congestion status, and queue length within the region of interest.

10. The vehicle-road cooperative roadside signal processing method according to claim 6, characterized in that, Optimizing the fusion algorithm for the target objects includes: The raw data output by the sensing devices are distinguished and observed using different colors. An overlap threshold is set to determine whether the overlap of the detection bounding boxes of the same traffic participant output by multiple sensing devices is within the overlap threshold. If it is within the overlap threshold, the data processed by the fusion algorithm is then judged. Based on the data processed by the fusion algorithm, observe the process of a vehicle waiting at the red light before the stop line and entering the fusion zone in the center of the intersection, or observe the traffic flow, observe the handover between the output data of the high-definition camera and the output data of the radar, and observe whether there are vehicle splitting and identity switching errors. If there are no errors, end the target object fusion algorithm optimization. Optimizing the target object sensing performance includes: The system optimizes the target positioning accuracy, detection rate, accuracy, target size accuracy, system latency, target frame loss rate, loss and discontinuity of perceived target ID, number of abrupt changes in positioning and heading angle, queue detection, parking detection, lane ID accuracy, target splitting, false targets, target position jitter, inter-frame time variation, and perception range. Optimization of ROI region adjustment, sensitivity adjustment, and event detection performance includes: Adjust the ROI (Region of Interest) range for image or video processing to exclude interference from irrelevant traffic participants; Set a sensitivity level of 5, and adjust the sensitivity level according to the false alarm or missed alarm ratio; In simulated or real-world scenarios, traffic events are triggered via OBU, MEC, or cloud platform. Expected detection and accuracy values ​​are set according to requirements. The detection and accuracy values ​​in the triggered traffic events are calculated and compared with the set expected detection and accuracy values. If they match, the optimization ends. The detection rate is calculated as follows: (Number of successfully detected events / Number of actual triggered events) × 100%. Accuracy = (Number of correctly detected events / Total number of detected events) × 100% Optimizing traffic flow detection specifically includes: collecting MEC structured traffic flow data during peak hours, outputting the actual traffic flow and the traffic flow output by MEC for the same time period, and calculating the traffic flow accuracy. Wherein, traffic flow accuracy = |actual traffic volume - MEC output traffic volume| / actual traffic volume The calculated accuracy is compared with the set expected value for traffic flow accuracy. If it meets the set expected value, the traffic flow detection optimization ends.

Images