Perceptual data communication method and device based on vehicle infrastructure cooperation, medium and equipment
By combining high-precision maps and camera intrinsic and extrinsic parameters to dynamically generate image masks in the vehicle-road cooperative system, the problem of mask misalignment caused by the explosive growth of perception data and camera pose drift is solved. This achieves efficient data transmission and low-latency communication, reduces system maintenance costs, and improves the safety and efficiency of autonomous driving.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-17
- Publication Date
- 2026-04-03
AI Technical Summary
The explosive growth of perception data in vehicle-road cooperative systems leads to communication bandwidth bottlenecks, while static ROI methods suffer from problems such as mask misalignment, frequent manual calibration, and high maintenance costs caused by camera pose drift.
By combining high-precision maps and camera intrinsic and extrinsic parameters, the current image mask is dynamically generated. The difference between the theoretical feature set and the observed feature set determines whether to update the camera extrinsic parameters. The video frames are encoded and transmitted through the image mask, retaining only the key information related to driving decisions.
It effectively solves the mask misalignment problem caused by camera pose drift, eliminates the need for frequent manual calibration, reduces system maintenance costs, eliminates safety blind spots, significantly reduces redundant data transmission, ensures that communication latency meets the requirements of L2+/L3 level autonomous driving, breaks through the bandwidth bottleneck, and provides support for the large-scale deployment of vehicle-road cooperative systems.
Smart Images
Figure CN121789476A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of vehicle technology, and more specifically, to a perception data communication method, device, medium, and equipment based on vehicle-road cooperation. Background Technology
[0002] With the rapid development of autonomous driving technology, vehicle-to-infrastructure (V2I) systems have become crucial for improving driving safety and efficiency. However, communication bandwidth has become a core bottleneck restricting their large-scale deployment and application.
[0003] (1) Explosive growth of sensing data: Modern Roadside Units (RSUs) typically integrate multiple high-definition cameras (such as 4-8 channels of 1080P / 4K), LiDAR, and millimeter-wave radar, generating massive amounts of raw data. Taking a camera as an example, the raw data stream of a single 1080P video at 30fps can reach 288Mbps, and the total data from 8 cameras exceeds 2.3Gbps, which far exceeds the typical uplink bandwidth of current 5G / C-V2X networks (usually 50-100Mbps).
[0004] (2) Stringent real-time requirements: L2+ / L3 level autonomous driving applications require end-to-end perception latency of less than 100 milliseconds (ms), of which the latency budget allocated to the communication link should be less than 50ms. When bandwidth is limited, the latency of the full data transmission scheme often exceeds 200ms, which cannot meet the basic safety requirements of autonomous driving.
[0005] (3) Extremely high information redundancy: In the raw data collected by roadside sensors, dynamic targets (such as vehicles and pedestrians) and drivable area information that are directly related to driving decisions account for only 20-30% of the total data. The remaining 70-80% is static background information, such as the sky, building facades, and distant green belts. Directly transmitting this information results in a serious waste of bandwidth resources.
[0006] Existing technologies attempt to transmit only lane areas by manually calibrating fixed Regions of Interest (ROIs). However, this method assumes the physical world is absolutely static. In reality, roadside lampposts are affected by strong winds, thermal expansion and contraction, or vehicle vibrations, causing slight drifts in camera pose (extrinsic parameters). Therefore, once the camera moves even slightly, the preset static ROI mask will be misaligned with the actual road surface (e.g., misidentifying and discarding real lane lines as background). This necessitates frequent manual recalibration on-site, resulting in extremely high maintenance costs and a significant blind spot between the occurrence of drift and manual intervention. Summary of the Invention
[0007] This application provides a perception data communication method, device, medium, and equipment based on vehicle-road cooperation, which can solve the communication bandwidth bottleneck caused by the explosive growth of perception data and high information redundancy in vehicle-road cooperative systems, as well as the problems of mask misalignment, frequent manual calibration, high maintenance costs, and security blind spots caused by camera pose drift in existing static ROI methods.
[0008] The specific technical solution is as follows: In a first aspect, embodiments of this application provide a perception data communication method based on vehicle-road cooperation, the method comprising: Based on the coordinates of each traffic element in the world coordinate system, the camera intrinsic parameters, and the current camera extrinsic parameters in the high-precision map, a theoretical feature set is determined. The theoretical feature set includes the projection points obtained by projecting each traffic element from the world coordinate system to the image coordinate system. Generate the current image mask based on the theoretical feature set and the high-precision map; Feature extraction is performed on the current video frame to obtain an observation feature set, which includes the edge point set of the actual traffic elements contained in the current video frame; Find the nearest neighbor of each projection point in the theoretical feature set in the observed feature set, and determine the current state error based on the difference between each projection point and its nearest neighbor. If the current state error is less than or equal to a preset safety threshold, the current video frame is encoded based on the current image mask so that the encoded current video frame can be transmitted to the vehicle. If the current state error is greater than the preset safety threshold, the current camera extrinsic parameters are updated, and the updated camera extrinsic parameters are used as the new current camera extrinsic parameters. Then, the execution steps are returned to determine the theoretical feature set based on the coordinates of each traffic element in the world coordinate system in the high-precision map, the camera intrinsic parameters, and the current camera extrinsic parameters.
[0009] In one possible implementation, determining the current state error based on the difference between each projection point and its nearest neighbor includes: Calculate the average Euclidean distance between each projection point and its nearest neighbor, and use the average Euclidean distance as the current state error.
[0010] In one possible implementation, video encoding of the current video frame based on the current image mask includes: The quantization parameters of macroblocks in video coding are adjusted based on the current image mask, and video coding is performed according to the adjusted quantization parameters.
[0011] In one possible implementation, adjusting the quantization parameters of macroblocks in video coding based on the current image mask includes: The formula is updated based on preset parameters to adjust the quantization parameters of macroblocks in video encoding. The preset parameter update formula includes: ; in, This indicates the adjusted quantization parameters. Represents the basic quantization parameters, the The background compression penalty coefficient is represented by the... This represents the current image mask.
[0012] In one possible implementation, updating the current camera extrinsic parameters to obtain the updated camera extrinsic parameters as the new current camera extrinsic parameters includes: With minimizing the matching error between the projection point and the nearest neighbor point as the optimization objective, an objective function containing a robust noise suppression mechanism is constructed. The objective function is solved, the optimal solution is used as the updated camera extrinsic parameters, and the updated camera extrinsic parameters are used as the new current camera extrinsic parameters.
[0013] In one possible implementation, the camera extrinsic parameters include a rotation matrix and a translation matrix, and the objective function includes: ; in, The projection function is represented by K, which represents the camera intrinsic parameter. This represents the coordinates of the i-th traffic element point in the world coordinate system. Represents any point in the set of observed features. The set of observed features represents the set of features. Let N represent the number of traffic element points, and let R and T represent the rotation matrix and translation matrix of the camera's position in the world coordinate system, respectively, and these are the parameters to be solved in the objective function. The initial value of R is the rotation matrix before the update, and the initial value of T is the translation matrix before the update. Represents the updated rotation matrix, the This represents the updated translation matrix.
[0014] Secondly, embodiments of this application provide a perception data communication device based on vehicle-road cooperation, the device comprising: The first determining unit is used to determine a theoretical feature set based on the coordinates of each traffic element in the world coordinate system, the camera intrinsic parameters, and the current camera extrinsic parameters in the high-precision map. The theoretical feature set includes projection points obtained by projecting each traffic element from the world coordinate system to the image coordinate system. The generation unit is used to generate a current image mask based on the theoretical feature set and the high-precision map; The extraction unit is used to extract features from the current video frame to obtain an observation feature set, which includes the edge point set of the actual traffic elements contained in the current video frame. A searching unit is used to find the nearest neighbor of each projection point in the theoretical feature set within the observed feature set; The second determining unit is used to determine the current state error based on the difference between each projection point and its nearest neighbor. The encoding unit is used to encode the current video frame based on the current image mask when the current state error is less than or equal to a preset safety threshold, so as to transmit the encoded current video frame to the vehicle. The update unit is used to update the current camera extrinsic parameters when the current state error is greater than the preset safety threshold, obtain the updated camera extrinsic parameters as the new current camera extrinsic parameters, and return to the execution step to determine the theoretical feature set based on the coordinates of each traffic element in the world coordinate system, the camera intrinsic parameters, and the current camera extrinsic parameters in the high-precision map.
[0015] In one possible implementation, the second determining unit is configured to calculate the average Euclidean distance between each projection point and its nearest neighbor, and use the average Euclidean distance as the current state error.
[0016] In one possible implementation, the encoding unit includes: The adjustment module is used to adjust the quantization parameters of macroblocks in video coding based on the current image mask; The encoding module is used to encode the video based on the adjusted quantization parameters.
[0017] In one possible implementation, the adjustment module is used to adjust the quantization parameters of macroblocks in video coding according to a preset parameter update formula. The preset parameter update formula includes: ; in, This indicates the adjusted quantization parameters. Represents the basic quantization parameters, the The background compression penalty coefficient is represented by the... This represents the current image mask.
[0018] In one possible implementation, the updating unit includes: The construction module is used to construct an objective function containing a robust noise suppression mechanism with the goal of minimizing the matching error between the projection point and the nearest neighbor point; The solution module is used to solve the objective function and use the optimal solution as the updated camera extrinsic parameters; The determination module is used to use the updated camera extrinsic parameters as the new current camera extrinsic parameters.
[0019] In one possible implementation, the camera extrinsic parameters include a rotation matrix and a translation matrix, and the objective function includes: ; in, The projection function is represented by K, which represents the camera intrinsic parameter. This represents the coordinates of the i-th traffic element point in the world coordinate system. Represents any point in the set of observed features. The set of observed features represents the set of features. Let N represent the number of traffic element points, and let R and T represent the rotation matrix and translation matrix of the camera's position in the world coordinate system, respectively, and these are the parameters to be solved in the objective function. The initial value of R is the rotation matrix before the update, and the initial value of T is the translation matrix before the update. Represents the updated rotation matrix, the This represents the updated translation matrix.
[0020] Thirdly, embodiments of this application provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any possible implementation of the first aspect.
[0021] Fourthly, embodiments of this application provide an electronic device, which includes: One or more processors; The processor is coupled to a storage device for storing one or more programs; When one or more programs are executed by one or more processors, the electronic device performs the method as described in any possible implementation of the first aspect.
[0022] Fifthly, embodiments of this application provide a computer program product containing instructions that, when executed on a computer or processor, cause the computer or processor to perform the method described in any possible implementation of the first aspect.
[0023] As can be seen from the above solutions, the perception data communication method, device, medium, and equipment based on vehicle-road cooperation provided in this application effectively solves the problem of mask misalignment with the real road surface caused by camera pose drift in existing static ROI methods by dynamically generating the current image mask by combining high-precision maps and camera intrinsic and extrinsic parameters, and determining whether to update the camera extrinsic parameters based on the difference between the theoretical feature set and the observed feature set. This eliminates the need for frequent manual on-site calibration, significantly reducing system maintenance costs, and eliminating the safety blind spot between drift and manual intervention. In addition, by encoding and transmitting video frames through image masks, only key information related to driving decisions is retained, significantly reducing the amount of redundant data such as static backgrounds transmitted, greatly saving communication bandwidth, ensuring that the communication latency meets the requirements of L2+ / L3 level autonomous driving, successfully breaking through the bandwidth bottleneck, providing strong support for the large-scale deployment and application of vehicle-road cooperative systems, and further improving the safety and operating efficiency of autonomous driving.
[0024] Furthermore, the technical effects that can be achieved by the embodiments of this application include: 1. By calculating the average Euclidean distance between each projection point in the theoretical feature set and the corresponding nearest neighbor point in the observation feature set, the current state error is accurately quantified, realizing an objective and accurate determination of the degree of camera pose drift, and optimizing the dynamic adaptation capability from the core logic.
[0025] 2. By combining the macroblock quantization parameters with the current image mask Precise association enables differentiated allocation of coding resources: when a macroblock is located in a critical region within the mask (such as a vehicle or drivable area), The value of is 1, at which point The quantization parameter is set to 0, maintaining its base value. This ensures the encoding accuracy and detail integrity of critical sensing information, preventing the loss of core data due to excessive compression; when macroblocks are located in static background areas outside the mask, The value is 0, and the quantization parameter is superimposed with the background compression penalty coefficient. This significantly improves quantization to deeply compress redundant data. This precisely adapted differentiated coding method not only ensures the accuracy of the perception information required for L2+ / L3 level autonomous driving, but also maximizes the compression of the large proportion of background redundant data, saving communication bandwidth more efficiently. At the same time, the targeted adjustment of coding parameters improves coding efficiency, helping the communication latency meet the stringent requirements of autonomous driving. Furthermore, this solution works in conjunction with the previous dynamic mask generation logic to completely solve the problem of static ROI misalignment, reduce the maintenance cost of manual calibration, eliminate safety blind spots, and achieve refined utilization of bandwidth resources. This provides precise coding-level support for the stable and efficient operation of the vehicle-road cooperative system, further improving the safety and overall operational efficiency of autonomous driving.
[0026] 3. By minimizing the matching error between the projection point and the nearest neighbor point as the core optimization objective, and combining a robust noise suppression mechanism to construct an objective function and solve for the optimal solution to update the camera extrinsic parameters, the accuracy and anti-interference capabilities of extrinsic parameter adjustment are improved simultaneously. The robust noise suppression mechanism can effectively filter outliers and pseudo-errors caused by sensor noise and environmental interference during feature extraction, avoiding such interference from misleading the calculation of matching error. This makes the error information on which the extrinsic parameters are updated more pure and reliable, significantly improving the stability and accuracy of extrinsic parameter estimation. At the same time, the optimization logic guided by minimizing matching error can specifically correct camera pose drift caused by strong winds, thermal expansion and contraction, or vehicle vibrations on roadside lampposts. This ensures that the updated extrinsic parameters accurately match the current actual installation status, ensuring that the image mask generated based on the high-precision map and extrinsic parameters always highly matches the real road surface. This fundamentally eliminates the problem of key information loss or redundant data mistransmission caused by mask misalignment in static ROIs. This intelligent external parameter self-updating mechanism not only completely eliminates the reliance on frequent manual calibration, significantly reducing system operation and maintenance costs, but also eliminates the safety blind spot between drift and manual intervention, ensuring the continuity and accuracy of perception data transmission. In synergy with subsequent differential quantization parameter encoding based on precise masks, it maximizes the filtering of a large proportion of static redundant data, efficiently saving communication bandwidth, while ensuring the transmission quality of core perception information. This allows communication latency to meet the stringent requirements of L2+ / L3 level autonomous driving, providing key technical support for the large-scale stable deployment of vehicle-road cooperative systems and further improving the operational safety and overall efficiency of autonomous driving. Attached Figure Description
[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0028] Figure 1 A flowchart illustrating a vehicle-road cooperative sensing data communication method provided in an embodiment of this application; Figure 2 This is a block diagram of a vehicle-road cooperative sensing data communication device provided in an embodiment of this application. Detailed Implementation
[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.
[0030] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.
[0031] This application provides a perception data communication method based on vehicle-road cooperation. This method can be applied to electronic devices, specifically roadside devices. Figure 1 As shown, the method includes: S110: Determine the theoretical feature set based on the coordinates of each traffic element in the world coordinate system, the camera intrinsic parameters, and the current camera extrinsic parameters in the high-precision map.
[0032] The camera extrinsic parameters include rotation and translation matrices. The theoretical feature set includes projection points obtained by projecting each traffic element from the world coordinate system to the image coordinate system.
[0033] Assume the world coordinates of a traffic element (such as a point on a lane line) in a high-definition map are: The camera's intrinsic parameter matrix is K, and the current rotation matrix is... The translation matrix is .
[0034] Based on the pinhole camera model, the pixel coordinates of this point in the image plane are... The calculation is as follows: ; Where s is the scale factor, u represents the pixel position in the horizontal direction, and v represents the pixel position in the vertical direction.
[0035] S120: Generate the current image mask based on the theoretical feature set and the high-precision map.
[0036] The roadside equipment combines the actual range of traffic elements in the high-precision map (such as the width of lane lines and the boundary of drivable areas) with the projection position in the theoretical feature set to delineate "key areas" on the image. These are areas in the image that match the projection points and correspond to traffic elements in the high-precision map (such as areas that cover the projection points of lane lines on pixel coordinates). These areas are marked as mask 1 (key areas that need to be retained), while other static backgrounds that do not correspond to any traffic elements (such as the sky and distant green belts) are marked as mask 0 (redundant areas that need to be filtered). Finally, a current image mask adapted to the current scene is generated.
[0037] S130: Extract features from the current video frame to obtain the set of observed features.
[0038] The observation feature set includes the edge point set of actual traffic elements (such as lanes) contained in the current video frame. Roadside equipment can extract the edge point set of actual traffic elements by performing edge detection (such as Canny) or semantic segmentation on the current video frame.
[0039] S140: Find the nearest neighbor of each projection point in the theoretical feature set in the observed feature set, and determine the current state error based on the difference between each projection point and its nearest neighbor.
[0040] After obtaining the observed feature set and the theoretical feature set, the roadside equipment can calculate the average Euclidean distance between each projected point and its nearest neighbor, and use the average Euclidean distance as the current state error.
[0041] This process can be represented by the following formula: ; in, This represents the current state error, and N represents the number of traffic element points. Represents the set of observed features. This represents the i-th projection point. It represents any point in the set of observed features.
[0042] S150: If the current state error is less than or equal to the preset safety threshold, the current video frame is encoded based on the current image mask so that the encoded current video frame can be transmitted to the vehicle.
[0043] The preset safety threshold can be determined based on practical experience.
[0044] If the current state error is less than or equal to a preset safety threshold, it indicates that the current camera pose is stable, and the current video frame can be encoded using the current image mask. The specific implementation method for encoding the current video frame based on the current image mask includes: adjusting the quantization parameters of macroblocks in video encoding based on the current image mask, and then performing video encoding based on the adjusted quantization parameters.
[0045] Specifically, roadside equipment can update formulas based on preset parameters to adjust the quantization parameters of macroblocks in video encoding; The preset parameter update formulas include: ; in, This indicates the adjusted quantization parameters. Indicates the basic quantization parameters, Indicates the background compression penalty coefficient (e.g., ), This represents the current image mask.
[0046] In addition, video encoding methods can include H.264, H.265, AV1, etc.
[0047] S160: If the current state error is greater than the preset safety threshold, update the current camera extrinsic parameters, obtain the updated camera extrinsic parameters as the new current camera extrinsic parameters, and return to step S110.
[0048] If the current state error is greater than the preset safety threshold, it indicates that the pose of the current camera has drifted. If the current image mask is used for video encoding, errors will occur. Therefore, the current camera extrinsic parameters need to be updated first. Based on the updated camera extrinsic parameters, steps S110-S140 are re-executed to obtain a new current image mask and a new current state error. If the new current state error is less than or equal to the preset safety threshold, the current video frame is encoded based on the new current image mask so that the encoded current video frame can be transmitted to the vehicle.
[0049] The method for updating the current camera extrinsic parameters includes: constructing an objective function containing a robust noise suppression mechanism with the goal of minimizing the matching error between the projection point and the nearest neighbor point; solving the objective function; using the optimal solution as the updated camera extrinsic parameters; and using the updated camera extrinsic parameters as the new current camera extrinsic parameters.
[0050] The objective function includes: ; in, Let K represent the projection function, and K represent the camera intrinsic parameters. This represents the coordinates of the i-th traffic element point in the world coordinate system. Represents any point in the set of observed features. Represents the set of observed features. This represents a robust kernel function, such as the Huber Loss function. N represents the number of traffic feature points. R and T are the rotation matrix and translation matrix representing the camera's attitude and position in the world coordinate system, respectively, and are the parameters to be solved in the objective function. The initial value of R is the rotation matrix before the update, and the initial value of T is the translation matrix before the update. This represents the updated rotation matrix. This represents the updated translation matrix.
[0051] The perception data communication method based on vehicle-road cooperation provided in this application dynamically generates the current image mask by combining high-precision maps and camera intrinsic and extrinsic parameters, and determines whether to update the camera extrinsic parameters based on the difference between the theoretical feature set and the observed feature set. This effectively solves the problem of mask misalignment with the real road surface caused by camera pose drift in existing static ROI methods. It eliminates the need for frequent manual on-site calibration, significantly reducing system maintenance costs, and also eliminates the safety blind spot between drift and manual intervention. In addition, by encoding and transmitting video frames through image masks, only key information related to driving decisions is retained, significantly reducing the amount of redundant data such as static backgrounds transmitted, greatly saving communication bandwidth, and ensuring that the communication latency meets the requirements of L2+ / L3 level autonomous driving. This successfully breaks through the bandwidth bottleneck, provides strong support for the large-scale deployment and application of vehicle-road cooperative systems, and further improves the safety and operational efficiency of autonomous driving.
[0052] Furthermore, by calculating the average Euclidean distance between each projection point in the theoretical feature set and the corresponding nearest neighbor point in the observed feature set, the current state error is accurately quantified, thus achieving an objective and accurate determination of the degree of camera pose drift and optimizing the dynamic adaptation capability from the core logic.
[0053] By combining macroblock quantization parameters with the current image mask Precise association enables differentiated allocation of coding resources: when a macroblock is located in a critical region within the mask (such as a vehicle or drivable area), The value of is 1, at which point The quantization parameter is set to 0, maintaining its base value. This ensures the encoding accuracy and detail integrity of critical sensing information, preventing the loss of core data due to excessive compression; when macroblocks are located in static background areas outside the mask, The value is 0, and the quantization parameter is superimposed with the background compression penalty coefficient. This significantly improves quantization to deeply compress redundant data. This precisely adapted differentiated coding method not only ensures the accuracy of the perception information required for L2+ / L3 level autonomous driving, but also maximizes the compression of the large proportion of background redundant data, saving communication bandwidth more efficiently. At the same time, the targeted adjustment of coding parameters improves coding efficiency, helping the communication latency meet the stringent requirements of autonomous driving. Furthermore, this solution works in conjunction with the previous dynamic mask generation logic to completely solve the problem of static ROI misalignment, reduce the maintenance cost of manual calibration, eliminate safety blind spots, and achieve refined utilization of bandwidth resources. This provides precise coding-level support for the stable and efficient operation of the vehicle-road cooperative system, further improving the safety and overall operational efficiency of autonomous driving.
[0054] By minimizing the matching error between the projection point and the nearest neighbor point as the core optimization objective, and combining a robust noise suppression mechanism to construct an objective function and solve for the optimal solution to update the camera extrinsic parameters, the accuracy and anti-interference capabilities of extrinsic parameter adjustment are improved simultaneously. The robust noise suppression mechanism can effectively filter outliers and pseudo-errors caused by sensor noise and environmental interference during feature extraction, avoiding such interference from misleading the calculation of matching error. This makes the error information on which the extrinsic parameters are updated more pure and reliable, significantly improving the stability and accuracy of extrinsic parameter estimation. At the same time, the optimization logic guided by minimizing the matching error can specifically correct the camera pose drift caused by strong winds, thermal expansion and contraction, or vehicle vibration of the roadside lampposts. This ensures that the updated extrinsic parameters accurately match the current actual installation status, ensuring that the image mask generated based on the high-precision map and extrinsic parameters always highly matches the real road surface. This fundamentally eliminates the problem of key information loss or redundant data mistransmission caused by mask misalignment in static ROIs. This intelligent external parameter self-updating mechanism not only completely eliminates the reliance on frequent manual calibration, significantly reducing system operation and maintenance costs, but also eliminates the safety blind spot between drift and manual intervention, ensuring the continuity and accuracy of perception data transmission. In synergy with subsequent differential quantization parameter encoding based on precise masks, it maximizes the filtering of a large proportion of static redundant data, efficiently saving communication bandwidth, while ensuring the transmission quality of core perception information. This allows communication latency to meet the stringent requirements of L2+ / L3 level autonomous driving, providing key technical support for the large-scale stable deployment of vehicle-road cooperative systems and further improving the operational safety and overall efficiency of autonomous driving.
[0055] Based on the above method embodiments, another embodiment of this application provides a vehicle-road cooperative sensing data communication device. This device is applied to electronic equipment, specifically roadside equipment, such as... Figure 2 As shown, the device includes: The first determining unit 210 is used to determine a theoretical feature set based on the coordinates of each traffic element in the world coordinate system, camera intrinsic parameters and current camera extrinsic parameters in the high-precision map. The theoretical feature set includes projection points obtained by projecting each traffic element from the world coordinate system to the image coordinate system. Generation unit 220 is used to generate a current image mask based on the theoretical feature set and the high-precision map; Extraction unit 230 is used to extract features from the current video frame to obtain an observation feature set, wherein the observation feature set includes the edge point set of the actual traffic elements contained in the current video frame; The searching unit 240 is used to find the nearest neighbor of each projection point in the theoretical feature set in the observed feature set; The second determining unit 250 is used to determine the current state error based on the difference between each projection point and its nearest neighbor. Encoding unit 260 is used to encode the current video frame based on the current image mask when the current state error is less than or equal to a preset safety threshold, so as to transmit the encoded current video frame to the vehicle. The update unit 270 is used to update the current camera extrinsic parameters when the current state error is greater than the preset safety threshold, obtain the updated camera extrinsic parameters as the new current camera extrinsic parameters, and return to the execution step to determine the theoretical feature set based on the coordinates of each traffic element in the world coordinate system, the camera intrinsic parameters, and the current camera extrinsic parameters in the high-precision map.
[0056] In one possible implementation, the second determining unit 250 is used to calculate the average Euclidean distance between each projection point and its nearest neighbor, and to use the average Euclidean distance as the current state error.
[0057] In one possible implementation, the encoding unit 260 includes: The adjustment module is used to adjust the quantization parameters of macroblocks in video coding based on the current image mask; The encoding module is used to encode the video based on the adjusted quantization parameters.
[0058] In one possible implementation, the adjustment module is used to adjust the quantization parameters of macroblocks in video coding according to a preset parameter update formula. The preset parameter update formula includes: ; in, This indicates the adjusted quantization parameters. Represents the basic quantization parameters, the The background compression penalty coefficient is represented by the... This represents the current image mask.
[0059] In one possible implementation, the update unit 270 includes: The construction module is used to construct an objective function containing a robust noise suppression mechanism with the goal of minimizing the matching error between the projection point and the nearest neighbor point; The solution module is used to solve the objective function and use the optimal solution as the updated camera extrinsic parameters; The determination module is used to use the updated camera extrinsic parameters as the new current camera extrinsic parameters.
[0060] In one possible implementation, the camera extrinsic parameters include a rotation matrix and a translation matrix, and the objective function includes: ; in, The projection function is represented by K, which represents the camera intrinsic parameter. This represents the coordinates of the i-th traffic element point in the world coordinate system. Represents any point in the set of observed features. The set of observed features represents the set of features. Let N represent the number of traffic element points, and let R and T represent the rotation matrix and translation matrix of the camera's position in the world coordinate system, respectively, and these are the parameters to be solved in the objective function. The initial value of R is the rotation matrix before the update, and the initial value of T is the translation matrix before the update. Represents the updated rotation matrix, the This represents the updated translation matrix.
[0061] The vehicle-road cooperative perception data communication device provided in this application dynamically generates the current image mask by combining high-precision maps and camera intrinsic and extrinsic parameters, and determines whether to update the camera extrinsic parameters based on the difference between the theoretical feature set and the observed feature set. This effectively solves the problem of mask misalignment with the real road surface caused by camera pose drift in existing static ROI methods. It eliminates the need for frequent manual on-site calibration, significantly reducing system maintenance costs, and also eliminates the safety blind spot between drift and manual intervention. In addition, by encoding and transmitting video frames through image masks, only key information related to driving decisions is retained, significantly reducing the amount of redundant data such as static backgrounds transmitted, greatly saving communication bandwidth, and ensuring that the communication latency meets the requirements of L2+ / L3 level autonomous driving. This successfully breaks through the bandwidth bottleneck, provides strong support for the large-scale deployment and application of vehicle-road cooperative systems, and further improves the safety and operating efficiency of autonomous driving.
[0062] Based on the above method embodiments, another embodiment of this application provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method as described in any of the above embodiments.
[0063] Based on the above method embodiments, another embodiment of this application provides an electronic device or computer device, including: One or more processors; The processor is coupled to a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the electronic device or computer device performs the method as described in any of the above embodiments.
[0064] Based on the above embodiments, another embodiment of this application provides a computer program product, which includes instructions that, when executed on a computer or processor, cause the computer or processor to perform the method described in any of the above embodiments.
[0065] The above-described device and system embodiments correspond to the method embodiments and have the same technical effects. For detailed descriptions, please refer to the method embodiments. The device embodiments are derived from the method embodiments; detailed descriptions can be found in the method embodiments section, and will not be repeated here. Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application.
[0066] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.
[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A perception data communication method based on vehicle-road cooperation, characterized in that, The method includes: Based on the coordinates of each traffic element in the world coordinate system, the camera intrinsic parameters, and the current camera extrinsic parameters in the high-precision map, a theoretical feature set is determined. The theoretical feature set includes the projection points obtained by projecting each traffic element from the world coordinate system to the image coordinate system. Generate the current image mask based on the theoretical feature set and the high-precision map; Feature extraction is performed on the current video frame to obtain an observation feature set, which includes the edge point set of the actual traffic elements contained in the current video frame; Find the nearest neighbor of each projection point in the theoretical feature set in the observed feature set, and determine the current state error based on the difference between each projection point and its nearest neighbor. If the current state error is less than or equal to a preset safety threshold, the current video frame is encoded based on the current image mask so that the encoded current video frame can be transmitted to the vehicle. If the current state error is greater than the preset safety threshold, the current camera extrinsic parameters are updated, and the updated camera extrinsic parameters are used as the new current camera extrinsic parameters. Then, the execution steps are returned to determine the theoretical feature set based on the coordinates of each traffic element in the world coordinate system in the high-precision map, the camera intrinsic parameters, and the current camera extrinsic parameters.
2. The method according to claim 1, characterized in that, The current state error is determined based on the differences between each projected point and its nearest neighbor, including: Calculate the average Euclidean distance between each projection point and its nearest neighbor, and use the average Euclidean distance as the current state error.
3. The method according to claim 1, characterized in that, Video encoding of the current video frame based on the current image mask includes: The quantization parameters of macroblocks in video coding are adjusted based on the current image mask, and video coding is performed according to the adjusted quantization parameters.
4. The method according to claim 3, characterized in that, Adjusting the quantization parameters of macroblocks in video coding based on the current image mask includes: The formula is updated based on preset parameters to adjust the quantization parameters of macroblocks in video encoding. The preset parameter update formula includes: ； in, This indicates the adjusted quantization parameters. Represents the basic quantization parameters, the The background compression penalty coefficient is represented by the... This represents the current image mask.
5. The method according to any one of claims 1-4, characterized in that, Updating the current camera extrinsic parameters to obtain the updated camera extrinsic parameters as the new current camera extrinsic parameters includes: With minimizing the matching error between the projection point and the nearest neighbor point as the optimization objective, an objective function containing a robust noise suppression mechanism is constructed. The objective function is solved, the optimal solution is used as the updated camera extrinsic parameters, and the updated camera extrinsic parameters are used as the new current camera extrinsic parameters.
6. The method according to claim 5, characterized in that, The camera extrinsic parameters include rotation and translation matrices, and the objective function includes: ； in, The projection function is represented by K, which represents the camera intrinsic parameter. This represents the coordinates of the i-th traffic element point in the world coordinate system. Represents any point in the set of observed features. Represents the set of observed features, the Let N represent the number of traffic element points, and let R and T represent the rotation matrix and translation matrix of the camera's position in the world coordinate system, respectively, and these are the parameters to be solved in the objective function. The initial value of R is the rotation matrix before the update, and the initial value of T is the translation matrix before the update. Represents the updated rotation matrix, the This represents the updated translation matrix.
7. A sensing data communication device based on vehicle-road cooperation, characterized in that, The device includes: The first determining unit is used to determine a theoretical feature set based on the coordinates of each traffic element in the world coordinate system, the camera intrinsic parameters, and the current camera extrinsic parameters in the high-precision map. The theoretical feature set includes projection points obtained by projecting each traffic element from the world coordinate system to the image coordinate system. The generation unit is used to generate a current image mask based on the theoretical feature set and the high-precision map; The extraction unit is used to extract features from the current video frame to obtain an observation feature set, which includes the edge point set of the actual traffic elements contained in the current video frame. A searching unit is used to find the nearest neighbor of each projection point in the theoretical feature set within the observed feature set; The second determining unit is used to determine the current state error based on the difference between each projection point and its nearest neighbor. The encoding unit is used to encode the current video frame based on the current image mask when the current state error is less than or equal to a preset safety threshold, so as to transmit the encoded current video frame to the vehicle. The update unit is used to update the current camera extrinsic parameters when the current state error is greater than the preset safety threshold, obtain the updated camera extrinsic parameters as the new current camera extrinsic parameters, and return to the execution step to determine the theoretical feature set based on the coordinates of each traffic element in the world coordinate system, the camera intrinsic parameters, and the current camera extrinsic parameters in the high-precision map.
8. The apparatus according to claim 7, characterized in that, The second determining unit is used to calculate the average Euclidean distance between each projection point and its nearest neighbor, and to use the average Euclidean distance as the current state error.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.
10. An electronic device, characterized in that, The electronic device includes: One or more processors; The processor is coupled to a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the electronic device performs the method as described in any one of claims 1-6.