A space-time quality closed-loop control method for vehicle-road cooperative perception fusion and planning

Abstract

The present application relates to the technical field of intelligent networked vehicles and vehicle-road cooperation, and particularly relates to a time-space quality closed-loop control method for vehicle-road cooperative perception fusion and planning. The method comprises: in a target fusion period, analyzing first perception information of a vehicle-side device and second perception information of a roadside device to determine a perception time deviation, a perception position deviation, a point cloud registration quality loss, and a pose fluctuation degree of the vehicle-side device between the vehicle-side device and the roadside device, and fusing the four to form a fusion vector; then dynamically determining a first fusion weight of the vehicle-side device and a second fusion weight of the roadside device according to the fusion vector; and fusing the first perception information and the second perception information according to the first fusion weight and the second fusion weight to obtain target perception information. The present application can improve the accuracy of vehicle-road cooperative perception information.

Description

Technical Field

[0001] This invention relates to the technical field of intelligent connected vehicles and vehicle-road cooperation, specifically to a spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning. Background Technology

[0002] With the widespread application of intelligent connected vehicles and vehicle-to-everything (V2X) technologies, the collaborative integration of vehicle-side and roadside perception systems has become an important direction for improving traffic safety and environmental perception accuracy. Currently, mainstream research and applications largely focus on the fusion of data from vehicles and roadside sensors (such as LiDAR, cameras, and millimeter-wave radar) to achieve global perception through information sharing. For example, multimodal fusion frameworks based on V2X have been developed both domestically and internationally, improving target detection and tracking accuracy through feature stitching or intermediate feature sharing; some research has also introduced deep learning networks to achieve end-to-end perception result fusion.

[0003] However, most existing methods are based on the assumptions of ideal synchronization and static calibration, which cannot effectively adapt to the ever-changing differences in perception capabilities between vehicle-mounted devices and roadside devices in real dynamic scenarios, resulting in the perception results obtained by fusion that fail to truly reflect road conditions. Summary of the Invention

[0004] The purpose of this invention is to provide a spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning, which solves the technical problem of low accuracy of cooperative perception information obtained by integrating vehicle-side equipment and roadside equipment in the prior art.

[0005] In a first aspect, one embodiment of the present invention provides a spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning, the method comprising:

[0006] Within the target fusion period, the first perception information of the vehicle-mounted device and the second perception information of the roadside device are analyzed to determine the time factor, spatial factor, attitude factor and registration factor. The time factor represents the perception time deviation between the vehicle-mounted device and the roadside device, the spatial factor represents the perception position deviation between the vehicle-mounted device and the roadside device, the attitude factor represents the pose fluctuation of the vehicle-mounted device, and the registration factor represents the point cloud registration quality loss between the vehicle-mounted device and the roadside device.

[0007] The time factor, the spatial factor, the attitude factor, and the registration factor are fused to obtain a fusion vector;

[0008] The first fusion weight of the vehicle-side device and the second fusion weight of the roadside device are determined based on the fusion vector, wherein the first fusion weight is negatively correlated with the L2 norm of the fusion vector, and the sum of the first fusion weight and the second fusion weight is 1.

[0009] Based on the first fusion weight and the second fusion weight, the first perception information and the second perception information are fused to obtain the target perception information.

[0010] In some embodiments, the first sensing information includes multiple vehicle-side sensing timestamps, and the second sensing information includes multiple roadside sensing timestamps.

[0011] The steps for obtaining the time factor include:

[0012] Among the multiple vehicle-side sensing timestamps, the roadside sensing timestamp associated with each vehicle-side sensing timestamp is determined. The roadside sensing timestamp associated with the vehicle-side sensing timestamp corresponds to the smallest timestamp distance among the multiple roadside sensing timestamps. The timestamp distance indicates the time difference between the corresponding vehicle-side sensing timestamp and the corresponding roadside sensing timestamp.

[0013] The time factor is obtained based on the central tendency of the timestamp distance between multiple vehicle-side sensing timestamps and their associated roadside sensing timestamps.

[0014] In some embodiments, the first sensing information includes multiple vehicle-side sensing spatial points, and the second sensing information includes multiple roadside sensing spatial points.

[0015] The steps for obtaining the space factor include:

[0016] Among the multiple vehicle-end sensing spatial points, the nearest roadside spatial point of each vehicle-end sensing spatial point is determined. The nearest roadside spatial point is the roadside sensing spatial point with the smallest spatial distance to the corresponding vehicle-end sensing spatial point among the multiple roadside sensing spatial points.

[0017] The spatial factor is obtained based on the convergence trend of the spatial distance between multiple vehicle-end perception timestamps and their corresponding nearest roadside spatial points.

[0018] In some embodiments, the first sensing information includes multiple vehicle-side sensing spatial points, and the second sensing information includes multiple roadside sensing spatial points.

[0019] The steps for obtaining the registration factor include:

[0020] The multiple vehicle-side sensing spatial points are registered with the multiple roadside sensing spatial points to obtain spatial point registration results.

[0021] The overlap ratio and root mean square error are determined based on the spatial point registration results.

[0022] The registration factor is determined based on the overlap ratio and the root mean square error, wherein the overlap ratio is negatively correlated with the registration factor, and the root mean square error is positively correlated with the registration factor.

[0023] In some embodiments, the step of fusing the time factor, the spatial factor, the attitude factor, and the registration factor to obtain a fused vector includes:

[0024] The time factor is subjected to time standardization to obtain a time component, the space factor is subjected to spatial standardization to obtain a space component, and the attitude factor is subjected to attitude standardization to obtain an attitude component.

[0025] The time component, the spatial component, the attitude component, and the registration factor are concatenated to obtain a fusion vector.

[0026] In some embodiments, the step of determining the first fusion weight of the vehicle-side device and the second fusion weight of the roadside device based on the fusion vector includes:

[0027] Based on the L2 norm of the fusion vector, the first generation weight of the corresponding vehicle-side device and the second generation weight of the corresponding roadside device are determined. The first generation weight is negatively correlated with the L2 norm of the fusion vector, and the second generation weight is positively correlated with the L2 norm of the fusion vector.

[0028] The real-time fusion weight of the vehicle-side device is determined based on the first generation weight, the second generation weight, the initial weight of the vehicle-side device corresponding to the target fusion period, and the initial weight of the roadside device corresponding to the target fusion period.

[0029] The real-time fusion weight of the vehicle-side device is weighted and calculated with the initial weight of the vehicle-side device to obtain the first fusion weight;

[0030] The second fusion weight is determined based on the first fusion weight.

[0031] In some embodiments, the initial weight of the vehicle-side corresponding to the target fusion period is the fusion weight determined within the historical fusion period, and the initial weight of the roadside corresponding to the target fusion period is the fusion weight determined within the historical fusion period, wherein the historical fusion period is the fusion period preceding the target fusion period.

[0032] In some embodiments, the calculation weight of the real-time fusion weight of the vehicle-side device is determined based on the L2 norm of the fusion vector and the registration factor. The calculation weight of the real-time fusion weight of the vehicle-side device is negatively correlated with the L2 norm of the fusion vector, and the calculation weight of the real-time fusion weight of the vehicle-side device is negatively correlated with the registration factor.

[0033] In some embodiments, the step of fusing the first sensing information and the second sensing information according to the first fusion weight and the second fusion weight to obtain target sensing information includes:

[0034] Based on the calculated weight of the real-time fusion weight of the vehicle-end device, a first redundancy factor is determined, wherein the first redundancy factor indicates the difficulty of information fusion between the first sensing information and the second sensing information.

[0035] A second redundancy factor is determined based on the L2 norm of the fusion vector, wherein the second redundancy factor indicates the spatiotemporal alignment quality loss between the first and second sensing information.

[0036] A third redundancy factor is determined based on the weight difference between the first fusion weight and the second fusion weight, wherein the third redundancy factor indicates the spatiotemporal perception capability deviation between the vehicle-mounted equipment and the roadside equipment.

[0037] The initial fusion radius is amplified based on the first redundancy factor, the second redundancy factor, and the third redundancy factor to obtain the target fusion radius;

[0038] Based on the target fusion radius, the first perception information and the second perception information are fused according to the first fusion weight and the second fusion weight to obtain the target perception information.

[0039] In some embodiments, the first redundancy factor is negatively correlated with the calculated weight of the real-time fusion weight of the vehicle-side device, the second redundancy factor is positively correlated with the L2 norm of the fusion vector, and the third redundancy factor is positively correlated with the weight difference value, wherein the weight difference value is the absolute difference between the first fusion weight and the second fusion weight.

[0040] Secondly, another embodiment of the present invention provides a spatiotemporal quality closed-loop control system for vehicle-road cooperative perception fusion and planning, the system comprising:

[0041] The factor determination module is used to analyze the first perception information of the vehicle-mounted device and the second perception information of the roadside device within the target fusion period, and determine the time factor, spatial factor, attitude factor and registration factor. The time factor represents the perception time deviation between the vehicle-mounted device and the roadside device, the spatial factor represents the perception position deviation between the vehicle-mounted device and the roadside device, the attitude factor represents the pose fluctuation degree of the vehicle-mounted device, and the registration factor represents the point cloud registration quality loss between the vehicle-mounted device and the roadside device.

[0042] The factor fusion module is used to fuse the time factor, the spatial factor, the attitude factor, and the registration factor to obtain a fusion vector;

[0043] The weight determination module is used to determine the first fusion weight of the vehicle-side device and the second fusion weight of the roadside device based on the fusion vector, wherein the first fusion weight is negatively correlated with the L2 norm of the fusion vector, and the sum of the first fusion weight and the second fusion weight is 1.

[0044] The information fusion module is used to fuse the first perception information and the second perception information according to the first fusion weight and the second fusion weight to obtain target perception information.

[0045] Thirdly, in another embodiment of the present invention, an electronic device is provided, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method described in the first aspect.

[0046] Fourthly, in another embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0047] The present invention has the following beneficial effects:

[0048] This invention analyzes the first perception information from vehicle-mounted devices and the second perception information from roadside devices within the target fusion cycle. It assesses the unreliability of vehicle-mounted perception information during collaborative road perception from temporal, spatial, vehicle pose, and point cloud registration dimensions. Then, it fuses the quantified values ​​of these four dimensions to form a fusion vector, ensuring the coupling relationship between different dimensional indicators is fully preserved. The L2 norm of the fusion vector is then calculated to quantify the unreliability of the vehicle-mounted perception information, and the fusion weights of vehicle-mounted devices and roadside devices are dynamically adjusted accordingly. When vehicle-mounted perception information exhibits a high error risk, its weight during fusion is dynamically reduced to suppress the contamination of the final fused collaborative perception information by strongly interfering vehicle-mounted perception information. Conversely, when vehicle-mounted perception information exhibits a low error risk, its weight during fusion is dynamically increased to amplify the gain of vehicle-mounted perception information on the final fused collaborative perception information, thereby improving the accuracy and reliability of the collaborative perception information. Attached Figure Description

[0049] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating a spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning provided in an embodiment of the present invention.

[0051] Figure 2 This is a schematic diagram of the structure of a spatiotemporal quality closed-loop control system for vehicle-road cooperative perception fusion and planning provided in an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0053] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0055] The following description, in conjunction with the accompanying drawings, details the specific scheme of the spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning provided by the present invention.

[0056] In one embodiment, the present invention provides a spatiotemporal quality closed-loop control method based on vehicle-road cooperative perception fusion and planning, such as... Figure 1 As shown, the method includes:

[0057] Step S1: Within the target fusion period, analyze the first perception information of the vehicle-side equipment and the second perception information of the roadside equipment to determine the time factor, spatial factor, attitude factor and registration factor.

[0058] Wherein, the time factor represents the perception time deviation between the vehicle-mounted device and the roadside device, the spatial factor represents the perception position deviation between the vehicle-mounted device and the roadside device, the attitude factor represents the pose fluctuation of the vehicle-mounted device, and the registration factor represents the point cloud registration quality loss between the vehicle-mounted device and the roadside device.

[0059] The aforementioned vehicle-mounted equipment can be understood as sensing devices mounted on vehicles in vehicle-road cooperative sensing scenarios (such as vehicle-mounted cameras, vehicle-mounted LiDAR, vehicle-mounted millimeter-wave radar, etc.). These vehicle-road cooperative sensing scenarios can include intelligent driving scenarios, intelligent transportation scenarios, etc. Similarly, the aforementioned roadside equipment can be understood as sensing devices (such as LiDAR, cameras, millimeter-wave radar, etc.) fixedly installed on both sides of the road in vehicle-road cooperative sensing scenarios.

[0060] The first perception information can be understood as relevant data collected by vehicle-mounted devices at a set sampling frequency within the target fusion period to indicate the vehicle's surrounding environment (such as road markings, roadside building distribution, and nearby vehicles on the road). This data includes three-dimensional spatial points reflecting the surface contours of various physical entities and timestamps indicating specific sampling times. Similarly, the second perception information can be understood as relevant data collected by roadside devices at a set sampling frequency within the target fusion period to indicate the real-time status of the target area. The target area should be understood as the area that the roadside devices are fixedly sensing (e.g., the specific area that the lens of the roadside camera is pointing towards). The target area at least partially overlaps with the vehicle's surrounding environment.

[0061] In this invention, the weights of the vehicle-mounted devices and roadside devices in the fusion of perception information are dynamically adjusted based on the vehicle's surrounding environment and vehicle status. Therefore, multiple fusion cycles are defined. Each fusion cycle uses fusion weights adapted to the road conditions and vehicle status of the current fusion cycle to fuse the perception information of the vehicle-mounted devices and roadside devices. The aforementioned target fusion cycle can be understood as any one of the multiple fusion cycles defined. For example, the duration of the fusion cycle can be 200ms-300ms.

[0062] It should be understood that before determining the time factor, spatial factor, attitude factor, and registration factor based on the first and second sensing information, the first and second sensing information will be preprocessed (such as noise reduction, data format standardization, and outlier removal) to initially suppress noise and standardize the original sampled data.

[0063] Specifically, the first sensing information includes multiple vehicle-side sensing timestamps, and the second sensing information includes multiple roadside sensing timestamps;

[0064] The steps for obtaining the time factor include:

[0065] Among the multiple vehicle-side sensing timestamps, the roadside sensing timestamp associated with each vehicle-side sensing timestamp is determined. The roadside sensing timestamp associated with the vehicle-side sensing timestamp corresponds to the smallest timestamp distance among the multiple roadside sensing timestamps. The timestamp distance indicates the time difference between the corresponding vehicle-side sensing timestamp and the corresponding roadside sensing timestamp.

[0066] The time factor is obtained based on the central tendency of the timestamp distance between multiple vehicle-side sensing timestamps and their associated roadside sensing timestamps.

[0067] The timestamps mentioned above are used to record the specific time when the corresponding sensing device (such as vehicle-mounted device or roadside device) collects the corresponding point cloud data.

[0068] In this invention, the time factor can be understood as the average time stamp distance between multiple vehicle-side sensing timestamps and their associated roadside sensing timestamps.

[0069] For example, if the data sequence corresponding to multiple vehicle-side perception timestamps is set as follows: The data sequences corresponding to multiple roadside sensing timestamps are Where N represents the total number of vehicle-side sensing timestamps (which can also be understood as the number of point clouds collected by the vehicle-side equipment), and M represents the total number of roadside sensing timestamps (which can also be understood as the number of point clouds collected by the roadside equipment), then the time factor It can be represented as:

[0070]

[0071] in, Represents the first of multiple vehicle-side perception timestamps Individual vehicle-side sensing timestamp Indicates the first The roadside sensing timestamp associated with the vehicle's sensing timestamp.

[0072] The larger the value of the time factor, the greater the time difference between the observation of the same real target (such as vehicles, buildings, road markings, etc.). Correspondingly, the higher the degree of unreliability shown by the vehicle-end equipment in the time dimension (such as when a vehicle is driving along the longitudinal lane towards a T-junction, there may be a delay in the observation of vehicles in the transverse lane due to the obstruction of buildings).

[0073] In addition, the first sensing information includes multiple vehicle-side sensing spatial points, and the second sensing information includes multiple roadside sensing spatial points.

[0074] The steps for obtaining the space factor include:

[0075] Among the multiple vehicle-end sensing spatial points, the nearest roadside spatial point of each vehicle-end sensing spatial point is determined. The nearest roadside spatial point is the roadside sensing spatial point with the smallest spatial distance to the corresponding vehicle-end sensing spatial point among the multiple roadside sensing spatial points.

[0076] The spatial factor is obtained based on the convergence trend of the spatial distance between multiple vehicle-end perception timestamps and their corresponding nearest roadside spatial points.

[0077] In this invention, the vehicle-end sensing spatial point should be understood as the original three-dimensional point collected by the vehicle-end device and mapped to the corresponding three-dimensional point in the preset global coordinate system through a chain coordinate transformation. The roadside sensing spatial point should be understood as the original three-dimensional point collected by the roadside device and mapped to the corresponding three-dimensional point in the aforementioned global coordinate system through a chain coordinate transformation, so as to achieve the unification of the spatial coordinate system between the vehicle-end device and the roadside device (the above coordinate transformation mapping can be completed based on the ICP algorithm). The above spatial distance can be understood as the Euclidean distance between the corresponding roadside sensing spatial point and the corresponding vehicle-end sensing spatial point.

[0078] For example, if the data sequence corresponding to multiple vehicle-side sensing spatial points is set as follows: The data sequences corresponding to multiple roadside sensing spatial points are Among them, the aforementioned spatial factors It can be represented as:

[0079]

[0080] in, Represents the first of multiple vehicle-side perception spatial points Individual vehicle-mounted spatial sensing points Indicates the first The nearest roadside spatial points of the vehicle-mounted sensing spatial points Indicates the first The spatial distance between a vehicle-mounted sensing point and its nearest roadside sensing point.

[0081] The larger the value of the spatial factor, the greater the difference in spatial perception between the vehicle-mounted equipment and the roadside equipment. In other words, the greater the difference in the shape of the same real target observed by the two. Correspondingly, the higher the degree of unreliability shown by the vehicle-mounted equipment in the spatial dimension (such as when a vehicle is driving along the longitudinal lane toward a T-junction, the lack of spatial data is caused by insufficient observation of vehicles in the transverse lane due to building obstruction).

[0082] In one example, Kalman filtering can be used to take the vehicle attitude output by the onboard inertial navigation system as the predicted value and the vehicle attitude output by the lidar as the measured value, so as to obtain the relative pose matrix (including rotation angle and translation) of the vehicle at each moment in the target fusion period. Then, the difference (matrix difference) between the first and last relative pose matrices in the target fusion period is analyzed, and its Frobenius norm is determined as the attitude factor.

[0083] In another example, the ICP algorithm can be used to register the plurality of vehicle-side sensing spatial points with the plurality of roadside sensing spatial points to obtain spatial point registration results. Based on the spatial point registration results, a relative pose transformation matrix (i.e., a rigid body transformation matrix that best aligns the plurality of vehicle-side sensing spatial points with the plurality of roadside sensing spatial points) can be obtained, and its Frobenius norm can be determined as the pose factor.

[0084] The larger the value of the attitude factor, the more significant the change in attitude of the vehicle-end equipment, which means that the vehicle-end equipment exhibits a higher degree of unreliability in the spatial dimension (such as during rapid turns, acceleration, or bumpy conditions).

[0085] The steps for obtaining the registration factor include:

[0086] The multiple vehicle-side sensing spatial points are registered with the multiple roadside sensing spatial points to obtain spatial point registration results.

[0087] The overlap ratio and root mean square error are determined based on the spatial point registration results.

[0088] The registration factor is determined based on the overlap ratio and the root mean square error, wherein the overlap ratio is negatively correlated with the registration factor, and the root mean square error is positively correlated with the registration factor.

[0089] In this invention, the ICP algorithm is used to complete the above registration operation. Therefore, the spatial point registration result includes at least the aforementioned relative pose change matrix, overlap ratio (the number of effective matching point pairs and the proportion of the total number of points), root mean square error, and error convergence curve.

[0090] For example, the above registration factors It can be represented as:

[0091]

[0092] in, This indicates the aforementioned overlap ratio. This represents the root mean square error. This represents the weighting coefficient corresponding to the overlap ratio. This represents the maximum root mean square error that can be allowed during the registration process.

[0093] The larger the value of the registration factor, the worse the spatial point registration effect between the vehicle-mounted equipment and the roadside equipment, which means that the roadside equipment is more unreliable in the spatial dimension (such as the dense passage of pedestrians and electric vehicles at intersection red lights, resulting in highly mixed point clouds collected by the roadside equipment).

[0094] Step S2: Fuse the time factor, the spatial factor, the attitude factor, and the registration factor to obtain a fused vector.

[0095] Specifically, the step of fusing the time factor, the spatial factor, the attitude factor, and the registration factor to obtain a fusion vector includes:

[0096] The time factor is subjected to time standardization to obtain a time component, the space factor is subjected to spatial standardization to obtain a space component, and the attitude factor is subjected to attitude standardization to obtain an attitude component.

[0097] The time component, the spatial component, the attitude component, and the registration factor are concatenated to obtain a fusion vector.

[0098] For example, the above time components It can be represented as:

[0099]

[0100] in, This indicates the maximum time difference under extreme operating conditions (based on the maximum vehicle speed adaptive setting).

[0101] The above spatial components It can be represented as:

[0102]

[0103] in, This indicates the maximum spatial deviation value within the error range (based on road width adaptive settings).

[0104] The above attitude components It can be represented as:

[0105]

[0106] in, This represents the relative pose change matrix. The Frobenius norm (also known as the aforementioned pose factor) represents the relative pose change matrix. This represents the maximum positional change of the vehicle within the error range.

[0107] If the fusion vector is set to ,but .

[0108] It should be noted that in real-world scenarios, time factors, spatial factors, pose factors, and registration factors influence each other. For example, when the vehicle-mounted device is obstructed and there is a significant time difference between it and the roadside device, not only will the time factor be larger, but the spatial factor and the registration factor will also be larger. In addition, when the pose of the vehicle-mounted device changes drastically, not only will the pose factor be larger, but the spatial factor will also be larger. Since the time factor, spatial factor, pose factor, and registration factor interact and have a certain coupling relationship, this invention chooses to concatenate the four factors to form a fusion vector to ensure that the coupling relationship between the four factors corresponding to different dimensions is fully preserved, thus ensuring the reliability and accuracy of subsequent processing based on this vector.

[0109] The aforementioned fusion vector can be understood as a composite vector indicating the degree of unreliability of the vehicle-side perceived information.

[0110] Step S3: Determine the first fusion weight of the vehicle-side equipment and the second fusion weight of the roadside equipment based on the fusion vector.

[0111] The first fusion weight is negatively correlated with the L2 norm of the fusion vector, and the sum of the first fusion weight and the second fusion weight is 1.

[0112] Specifically, the step of determining the first fusion weight of the vehicle-side equipment and the second fusion weight of the roadside equipment based on the fusion vector includes:

[0113] Based on the L2 norm of the fusion vector, the first generation weight of the corresponding vehicle-side device and the second generation weight of the corresponding roadside device are determined. The first generation weight is negatively correlated with the L2 norm of the fusion vector, and the second generation weight is positively correlated with the L2 norm of the fusion vector.

[0114] The real-time fusion weight of the vehicle-side device is determined based on the first generation weight, the second generation weight, the initial weight of the vehicle-side device corresponding to the target fusion period, and the initial weight of the roadside device corresponding to the target fusion period.

[0115] The real-time fusion weight of the vehicle-side device is weighted and calculated with the initial weight of the vehicle-side device to obtain the first fusion weight;

[0116] The second fusion weight is determined based on the first fusion weight.

[0117] Furthermore, the initial weight of the vehicle-side corresponding to the target fusion period is the fusion weight determined within the historical fusion period, and the initial weight of the roadside corresponding to the target fusion period is the fusion weight determined within the historical fusion period, wherein the historical fusion period is the fusion period preceding the target fusion period.

[0118] The calculation weight of the real-time fusion weight of the vehicle-end device is determined based on the L2 norm of the fusion vector and the registration factor. The calculation weight of the real-time fusion weight of the vehicle-end device is negatively correlated with the L2 norm of the fusion vector, and the calculation weight of the real-time fusion weight of the vehicle-end device is negatively correlated with the registration factor.

[0119] In applications, a nonlinear function can be used to process the L2 norm of the fusion vector and the registration factor to obtain a relatively accurate calculated weight for the real-time fusion weight of the vehicle-side device.

[0120] For example, the calculation weight of the real-time fusion weight of the vehicle-side device. It can be represented as:

[0121]

[0122] in, This represents an exponential function with base e. Denotes the L2 norm of the fused vector. The sensitivity coefficient represents the L2 norm of the fusion vector. Registration factor The corresponding sensitivity coefficients, the two sensitivity coefficients mentioned above can be adjusted adaptively according to actual needs.

[0123] The aforementioned first weight It can be represented as:

[0124]

[0125] The above The weight sensitivity adjustment coefficient corresponding to the L2 norm of the fusion vector can be set based on empirical adaptability, while the second-generation weight is... .

[0126] Real-time fusion weights of vehicle-side devices It can be represented as:

[0127]

[0128] in, This indicates the initial weights on the vehicle side corresponding to the target fusion period. This represents the initial roadside weights corresponding to the target fusion period. .

[0129] First fusion weight of the target fusion cycle It can be represented as:

[0130]

[0131] Second fusion weight of the target fusion cycle It can be represented as:

[0132]

[0133] In the above setup, the calculation of the first and second weight generation is used to adapt to the degree of unreliability of the vehicle-side perception information indicated by the fusion vector, and the fusion weight allocation of the vehicle-side equipment and the roadside equipment is initially completed. On this basis, the fusion weights actually configured for both in the previous fusion cycle are superimposed with the fusion weights initially allocated for both in the target fusion cycle, and weight smoothing is performed. This ensures that the first and second fusion weights are adapted to the actual working conditions of the vehicle-side equipment and the roadside equipment, while making the dynamic weight adjustment process in multiple fusion cycles smoother and more coherent, thereby suppressing the risk of abnormal changes in perception fusion information caused by sudden weight changes.

[0134] It should be understood that the first fusion weight is calculated based on the target fusion period. With the second fusion weight They will be used as the initial weights for the vehicle and roadside in the next fusion cycle, respectively.

[0135] Step S4: Based on the first fusion weight and the second fusion weight, fuse the first perception information and the second perception information to obtain the target perception information.

[0136] Specifically, the step of fusing the first perceived information and the second perceived information according to the first fusion weight and the second fusion weight to obtain the target perceived information includes:

[0137] Based on the calculated weight of the real-time fusion weight of the vehicle-end device, a first redundancy factor is determined, wherein the first redundancy factor indicates the difficulty of information fusion between the first sensing information and the second sensing information.

[0138] A second redundancy factor is determined based on the L2 norm of the fusion vector, wherein the second redundancy factor indicates the spatiotemporal alignment quality loss between the first and second sensing information.

[0139] A third redundancy factor is determined based on the weight difference between the first fusion weight and the second fusion weight, wherein the third redundancy factor indicates the spatiotemporal perception capability deviation between the vehicle-mounted equipment and the roadside equipment.

[0140] The initial fusion radius is amplified based on the first redundancy factor, the second redundancy factor, and the third redundancy factor to obtain the target fusion radius;

[0141] Based on the target fusion radius, the first perception information and the second perception information are fused according to the first fusion weight and the second fusion weight to obtain the target perception information.

[0142] The first redundancy factor is negatively correlated with the calculated weight of the real-time fusion weight of the vehicle-side device, the second redundancy factor is positively correlated with the L2 norm of the fusion vector, the third redundancy factor is positively correlated with the weight difference value, and the weight difference value is the absolute difference between the first fusion weight and the second fusion weight.

[0143] Based on the above settings, the fusion range is dynamically adjusted according to the actual perception quality and perception differences between the vehicle-mounted equipment and the roadside equipment. This can significantly expand the initial fusion radius to obtain a greater planning safety margin and effectively ensure driving safety when the perception quality is poor or the perception difference is too large. When the perception quality is good and the perception difference is small, the initial fusion radius is slightly expanded to reduce the amount of data to be processed and improve planning efficiency while ensuring driving safety. At the same time, the accuracy and reliability of the final output planning information (such as the optimal driving path of the vehicle in the autonomous driving scenario) are improved.

[0144] For example, the above-mentioned target fusion radius It can be represented as:

[0145]

[0146] in, Indicates the initial fusion radius (adaptively set based on actual needs). Indicates the first redundancy factor. Indicates the second redundancy factor. Indicates the third redundancy factor. , as well as These are the redundancy sensitivity coefficients of the corresponding redundancy factors, used to adjust the degree of influence of the corresponding redundancy factors on the target fusion radius.

[0147] In applications, upper and lower limits can be configured for the target fusion radius to ensure that the target fusion radius is not lower than the minimum fusion radius threshold and to ensure that the target fusion radius is not higher than the maximum fusion radius threshold, so as to avoid abnormal fluctuations in planning under extreme working conditions or strong interference.

[0148] It should be understood that the center point corresponding to the target fusion radius is the center or centroid of the vehicle where the vehicle-end device is located.

[0149] Based on the aforementioned center point and target fusion radius, a spherical fusion region can be adaptively determined. In the subsequent fusion process, several vehicle-end perception spatial points located within the fusion region will be matched with several roadside perception spatial points (meaning finding the nearest roadside perception spatial point for each vehicle-end perception spatial point). Based on the first fusion weight and the second fusion weight, the matched vehicle-end perception spatial points and roadside perception spatial points will be fused to obtain several fused spatial points to accurately indicate the surface contours and positional distribution of each object in the fusion region. The target perception information includes the aforementioned several fused spatial points.

[0150] In summary, this invention, within the target fusion cycle, analyzes the first perception information from vehicle-mounted devices and the second perception information from roadside devices to assess the unreliability of vehicle-mounted perception information during collaborative road perception from temporal, spatial, vehicle pose, and point cloud registration dimensions. Then, it fuses the quantified values ​​of these four dimensions to form a fusion vector, ensuring that the coupling relationship between different dimensional indicators is fully preserved. Next, it calculates the L2 norm of the fusion vector to quantify the unreliability of vehicle-mounted perception information and dynamically adjusts the fusion weights of vehicle-mounted devices and roadside devices during the perception information fusion stage. When vehicle-mounted perception information exhibits a high error risk, its weight during fusion is dynamically reduced to suppress the contamination of the final fused collaborative perception information by vehicle-mounted perception information with strong interference. Conversely, when vehicle-mounted perception information exhibits a low error risk, its weight during fusion is dynamically increased to amplify the gain of vehicle-mounted perception information on the final fused collaborative perception information, improving the accuracy and reliability of collaborative perception information and significantly enhancing safety, stability, and adaptability in complex traffic environments.

[0151] To verify the effectiveness of the proposed vehicle-road cooperative perception fusion and planning spatiotemporal quality closed-loop control method, system simulations and comparative experiments were conducted to assess aspects such as time synchronization accuracy, spatial registration error, fusion weight stability, and dynamic response of the safety radius. The experiments focused on verifying the core mechanism of this invention: using the quality vector Q (covering time residuals, spatial deviations, ICP stability, etc.) from the spatiotemporal alignment stage as the control quantity for fusion weighting, allowing the fusion weights to be adjusted in real time according to data quality; and mapping the fusion confidence (i.e., the calculated weight of the real-time fusion weights from the aforementioned vehicle-side equipment) to the planning safety radius, forming a quality closed-loop control that spans perception, fusion, and planning. This effectively improves the system's safety and adaptability in complex scenarios.

[0152] The experiment was conducted in a Windows 11 operating system environment, using the Python 3.8 programming language and a simulation platform built with tools such as Open3D, NumPy, and Matplotlib. The experiment used the V2X-Seq-SPD dataset, which contains LiDAR point clouds and timestamp information from both vehicle and roadside locations, realistically reflecting the vehicle-road cooperative scenario. During the experiment, four core modules were jointly verified: time synchronization (represented by time factors), spatial registration (represented by spatial factors), adaptive weighted fusion (i.e., the calculation process of the first and second fusion weights mentioned above), and safety radius mapping (i.e., the calculation process of the target fusion radius mentioned above).

[0153] Simulation results show that the spatiotemporal quality vector Q of the present invention can effectively drive time compensation and spatial registration, enabling the vehicle and roadside to maintain stable alignment under scenarios such as high speed, occlusion, and attitude changes. During the fusion stage, the first fusion weight and the second fusion weight are adjusted in real time by the fusion vector, which can suppress the interference of low-confidence frames and abnormal point clouds, enabling the system to output continuous and reliable fused point clouds and BEV reconstruction results even under communication delay, occlusion, or asynchronous conditions. At the same time, the fusion confidence and the quality change of the fusion vector can be linked to adjust the planned target fusion radius, enabling the system to automatically increase safety redundancy when the perceived quality deteriorates, demonstrating the advantages of quality closed-loop control.

[0154] Experimental results demonstrate that the proposed solution possesses excellent real-time performance, robustness, and engineering feasibility, enabling high-precision, multi-source consistent perception fusion and safety planning output in complex traffic environments.

[0155] In one embodiment, the present invention also provides a spatiotemporal quality closed-loop control system for vehicle-road cooperative perception fusion and planning, such as... Figure 2 As shown, the system 200 includes:

[0156] The factor determination module 201 is used to analyze the first perception information of the vehicle-mounted device and the second perception information of the roadside device within the target fusion period, and determine the time factor, spatial factor, attitude factor and registration factor. The time factor represents the perception time deviation between the vehicle-mounted device and the roadside device, the spatial factor represents the perception position deviation between the vehicle-mounted device and the roadside device, the attitude factor represents the pose fluctuation degree of the vehicle-mounted device, and the registration factor represents the point cloud registration quality loss between the vehicle-mounted device and the roadside device.

[0157] The factor fusion module 202 is used to fuse the time factor, the spatial factor, the attitude factor, and the registration factor to obtain a fusion vector;

[0158] The weight determination module 203 is used to determine the first fusion weight of the vehicle-end device and the second fusion weight of the roadside device based on the fusion vector, wherein the first fusion weight is negatively correlated with the L2 norm of the fusion vector, and the sum of the first fusion weight and the second fusion weight is 1.

[0159] The information fusion module 204 is used to fuse the first perception information and the second perception information according to the first fusion weight and the second fusion weight to obtain target perception information.

[0160] It should be noted that the system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the spatiotemporal quality closed-loop control system for vehicle-road cooperative perception fusion and planning provided in the above embodiments and the spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0161] This invention also provides an electronic device. Please refer to [link to relevant documentation]. Figure 3 The electronic device may include a processor 301, a memory 302, and a program 3021 stored in the memory 302 and capable of running on the processor 301.

[0162] When program 3021 is executed by processor 301, it can achieve the following: Figure 1 Any steps in the corresponding method embodiments and the achievement of the same beneficial effects will not be repeated here.

[0163] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by hardware related to program instructions, and the program can be stored in a readable medium.

[0164] This invention also provides a readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described functions. Figure 1 Any step in the corresponding method embodiment can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0165] The computer-readable storage medium of this invention can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0166] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0167] The program code contained on the storage medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0168] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or terminal. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0169] This invention also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to achieve the spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning provided in the above embodiments.

[0170] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0171] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A spatiotemporal quality closed-loop control method based on vehicle-road cooperative perception fusion and planning, characterized in that, The method includes: Within the target fusion period, the first perception information of the vehicle-mounted device and the second perception information of the roadside device are analyzed to determine the time factor, spatial factor, attitude factor and registration factor. The time factor represents the perception time deviation between the vehicle-mounted device and the roadside device, the spatial factor represents the perception position deviation between the vehicle-mounted device and the roadside device, the attitude factor represents the pose fluctuation of the vehicle-mounted device, and the registration factor represents the point cloud registration quality loss between the vehicle-mounted device and the roadside device. The time factor, the spatial factor, the attitude factor, and the registration factor are fused to obtain a fusion vector; The first fusion weight of the vehicle-side device and the second fusion weight of the roadside device are determined based on the fusion vector, wherein the first fusion weight is negatively correlated with the L2 norm of the fusion vector, and the sum of the first fusion weight and the second fusion weight is 1. Based on the first fusion weight and the second fusion weight, the first perception information and the second perception information are fused to obtain the target perception information; First fusion weight of vehicle-side equipment for: In the above formula, The weights are used to calculate the real-time fusion weights of the vehicle-side devices. Real-time fusion weights for vehicle-side devices, The initial weights on the vehicle side corresponding to the target fusion cycle; Second fusion weight of roadside equipment for: Calculation of real-time fusion weights for vehicle-side devices Specifically: In the above formula, This represents an exponential function with base e. Denotes the L2 norm of the fused vector. The sensitivity coefficient represents the L2 norm of the fusion vector. Registration factor The corresponding sensitivity coefficient; Real-time fusion weights of vehicle-side devices for: in, This indicates the initial weights on the vehicle side corresponding to the target fusion period. This represents the initial roadside weights corresponding to the target fusion period. The initial weight of the vehicle-side corresponding to the target fusion period is the vehicle-side fusion weight determined within the historical fusion period, and the initial weight of the roadside corresponding to the target fusion period is the roadside fusion weight determined within the historical fusion period, wherein the historical fusion period is the fusion period preceding the target fusion period. First birth weight for: The weight sensitivity adjustment coefficient represents the L2 norm of the fusion vector, and the second generation weight is... .

2. The spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning according to claim 1, characterized in that, The first sensing information includes multiple vehicle-side sensing timestamps, and the second sensing information includes multiple roadside sensing timestamps; The steps for obtaining the time factor include: Among the multiple vehicle-side sensing timestamps, the roadside sensing timestamp associated with each vehicle-side sensing timestamp is determined. The roadside sensing timestamp associated with the vehicle-side sensing timestamp corresponds to the smallest timestamp distance among the multiple roadside sensing timestamps. The timestamp distance indicates the time difference between the corresponding vehicle-side sensing timestamp and the corresponding roadside sensing timestamp. The time factor is obtained based on the central tendency of the timestamp distance between multiple vehicle-side sensing timestamps and their associated roadside sensing timestamps.

3. The spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning according to claim 1, characterized in that, The first sensing information includes multiple vehicle-side sensing spatial points, and the second sensing information includes multiple roadside sensing spatial points; The steps for obtaining the space factor include: Among the multiple vehicle-end sensing spatial points, the nearest roadside spatial point of each vehicle-end sensing spatial point is determined. The nearest roadside spatial point is the roadside sensing spatial point with the smallest spatial distance to the corresponding vehicle-end sensing spatial point among the multiple roadside sensing spatial points. The spatial factor is obtained based on the convergence trend of the spatial distance between multiple vehicle-end perception timestamps and their corresponding nearest roadside spatial points.

4. The spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning according to claim 1, characterized in that, The first sensing information includes multiple vehicle-side sensing spatial points, and the second sensing information includes multiple roadside sensing spatial points; The steps for obtaining the registration factor include: The multiple vehicle-side sensing spatial points are registered with the multiple roadside sensing spatial points to obtain spatial point registration results. The overlap ratio and root mean square error are determined based on the spatial point registration results. The registration factor is determined based on the overlap ratio and the root mean square error, wherein the overlap ratio is negatively correlated with the registration factor, and the root mean square error is positively correlated with the registration factor.

5. The spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning according to claim 1, characterized in that, The step of fusing the time factor, the spatial factor, the attitude factor, and the registration factor to obtain a fusion vector includes: The time factor is subjected to time standardization to obtain a time component, the space factor is subjected to spatial standardization to obtain a space component, and the attitude factor is subjected to attitude standardization to obtain an attitude component. The time component, the spatial component, the attitude component, and the registration factor are concatenated to obtain a fusion vector.

6. The spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning according to claim 1, characterized in that, The step of fusing the first perceived information and the second perceived information according to the first fusion weight and the second fusion weight to obtain the target perceived information includes: Based on the calculated weight of the real-time fusion weight of the vehicle-end device, a first redundancy factor is determined, wherein the first redundancy factor indicates the difficulty of information fusion between the first sensing information and the second sensing information. A second redundancy factor is determined based on the L2 norm of the fusion vector, wherein the second redundancy factor indicates the spatiotemporal alignment quality loss between the first and second sensing information. A third redundancy factor is determined based on the weight difference between the first fusion weight and the second fusion weight, wherein the third redundancy factor indicates the spatiotemporal perception capability deviation between the vehicle-mounted equipment and the roadside equipment. The initial fusion radius is amplified based on the first redundancy factor, the second redundancy factor, and the third redundancy factor to obtain the target fusion radius; Based on the target fusion radius, the first perception information and the second perception information are fused according to the first fusion weight and the second fusion weight to obtain the target perception information.

7. The spatiotemporal quality closed-loop control method for vehicle-road cooperative perception fusion and planning according to claim 6, characterized in that, The first redundancy factor is negatively correlated with the calculated weight of the real-time fusion weight of the vehicle-side device, the second redundancy factor is positively correlated with the L2 norm of the fusion vector, and the third redundancy factor is positively correlated with the weight difference value, wherein the weight difference value is the absolute difference between the first fusion weight and the second fusion weight.

Images