A Multi-Target Cooperative Perception Method for Connected Autonomous Vehicles
By using deep learning and multi-objective optimization techniques, local perception features are constructed and compressed. Combined with path loss and channel gain models, an end-to-end perception delay and energy consumption model is established. An asynchronous decision-making mechanism is adopted to solve the problems of redundant communication and computational overhead in cooperative perception, thereby improving the perception accuracy and system efficiency of connected autonomous vehicles.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XI AN JIAOTONG UNIV
- Filing Date
- 2026-06-10
- Publication Date
- 2026-07-10
AI Technical Summary
Existing collaborative sensing schemes suffer from redundant communication and computational overhead in dynamic traffic environments, making it difficult to balance sensing accuracy, computational energy consumption, and decision delay, and they also lack effective utilization of heterogeneous computing resources.
Local sensing features are constructed and compressed using deep learning networks. An end-to-end sensing delay and energy consumption model is established by combining path loss and channel gain models. Chebyshev multi-objective optimization and asynchronous decision-making mechanisms are adopted, and the sensing strategy is optimized using Actor and Critic networks.
It achieves reduced communication overhead in dynamic environments, balances perception accuracy, computing power consumption and decision-making frequency, and improves the accuracy and efficiency of collaborative perception.
Smart Images
Figure CN122365407A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous driving technology, and more specifically to a multi-target cooperative perception method for connected autonomous vehicles. Background Technology
[0002] In the field of autonomous driving technology, onboard sensors are the core means for vehicles to perceive and respond to dynamic road environments. With the development of vehicle-to-everything (V2X) communication technology, cooperative perception technology allows multiple connected autonomous vehicles to share and fuse their respective perception data through real-time communication, thereby effectively eliminating blind spots caused by buildings or other vehicles obstructing perception in single-vehicle scenarios and expanding the field of environmental perception. Cooperative perception has become an important technical path to improve the safety and reliability of autonomous driving.
[0003] In existing technologies, cooperative perception schemes typically employ a synchronous decision-making model, where all vehicles simultaneously share and fuse perception data at predetermined time intervals, optimizing multiple performance objectives such as perception accuracy, computational energy consumption, and decision latency through linear weighted summation. However, in dynamic environments with fluctuating traffic density and frequent occlusion, the synchronous decision-making model generates excessive redundant communication and computational overhead, leading to overall system inefficiency. Furthermore, the linear weighted summation method struggles to effectively balance different objective magnitudes, easily sacrificing other key performance aspects for over-optimization of a particular metric. Additionally, existing cooperative perception schemes often assume homogeneous computing capabilities among vehicles, lacking effective modeling and utilization of asynchronous vehicle operations and heterogeneous computing resources. The fixed decision-making model is ill-suited to dynamically changing traffic scenarios and inter-vehicle topologies.
[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0005] In view of the above problems, the present invention is proposed to provide a multi-target cooperative perception method for connected autonomous vehicles that overcomes or at least partially solves the above problems.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] In a first aspect, embodiments of the present invention provide a multi-target cooperative perception method for connected autonomous vehicles, comprising:
[0008] Obtain raw environmental data, construct local perceptual features through a deep learning network, and compress the features to obtain compressed features;
[0009] Based on compression characteristics and the positional relationship between vehicles, a path loss and channel gain model is constructed, and the communication transmission delay is calculated using orthogonal frequency division multiple access.
[0010] Based on the role of vehicles in cooperative perception and communication transmission delay, and combined with the computational complexity of compressed features involved in each role, end-to-end perception delay and energy consumption models are established to obtain the perception delay and energy consumption corresponding to each role.
[0011] Based on sensing delay and energy consumption, a multi-objective optimization model based on Chebyshev's method is constructed to transform the three conflicting objectives of sensing accuracy, computational energy consumption and number of decisions into minimizing the maximum weighted distance.
[0012] Based on the local state of each vehicle, an action strategy is generated asynchronously through an Actor network with a self-attention mechanism, and the global state value is evaluated through a Critic network with a graph attention mechanism to output the action strategy; the local state includes perception accuracy, cumulative energy consumption and motion information.
[0013] The collaborative perception task is performed based on compressed features and action strategies. The compressed features are decompressed and fused to obtain fused features, and target detection is performed based on the fused features to obtain the detection results.
[0014] Based on the perception accuracy, energy consumption, and time delay performance after the collaborative perception task is executed, the multi-objective reward value is calculated according to the multi-objective optimization model. The parameters of the Actor network and the Critic network are updated by truncating the near-end policy optimization loss function and the mean square error loss function, respectively, to achieve iterative optimization.
[0015] Further, the steps of acquiring raw environmental data, constructing local perceptual features through a deep learning network, and compressing these features to obtain compressed features include:
[0016] The raw point cloud is collected by vehicle-mounted LiDAR and preprocessed to obtain the preprocessed point cloud.
[0017] By using a feature extraction network and projection operations, the bird's-eye view features are obtained based on the preprocessed point cloud computing.
[0018] The features of the bird's-eye view are compressed to obtain compressed features.
[0019] Furthermore, the calculation formula for the preprocessed point cloud is as follows:
[0020]
[0021] in, For preprocessed point cloud data, The raw LiDAR point cloud data collected for vehicle i This is the function for voxel downsampling. Extraction function for region of interest, The transformation matrix for converting the point cloud to the vehicle coordinate system;
[0022] The formula for calculating the features of a bird's-eye view is:
[0023]
[0024] in, For the bird's-eye view feature tensor, A neural network for point cloud feature extraction. These are the weight parameters of the feature extraction network. This is an operation function that projects high-dimensional features onto a two-dimensional bird's-eye view plane;
[0025] The formula for calculating the compression feature is:
[0026]
[0027] in, For compression features, For feature compression neural network functions, These are the weight parameters for compressing the network.
[0028] Furthermore, based on compression characteristics and the positional relationship between vehicles, a path loss and channel gain model is constructed, and the communication transmission delay is calculated using orthogonal frequency division multiple access (OFDM). This includes the following steps:
[0029] Calculate the path loss between vehicles based on the Euclidean distance between them, and then calculate the channel gain based on the path loss.
[0030] The data transmission rate is calculated based on channel gain and through orthogonal frequency division multiple access.
[0031] Based on the data transmission rate and compression characteristics, the communication time required to transmit the compression characteristics is calculated, thus obtaining the communication transmission delay.
[0032] Furthermore, the formula for calculating the path loss between vehicles is as follows:
[0033]
[0034] in, This refers to path loss between vehicles. Let be the Euclidean distance between vehicle i and vehicle j. For carrier frequency;
[0035] The formula for calculating channel gain is:
[0036]
[0037] in, Channel gain;
[0038] The formula for calculating data transmission rate is:
[0039]
[0040] in, For data transmission rate, The number of subcarriers allocated to the link. The bandwidth of a single subcarrier. Let be the transmission power of vehicle i. The noise power spectral density;
[0041] The formula for calculating the communication time required to transmit compressed features is:
[0042]
[0043] in, The communication time required to transmit compressed features, To compress the bit size of feature data.
[0044] Furthermore, based on the vehicle's role in cooperative perception and communication transmission latency, and considering the computational complexity of compressed features involved in each role, end-to-end perception latency and energy consumption models are established to obtain the perception latency and energy consumption corresponding to each role. The steps include:
[0045] Define the computational complexity of each stage in the perception pipeline, including preprocessing cost, feature extraction cost, projection cost, feature compression cost, feature decompression cost, feature fusion cost, and target detection cost;
[0046] Construct an end-to-end perceived latency model and energy consumption model for independently operating roles, sender roles, and receiver roles;
[0047] Construct an end-to-end perceived latency model and energy consumption model with dual roles; where the dual role refers to simultaneously acting as a sender and a receiver.
[0048] Furthermore, based on sensing latency and energy consumption, a multi-objective optimization model based on the Chebyshev method is constructed to transform the three conflicting objectives of sensing accuracy, computational energy consumption, and number of decision-making steps into steps that minimize the maximum weighted distance, including:
[0049] The normalized deviation distance is calculated based on three objectives: sensing accuracy, energy consumption, and number of decisions, considering sensing latency and energy consumption.
[0050] Based on the normalized bias distance, the three conflicting objectives of perception accuracy, computational energy consumption, and number of decisions are transformed into minimizing the maximum weighted distance, and a multi-objective optimization model is constructed.
[0051] Furthermore, based on the local states of each vehicle, the action policy is asynchronously generated through an Actor network with a self-attention mechanism, and the global state value is evaluated through a Critic network with a graph attention mechanism to output the action policy. The steps include:
[0052] Construct a local state vector that includes single-vehicle perception accuracy, cumulative energy consumption, and vehicle motion information;
[0053] Based on the local state vector, the local state is processed by an Actor network with a self-attention mechanism to obtain the action policy;
[0054] The global state value is evaluated using a Critic network with a graph attention mechanism, and an action policy is output.
[0055] Furthermore, the steps of performing a collaborative perception task based on compressed features and action strategies, including decompressing and fusing the compressed features to obtain fused features, and performing target detection based on the fused features to obtain the detection results, include:
[0056] The receiving vehicle decompresses the received compressed features and fuses them with local features to obtain fused features;
[0057] The detection head performs target detection on the fused features to obtain the detection results.
[0058] Furthermore, based on the perception accuracy, energy consumption, and latency performance after the collaborative perception task is executed, multi-objective reward values are calculated according to the multi-objective optimization model. The parameters of the Actor network and Critic network are updated by truncating the near-end policy optimization loss function and the mean squared error loss function, respectively, to achieve iterative optimization steps, including:
[0059] The instantaneous reward at the current moment is calculated based on a multi-objective optimization model;
[0060] The Actor network parameters are updated by optimizing the loss function through a truncated proximal strategy.
[0061] Iterative optimization is achieved by updating the Critic network parameters using the mean squared error loss function.
[0062] The beneficial effects of the above-described technical solutions provided in the embodiments of the present invention include at least the following:
[0063] This invention provides a multi-objective cooperative perception method for connected autonomous vehicles. By extracting and compressing features from raw environmental data, the method significantly reduces the amount of data to be transmitted in cooperative perception, effectively reducing vehicle-to-everything (V2X) communication overhead. Furthermore, by distinguishing the different roles of vehicles in cooperative perception and establishing end-to-end perception latency and energy consumption models for each, and combining this with a Chebyshev multi-objective optimization model to balance the three conflicting objectives of perception accuracy, computational energy consumption, and decision-making frequency, the method achieves accurate quantification and robust optimization of system performance under heterogeneous computing resources. Simultaneously, by using an Actor network with a self-attention mechanism to asynchronously generate action strategies based on local states, and utilizing a Critic network with a graph attention mechanism to evaluate the global state value to guide strategy learning, each vehicle can adaptively determine the timing and target of cooperation and adjust the decision-making frequency, thereby improving the accuracy, robustness, and system efficiency of cooperative perception in dynamic environments. Attached Figure Description
[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0065] Figure 1 This is a flowchart illustrating a multi-target cooperative perception method for connected autonomous vehicles provided in an embodiment of the present invention;
[0066] Figure 2 This is a flowchart illustrating step S100 of a multi-target cooperative perception method for connected autonomous vehicles provided in an embodiment of the present invention.
[0067] Figure 3 This is a flowchart illustrating step S200 of a multi-target cooperative perception method for connected autonomous vehicles provided in an embodiment of the present invention.
[0068] Figure 4 This is a flowchart illustrating step S300 of a multi-target cooperative perception method for connected autonomous vehicles provided in an embodiment of the present invention.
[0069] Figure 5 This is a flowchart illustrating step S400 of a multi-target cooperative perception method for connected autonomous vehicles provided in an embodiment of the present invention.
[0070] Figure 6 This is a flowchart illustrating step S500 of a multi-target cooperative perception method for connected autonomous vehicles provided in an embodiment of the present invention.
[0071] Figure 7This is a flowchart illustrating step S600 of a multi-target cooperative perception method for connected autonomous vehicles provided in an embodiment of the present invention.
[0072] Figure 8 This is a flowchart illustrating step S700 of a multi-target cooperative perception method for connected autonomous vehicles provided in an embodiment of the present invention. Detailed Implementation
[0073] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0074] This invention discloses a multi-target cooperative perception method for connected autonomous vehicles, such as... Figure 1 As shown, it includes:
[0075] Step S100: Obtain the original environmental data, construct local perceptual features through a deep learning network, and compress the features to obtain compressed features.
[0076] Step S200: Based on compression characteristics and the positional relationship between vehicles, construct a path loss and channel gain model, and calculate the communication transmission delay using orthogonal frequency division multiple access.
[0077] Step S300: Based on the vehicle's role in cooperative perception and communication transmission delay, and combined with the computational complexity of compressed features involved in each role, establish end-to-end perception delay and energy consumption models respectively, and obtain the perception delay and energy consumption corresponding to each role.
[0078] Step S400: Based on sensing delay and energy consumption, construct a multi-objective optimization model based on Chebyshev's method to transform the three conflicting objectives of sensing accuracy, computational energy consumption, and number of decisions into minimizing the maximum weighted distance.
[0079] Step S500: Based on the local state of each vehicle, an action strategy is asynchronously generated through an Actor network with a self-attention mechanism, and the global state value is evaluated through a Critic network with a graph attention mechanism to output the action strategy; the local state includes perception accuracy, cumulative energy consumption and motion information.
[0080] Step S600: Perform a collaborative perception task based on compressed features and action strategies. Decompress and fuse the compressed features to obtain fused features, and perform target detection based on the fused features to obtain detection results.
[0081] Step S700: Based on the perception accuracy, energy consumption, and time delay performance after the collaborative perception task is executed, calculate the multi-objective reward value according to the multi-objective optimization model, and update the parameters of the Actor network and Critic network respectively by truncating the near-end policy optimization loss function and the mean square error loss function to achieve iterative optimization.
[0082] The above-mentioned multi-objective cooperative perception method for connected autonomous vehicles enables each vehicle to make asynchronous decisions on the timing and target of cooperation based on the local environment, and achieves a multi-objective balance between perception accuracy, computational energy consumption and decision-making frequency, forming a closed-loop iteration of perception, decision-making, execution and optimization, effectively improving the accuracy and system efficiency of cooperative perception in dynamic environments.
[0083] Below, we will refer to Figures 2 to 8 The steps of the multi-target cooperative perception method for connected autonomous vehicles described in this example embodiment will be explained in more detail.
[0084] In step S100, the original environmental data is acquired, local perceptual features are constructed through a deep learning network, and compressed features are obtained.
[0085] It should be noted that the local environmental information is characterized and lightweighted by the vehicle's sensors and deep learning network, providing basic data for subsequent collaborative transmission and processing.
[0086] In one possible implementation, step S100 may further include the following sub-steps:
[0087] In step S110, the original point cloud is acquired by the vehicle-mounted lidar. And perform preprocessing to obtain the preprocessed point cloud. .
[0088] It should be noted that the original LiDAR point cloud undergoes standardization and normalization processing. Voxel downsampling can effectively reduce the amount of point cloud data while preserving the core geometric structure features, thereby reducing the computational load of subsequent feature extraction. Region of interest extraction can filter out invalid point cloud data in non-road areas around the vehicle, focusing the processing range on the effective area for road perception and further reducing redundant calculations. Coordinate system transformation converts the original point cloud from the radar's own coordinate system to the vehicle's coordinate system, eliminating coordinate deviations caused by the radar's installation position and ensuring coordinate consistency for subsequent feature extraction and cross-vehicle feature fusion.
[0089] Furthermore, the calculation formula for the preprocessed point cloud is as follows:
[0090]
[0091] in, For preprocessed point cloud data, The raw LiDAR point cloud data collected for vehicle i This is the function for voxel downsampling. Extraction function for region of interest, This is the transformation matrix for converting the point cloud to the vehicle coordinate system.
[0092] In step S120, the bird's-eye view features are obtained by using a feature extraction network and projection operations based on the preprocessed point cloud computing.
[0093] It should be noted that the bird's-eye view (BEV) feature is a unified feature representation obtained by projecting the high-dimensional features of the three-dimensional point cloud onto a two-dimensional top-view plane. This feature can eliminate the difference in observation perspective between different connected vehicles, solve the problem of feature alignment under different perspectives, and the structure of the two-dimensional feature is more conducive to subsequent feature compression and cross-vehicle feature fusion, providing a unified feature space for multi-vehicle collaborative perception and reducing the complexity of collaborative processing.
[0094] Furthermore, the formula for calculating the features of a bird's-eye view is as follows:
[0095]
[0096] in, For the bird's-eye view feature tensor, A neural network for point cloud feature extraction. These are the weight parameters of the feature extraction network. This is an operation function that projects high-dimensional features onto a two-dimensional bird's-eye view plane.
[0097] In step S130, the bird's-eye view features are compressed to obtain compressed features.
[0098] It should be noted that feature compression is not a simple dimensionality reduction operation. Instead, it uses a specially trained compression neural network to significantly reduce the data volume of features while fully preserving the core feature information required for the perception task. This effectively reduces the communication overhead of vehicle-to-everything (V2X) transmission and also shortens the feature transmission latency. It avoids the problem of excessively high collaborative perception latency caused by the transmission of large-volume features and can be well adapted to the limited communication bandwidth conditions in V2X scenarios.
[0099] Furthermore, the formula for calculating the compression feature is as follows:
[0100]
[0101] in, For compression features, For feature compression neural network functions, These are the weight parameters for compressing the network.
[0102] In step S200, a path loss and channel gain model is constructed based on compression characteristics and the positional relationship between vehicles, and the communication transmission delay is calculated using orthogonal frequency division multiple access.
[0103] It should be noted that by using the position between vehicles and the size of the compression features, the communication latency in the collaborative transmission process is quantified, providing communication-level parameter support for subsequent system performance modeling.
[0104] In one possible implementation, step S200 may further include the following sub-steps:
[0105] In step S210, the path loss between vehicles is calculated based on the Euclidean distance between them, and the channel gain is calculated based on the path loss.
[0106] It should be noted that vehicle path loss and channel gain are the core basic parameters of V2V communication in the Internet of Vehicles. The modeling of vehicle Euclidean distance and carrier frequency is combined to fit the actual wireless propagation attenuation characteristics of urban roads.
[0107] Furthermore, the formula for calculating the path loss between vehicles is as follows:
[0108]
[0109] in, This refers to path loss between vehicles. Let be the Euclidean distance between vehicle i and vehicle j. The carrier frequency is measured in Hertz (Hz).
[0110] The formula for calculating channel gain is:
[0111]
[0112] in, This represents the channel gain.
[0113] In step S220, the data transmission rate is calculated based on the channel gain and through orthogonal frequency division multiple access.
[0114] It should be noted that in vehicle-to-everything (V2X) communication, subcarrier resources are allocated based on the Orthogonal Frequency Division Multiple Access (OFDMA) mechanism. The transmission rate is calculated by comprehensively considering bandwidth, transmit power, and noise interference. This adapts to the resource scheduling requirements of concurrent communication of multiple vehicles and conforms to the actual communication networking rules of connected vehicles.
[0115] Furthermore, the formula for calculating the data transmission rate is:
[0116]
[0117] in, For data transmission rate, The number of subcarriers allocated to the link. The bandwidth of a single subcarrier. Let be the transmission power of vehicle i. The noise power spectral density;
[0118] In step S230, based on the data transmission rate and compression characteristics, the communication time required to transmit the compression characteristics is calculated to obtain the communication transmission delay.
[0119] It should be noted that calculating the communication transmission time based on the compressed feature data bit volume can accurately match lightweight feature transmission scenarios and avoid latency estimation errors caused by the transmission of raw point cloud big data.
[0120] Furthermore, the formula for calculating the communication time required to transmit compressed features is as follows:
[0121]
[0122] in, The communication time required to transmit compressed features, To compress the bit size of feature data.
[0123] In step S300, based on the vehicle's role in cooperative perception and communication transmission delay, and combined with the computational complexity of the compressed features involved in each role, end-to-end perception delay and energy consumption models are established to obtain the perception delay and energy consumption corresponding to each role.
[0124] It should be noted that corresponding performance models were established to quantify end-to-end latency and energy consumption under different roles, covering different scenarios such as independent, sending, receiving, and dual roles, and adapting to the heterogeneous computing resources and asynchronous collaboration characteristics of vehicles.
[0125] In one possible implementation, step S300 may further include the following sub-steps:
[0126] In step S310, the computational complexity of each stage in the sensing pipeline is defined, including preprocessing costs. Feature extraction cost Projection cost Feature compression cost Feature decompression cost Feature fusion cost Cost of target detection All costs are measured in floating-point operations (FLOPs).
[0127] It should be noted that by uniformly quantifying the computational overhead of each stage of the entire process using the number of floating-point operations, the computational complexity of vehicles with different computing power can be standardized, providing a unified measurement benchmark for modeling differentiated latency and energy consumption.
[0128] In step S320, an end-to-end perceived delay model and energy consumption model are constructed for the independent running role, the sender role, and the receiver role.
[0129] It should be noted that separate models were created for the three basic roles: independent, sender, and receiver. The differences in task division and computational parallelism of each role were fully considered to obtain the latency and energy consumption patterns under different collaboration modes.
[0130] Furthermore, the end-to-end latency-aware model for independently running roles is as follows:
[0131]
[0132] in, End-to-end perceived latency for independently running roles, The computing power of vehicle i;
[0133] The end-to-end perceived energy consumption model for independently running roles is as follows:
[0134]
[0135] in, End-to-end energy consumption awareness for independently operating roles. The average operating power of the calculation unit for vehicle i is calculated.
[0136] The end-to-end perceived delay model for the sender role is as follows:
[0137]
[0138] in, , For the sender role's end-to-end perceived latency, The time required for local feature extraction to complete;
[0139] The end-to-end perceived energy consumption model for the sender role is as follows:
[0140]
[0141] in, End-to-end awareness energy consumption for the sender role. The end-to-end awareness latency and energy consumption model for this sender role is based on the optimal resource allocation assumption of parallel execution of compression tasks and local detection tasks.
[0142] The end-to-end perceived delay model for the receiver role is as follows:
[0143]
[0144] in, , For the end-to-end perceived latency of the receiver role, For the characteristic synchronization time point, The number of cooperating vehicles sending data to vehicle i. The time when the last collaborative feature arrives;
[0145] The end-to-end sensing energy consumption model for the receiver role is as follows:
[0146]
[0147] in, This represents the end-to-end perceived energy consumption for the receiver role. The end-to-end perceived latency and energy consumption model for this receiver role reflects the synchronization constraint that the receiver must wait for all collaborative data to arrive and be decompressed and merged.
[0148] In step S330, an end-to-end perceived delay model and energy consumption model with dual roles are constructed; wherein, dual roles refer to simultaneously acting as a sender and a receiver.
[0149] It should be noted that the dual-role approach takes into account both the parallel processing of its own feature encoding and compression and the reception and decompression of features from neighboring vehicles, which makes up for the shortcomings of existing technologies that only model a single role and is more in line with the actual multi-vehicle cross-cooperative perception application scenarios.
[0150] Furthermore, the end-to-end perceived latency model for dual roles is as follows:
[0151]
[0152] in, End-to-end perception latency for dual roles, The time required for the remaining compression and decompression tasks to be processed in parallel;
[0153] The end-to-end sensing energy consumption model for dual roles is as follows:
[0154]
[0155] in, End-to-end energy consumption sensing for dual roles The total time required to complete local feature encoding and compression. This dual-role end-to-end perception latency and energy consumption model covers the complex computational process of vehicles processing their own data transmission and collaborative data reception in parallel.
[0156] In step S400, based on sensing delay and energy consumption, a multi-objective optimization model based on Chebyshev's method is constructed to transform the three conflicting objectives of sensing accuracy, computational energy consumption, and number of decisions into minimizing the maximum weighted distance.
[0157] It should be noted that by transforming the three conflicting multi-objective optimization problems of perception accuracy, computational energy consumption, and number of decisions into a single-objective optimization problem, the dimensional differences between the different objectives are eliminated, providing a unified optimization objective for subsequent optimization decisions.
[0158] In one possible implementation, step S400 may further include the following sub-steps:
[0159] In step S410, the normalized deviation distance of the three objectives of sensing accuracy, energy consumption, and number of decisions is calculated based on sensing delay and energy consumption.
[0160] It should be noted that extreme value normalization is performed on the three optimization objectives to eliminate the differences in the dimensions and numerical magnitudes of each objective, thus ensuring the fairness and effectiveness of the multi-objective deviation distance calculation.
[0161] Furthermore, taking energy consumption as an example, the formula for calculating the normalized deviation distance of the energy consumption target is:
[0162]
[0163] in, The normalized deviation distance for energy consumption targets. This represents the current cumulative energy consumption value. This represents the minimum energy consumption under ideal conditions. This represents the maximum energy consumption under the worst-case scenario.
[0164] In step S420, based on the normalized deviation distance, the three conflicting objectives of perception accuracy, computational energy consumption, and number of decisions are transformed into minimizing the maximum weighted distance, and a multi-objective optimization model is constructed.
[0165] It should be noted that by constructing a single-objective substitution function by minimizing the maximum deviation distance through weighting, the optimal decision variables for multiple objectives can be solved quickly, which is suitable for the low-latency solution requirements of real-time decision-making in autonomous driving.
[0166] Furthermore, the multi-objective optimization model is as follows:
[0167]
[0168] Where X is the set of decision variables, including the decision interval and the transmission object. The weighting coefficients for perception accuracy. To achieve the normalized deviation distance of the target for sensing accuracy. This is a weighting factor for energy consumption. The normalized deviation distance for energy consumption targets. The weighting coefficients for the number of decisions. The normalized deviation distance is the target number of decision-making attempts.
[0169] In step S500, based on the local state of each vehicle, an action strategy is asynchronously generated through an Actor network with a self-attention mechanism, and the global state value is evaluated through a Critic network with a graph attention mechanism to output the action strategy.
[0170] It should be noted that by using a multi-agent reinforcement learning algorithm, asynchronous decision-making by vehicles can be achieved. Each vehicle can autonomously decide the timing, target, and decision interval for cooperation based on its own local state, without the need for global synchronization. At the same time, the accuracy of policy and value assessment is improved through the attention mechanism.
[0171] In one possible implementation, step S500 may further include the following sub-steps:
[0172] In step S510, a local state vector containing single-vehicle perception accuracy, cumulative energy consumption, and vehicle motion information is constructed. .
[0173] It should be noted that the constructed local state vector integrates multi-dimensional information such as perception, energy consumption, and motion to comprehensively represent the real-time operating conditions of a single vehicle, providing complete state input support for the generation of intelligent agent strategies.
[0174] In step S520, based on the local state vector, the local state is processed by an Actor network with a self-attention mechanism to obtain the action policy.
[0175] It should be noted that the self-attention mechanism can autonomously filter key features of local states, weaken the interference of redundant information, and improve the accuracy of Actor networks in dynamically generating collaborative decisions, target selection, and decision intervals.
[0176] Furthermore, the formula for calculating attention features is as follows:
[0177]
[0178] in, For attention features, Let h be the weight of the h-th attention head. For value vectors, To output the projection matrix, The hidden layer features are the input states.
[0179] In step S530, the global state value is evaluated through a Critic network with a graph attention mechanism, and an action policy is output.
[0180] It should be noted that the graph attention mechanism dynamically models the topological relationships and interaction weights between vehicles, adapting to the highly dynamic characteristics of vehicle network topology, enabling the Critic network to accurately assess the global collaborative value.
[0181] Furthermore, the formula for calculating the graph attention weights is as follows:
[0182]
[0183] in, Let j be the importance weight of vehicle j to vehicle i. The original attention coefficients between vehicle i and vehicle j are... Let V be the original attention coefficient between vehicle i and vehicle k, and V be the set of all vehicles within the current communication range;
[0184] The action strategy output is:
[0185]
[0186] in, To output the action, For a binary decision of whether to transmit, The target vehicle's identification number. The time interval for the next decision is selected from a preset discrete set D.
[0187] In step S600, a collaborative perception task is performed based on compressed features and action strategies. The compressed features are decompressed and fused to obtain fused features, and target detection is performed based on the fused features to obtain detection results.
[0188] It should be noted that the execution of collaborative perception is based on the action strategy obtained from the decision. The receiving end decompresses the compressed features from multiple sources and fuses them with local features to obtain more complete perception features, thereby completing target detection and eliminating the perception blind spots of the vehicle.
[0189] In one possible implementation, step S600 may further include the following sub-steps:
[0190] In step S610, the receiving vehicle i decompresses the received compressed features and fuses them with local features to obtain fused features.
[0191] It should be noted that the receiving end integrates local features with multi-vehicle collaborative compression features to achieve cross-view and anti-occlusion environmental feature enhancement, effectively compensating for the blind spot defects of single-vehicle perception.
[0192] Furthermore, the feature fusion calculation formula is as follows:
[0193]
[0194] in, To show the enhanced feature map after fusion, For multimodal feature fusion function, For feature decompression network, To decompress the network parameters, Let i be the set of sending vehicles that transmit data to receiving vehicle i.
[0195] In step S620, the detection head performs target detection on the fused features to obtain the detection result.
[0196] It should be noted that by relying on fusion-enhanced features to carry out 3D target detection, the location, size, category and confidence level of obstacles can be output simultaneously, which can meet the business needs of autonomous driving environment perception and path planning.
[0197] Furthermore, the calculation formula for object detection is as follows:
[0198]
[0199] in, For the first The 3D bounding box parameters of each detected object include its position, size, and angle. For object categories, To detect the confidence score, The total number of objects detected. To detect head neural networks, To detect the network parameters of the head.
[0200] In step S700, based on the perception accuracy, energy consumption, and time delay performance after the collaborative perception task is executed, the multi-objective reward value is calculated according to the multi-objective optimization model, and the parameters of the Actor network and the Critic network are updated by truncating the near-end policy optimization loss function and the mean square error loss function, respectively, to achieve iterative optimization.
[0201] It should be noted that the reward value is calculated based on the actual performance after collaborative execution, and the parameters of the decision network are updated so that the system can be continuously iterated and optimized to adapt to different traffic environments and continuously improve the system's performance.
[0202] In one possible implementation, step S700 may further include the following sub-steps:
[0203] In step S710, the instantaneous reward at the current moment is calculated based on the multi-objective optimization model. .
[0204] It should be noted that the reward function introduces a latency penalty constraint, and the indicator function determines whether the latency exceeds the limit. This takes into account multiple indicators such as perception accuracy, energy consumption and latency, and guides the agent to iterate towards balanced performance.
[0205] Furthermore, the formula for calculating instant rewards is as follows:
[0206]
[0207] in, For instant rewards, The delay penalty coefficient, For indicator functions, This represents the maximum latency of the current system. This is the maximum allowed delay threshold.
[0208] In step S720, the Actor network parameters are updated by optimizing the loss function using a truncated proximal policy. .
[0209] It should be noted that the truncated PPO loss function is used to limit the policy update magnitude, avoid sudden policy divergence during training, and ensure the smooth convergence of the collaborative decision-making policy of the Actor network.
[0210] Furthermore, the truncated proximal policy optimization loss function is:
[0211]
[0212] in, Optimize the loss function for the near-end policy. For Actor network parameters, The probability ratio between the old and new strategies. To calculate the advantage function value using generalized advantage estimation, This is a truncation function. To truncate hyperparameters.
[0213] In step S730, the Critic network parameters are updated using the mean square error loss function to achieve iterative optimization.
[0214] It should be noted that by fitting the true value of the global state through the mean squared error loss function and using a discount factor to consider long-term future returns, the accuracy of the Critic network in assessing the collaborative state of multiple vehicles can be improved.
[0215] Furthermore, the mean squared error loss function is:
[0216]
[0217] in, Let the mean squared error loss function be . As a discount factor, For the Critic network's estimation of the value of the state, Let r be the value estimate of the next state by the Critic network, and r be the immediate reward.
[0218] Furthermore, in this example embodiment, a multi-target cooperative perception system for connected autonomous vehicles is also provided, used to execute the above-described multi-target cooperative perception method for connected autonomous vehicles. The system may include:
[0219] The feature extraction and compression module is used to acquire raw environmental data, construct local perceptual features through a deep learning network, compress the features, and output compressed features.
[0220] The communication modeling module is used to construct path loss and channel gain models based on the positional relationships between vehicles, and to calculate communication transmission delay based on orthogonal frequency division multiple access.
[0221] The latency and energy consumption modeling module is used to establish end-to-end perception latency and energy consumption models according to the different roles of vehicles in cooperative perception.
[0222] The multi-objective optimization module is used to construct a multi-objective optimization model based on the Chebyshev method, which transforms sensing accuracy, computational energy consumption and decision-making times into minimizing the maximum weighted distance.
[0223] The asynchronous decision-making module is used to asynchronously generate action strategies based on the local states of each vehicle through an Actor network with a self-attention mechanism, and evaluate the global state value through a Critic network with a graph attention mechanism.
[0224] The collaborative perception and execution module is used to decompress and fuse compressed features according to the action strategy, and output the target detection results based on the fused features;
[0225] The network update module is used to calculate the multi-objective reward value based on the perception accuracy, energy consumption, and time latency performance after execution, and to update the parameters of the Actor network and Critic network respectively.
[0226] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0227] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A multi-target cooperative perception method for connected autonomous vehicles, characterized in that, include: Obtain raw environmental data, construct local perceptual features through a deep learning network, and compress the features to obtain compressed features; Based on the compression characteristics and the positional relationship between vehicles, a path loss and channel gain model is constructed, and the communication transmission delay is calculated using orthogonal frequency division multiple access. Based on the vehicle's role in collaborative perception and the communication transmission delay, and combined with the computational complexity of the compression features involved in each role, end-to-end perception delay and energy consumption models are established to obtain the perception delay and energy consumption corresponding to each role. Based on the aforementioned sensing delay and energy consumption, a multi-objective optimization model based on the Chebyshev method is constructed to transform the three conflicting objectives of sensing accuracy, computational energy consumption, and number of decision-making into minimizing the maximum weighted distance. Based on the local state of each vehicle, an action strategy is asynchronously generated through an Actor network with a self-attention mechanism, and the global state value is evaluated through a Critic network with a graph attention mechanism to output the action strategy; the local state includes perception accuracy, cumulative energy consumption, and motion information. Based on the compressed features and the action strategy, a collaborative perception task is performed. The compressed features are decompressed and fused to obtain fused features. Target detection is then performed based on the fused features to obtain detection results. Based on the perception accuracy, energy consumption, and latency performance after the collaborative perception task is executed, the multi-objective reward value is calculated according to the multi-objective optimization model. The parameters of the Actor network and the Critic network are updated by truncating the near-end policy optimization loss function and the mean square error loss function, respectively, to achieve iterative optimization.
2. The method as described in claim 1, characterized in that, The steps of acquiring raw environmental data, constructing local perceptual features through a deep learning network, and compressing the features to obtain compressed features include: The raw point cloud is collected by vehicle-mounted LiDAR and preprocessed to obtain the preprocessed point cloud. Based on the preprocessed point cloud computing bird's-eye view features, a feature extraction network and projection operation are used; The bird's-eye view features are compressed to obtain the compressed features.
3. The method as described in claim 2, characterized in that, The calculation formula for the preprocessed point cloud is as follows: in, For preprocessed point cloud data, The raw LiDAR point cloud data collected for vehicle i This is the function for voxel downsampling. Extraction function for region of interest, The transformation matrix for converting the point cloud to the vehicle coordinate system; The formula for calculating the bird's-eye view features is as follows: in, For the bird's-eye view feature tensor, A neural network for point cloud feature extraction. These are the weight parameters of the feature extraction network. This is an operation function that projects high-dimensional features onto a two-dimensional bird's-eye view plane; The formula for calculating the compression feature is: in, For compression features, For feature compression neural network functions, These are the weight parameters for compressing the network.
4. The method as described in claim 1, characterized in that, The step of constructing a path loss and channel gain model based on the compression characteristics and the positional relationship between vehicles, and calculating the communication transmission delay using orthogonal frequency division multiple access, includes: The path loss between vehicles is calculated based on the Euclidean distance between them, and the channel gain is calculated based on the path loss. Based on the channel gain, the data transmission rate is calculated using orthogonal frequency division multiple access; Based on the data transmission rate and the compression feature, the communication time required to transmit the compression feature is calculated to obtain the communication transmission delay.
5. The method as described in claim 4, characterized in that, The formula for calculating the path loss between vehicles is: in, This refers to path loss between vehicles. Let be the Euclidean distance between vehicle i and vehicle j. For carrier frequency; The formula for calculating the channel gain is: in, Channel gain; The formula for calculating the data transmission rate is: in, For data transmission rate, The number of subcarriers allocated to the link. The bandwidth of a single subcarrier. Let be the transmission power of vehicle i. The noise power spectral density; The formula for calculating the communication time required to transmit the compression feature is as follows: in, The communication time required to transmit compressed features, To compress the bit size of feature data.
6. The method as described in claim 1, characterized in that, The steps of establishing end-to-end perception latency and energy consumption models based on the vehicle's role in cooperative perception and the communication transmission latency, and combining the computational complexity of the compression features involved in each role, to obtain the perception latency and energy consumption corresponding to each role, include: Define the computational complexity of each stage in the perception pipeline, including preprocessing cost, feature extraction cost, projection cost, feature compression cost, feature decompression cost, feature fusion cost, and target detection cost; Construct an end-to-end perceived latency model and energy consumption model for independently operating roles, sender roles, and receiver roles; Construct an end-to-end perceived latency model and energy consumption model with dual roles; wherein, the dual roles refer to simultaneously acting as a sender and a receiver.
7. The method as described in claim 1, characterized in that, The step of constructing a multi-objective optimization model based on Chebyshev's method, which is used to transform the three conflicting objectives of sensing accuracy, computational energy consumption, and number of decision-making steps into minimizing the maximum weighted distance, includes: Based on the aforementioned perception delay and energy consumption, the normalized deviation distance is calculated for three objectives: perception accuracy, energy consumption, and number of decisions. Based on the normalized deviation distance, the three conflicting objectives of perception accuracy, computational energy consumption, and number of decisions are transformed into minimizing the maximum weighted distance, and a multi-objective optimization model is constructed.
8. The method as described in claim 1, characterized in that, The steps of asynchronously generating action policies based on the local states of each vehicle using an Actor network with a self-attention mechanism, evaluating the global state value using a Critic network with a graph attention mechanism, and outputting the action policies include: Construct a local state vector that includes single-vehicle perception accuracy, cumulative energy consumption, and vehicle motion information; Based on the local state vector, the local state is processed by an Actor network with a self-attention mechanism to obtain the action policy; The global state value is evaluated using a Critic network with a graph attention mechanism, and the action policy is output.
9. The method as described in claim 1, characterized in that, The steps of performing a collaborative perception task based on the compressed features and the action strategy, decompressing and fusing the compressed features to obtain fused features, and performing target detection based on the fused features to obtain detection results include: The receiving vehicle decompresses the received compressed features and fuses them with local features to obtain fused features; The detection head performs target detection on the fused features to obtain the detection results.
10. The method as described in claim 1, characterized in that, The steps of iterative optimization, based on the perception accuracy, energy consumption, and latency performance after the collaborative perception task is executed, calculating the multi-objective reward value according to the multi-objective optimization model, and updating the parameters of the Actor network and Critic network by truncating the near-end policy optimization loss function and the mean squared error loss function respectively, include: The instantaneous reward at the current moment is calculated based on the multi-objective optimization model. The Actor network parameters are updated by optimizing the loss function through a truncated proximal strategy. Iterative optimization is achieved by updating the Critic network parameters using the mean squared error loss function.