An unmanned flow vehicle perception feature output method and system oriented to end-to-end regulation and control

Abstract

This application relates to the field of autonomous driving perception technology, specifically a method and system for outputting perception features of unmanned logistics vehicles for end-to-end planning and control. By outputting continuous BEV fusion features and implicit feature vectors, it replaces discrete obstacle boxes and lane line point sets, preserving the semantic, spatial, motion, and texture details of the scene, providing rich environmental representation for the planning and control module. Both the implicit feature encoder and the uncertainty estimation head are connected to the planning controller through a differentiable interface, enabling the planning loss to propagate back to the perception module, achieving true end-to-end joint training and thus improving overall decision-making performance. The uncertainty estimation head outputs variance and other evaluation results for each BEV grid, allowing the planning and control module to perceive the reliability of the perception results. When the uncertainty exceeds a threshold, the system automatically switches to a conservative planner based on structured output for a safety fallback; when uncertainty is low, high-performance end-to-end planning is used, balancing efficiency and safety.

Description

Technical Field

[0001] This application relates to the field of autonomous driving perception technology, specifically a method and system for outputting perception features of unmanned logistics vehicles for end-to-end control. Background Technology

[0002] The two-stage end-to-end architecture divides the autonomous driving system into a perception module and a control module. The perception module is responsible for converting sensor inputs into a structured representation of the environment, while the control module performs trajectory planning and decision-making based on these representations. This architecture combines the fine-tunability of end-to-end learning with the interpretability of modular design.

[0003] However, traditional perception modules output discretized, deterministic results, such as obstacle boxes and lane line point sets. This output format has the following problems:

[0004] (1) Severe information loss: The continuous and rich perceptual information is compressed into discrete boxes, losing information such as the shape details, feature textures, and uncertainties of the object;

[0005] (2) Non-differentiable transitivity: Discrete outputs block the back propagation of gradients from the control module to the sensing module, making it impossible to achieve true end-to-end joint optimization;

[0006] (3) Lack of uncertainty: The confidence level, variance and other information of the perception module cannot be transmitted to the regulatory control module, resulting in a lack of risk perception in regulatory control decisions. Summary of the Invention

[0007] In view of this, the purpose of this application is to provide a method and system for outputting perception features of unmanned logistics vehicles for end-to-end control, so as to solve the problems in the background art.

[0008] To achieve the above objectives, this application adopts the following technical solution:

[0009] This application discloses a method for outputting perception features of unmanned logistics vehicles for end-to-end control, which is executed by a pre-built perception model and includes the following steps:

[0010] Acquire surround view images captured by cameras of unmanned logistics vehicles, laser point cloud data collected by lidar, vehicle status data collected by vehicle perception sensors, and high-precision map data;

[0011] Multi-scale features of the panoramic image are extracted, and point cloud features of the laser point cloud data are extracted. The multi-scale features and the point cloud features are then fused to obtain fused features, wherein the fused features are BEV features.

[0012] The fused features are input into a pre-built structured perception detection head to obtain structured perception results, wherein the structured perception results include obstacle perception results and lane line perception results; the fused features are input into a pre-built implicit feature encoder to obtain implicit feature vectors; and the fused features are input into a pre-built uncertainty estimation head to obtain uncertainty evaluation results.

[0013] The fused features, the implicit feature vector, the vehicle state data, and the high-precision map data are input into a pre-built planning controller to obtain the trajectory distribution for multiple future time steps. The implicit feature encoder and the uncertainty estimation head are both connected to the planning controller through a differentiable interface.

[0014] When the uncertainty assessment result exceeds a preset uncertainty threshold, the path of the unmanned logistics vehicle is planned based on the structured perception result and a pre-built rule-based conservative planner; otherwise, the trajectory distribution of the multiple future time steps is used as the path of the unmanned logistics vehicle.

[0015] In one embodiment of this application, the multi-scale features are extracted by a pre-constructed backbone network, and the point cloud features are extracted by a pre-constructed point cloud encoder; the structured perception detection head includes a 3D detection head, a lane line detection head, and a segmentation head; the implicit feature encoder is an MLP perception network; and the planning controller is a Transformer-based trajectory prediction network.

[0016] In one embodiment of this application, the multi-scale features and the point cloud features are fused to obtain fused features, including:

[0017] Define the BEV query vector and transform the BEV mesh coordinates to the 3D world coordinate system;

[0018] Based on deformable attention, the BEV query vector is interacted with the multi-scale features and multiple point cloud features in multiple rounds to obtain fused features.

[0019] In one embodiment of this application, the method for constructing the perception model includes:

[0020] Obtain structured perception training samples and their labels, wherein the structured perception training samples include surround view image samples and LiDAR point cloud data samples; the labels of the structured perception training samples include 3D obstacle detection box labels and lane line labels.

[0021] The backbone network extraction, the point cloud encoder, the structured perception detection head, and the implicit feature encoder are trained based on the structured perception training samples, the labels of the structured perception training samples, and the pre-constructed one-stage training loss function.

[0022] When training the backbone network extraction, the point cloud encoder, the structured perception detection head, and the implicit feature encoder is completed, planning samples and labels of planning samples are obtained. The planning samples include fused feature samples, implicit feature samples, vehicle state samples, and high-precision map samples. The labels of the planning samples include planning trajectory labels.

[0023] The parameters of the backbone network extraction, the point cloud encoder, the structured perception detection head, and the implicit feature encoder are frozen, and the planning controller is trained based on the acquired planning samples, the labels of the planning samples, and the pre-constructed two-stage training loss function.

[0024] When training the planning controller is completed, joint training samples and labels of the joint training samples are obtained. The joint training samples include surround view image samples, lidar point cloud data samples, vehicle status data samples and high-precision map samples. The labels of the joint training samples include 3D obstacle detection box labels, lane line labels and planning trajectory labels.

[0025] The perception network is jointly trained based on the joint training samples to obtain the perception model.

[0026] In one embodiment of this application, training is performed on the backbone network extraction, the point cloud encoder, the structured perception detection head, and the implicit feature encoder based on the structured perception training samples, the labels of the structured perception training samples, and a pre-constructed one-stage training loss function, including:

[0027] S11, the structured perception training samples are input into the network input layer to obtain the structured perception prediction results output from the structured perception detection head;

[0028] S12, calculate the perceptual loss between the structured perception prediction result and the labels of the structured perception training samples based on the pre-constructed one-stage training loss function, wherein the mathematical expression of the one-stage training loss function is:

[0029]

[0030] In the formula, Indicates perceived loss. Indicates 3D detection loss. The weights represent the 3D detection loss. For lane line segmentation loss, The weights representing the lane line segmentation loss are: This represents the implicit characteristic distillation loss. The weights representing the implicit feature distillation loss;

[0031] S13, Based on the backpropagation of the perceptual loss, the internal parameters of the backbone network extraction, the point cloud encoder, the structured perceptual detection head, and the implicit feature encoder are adjusted using the gradient descent method;

[0032] S14. Repeat steps S11-S13 until training is complete.

[0033] In one embodiment of this application, training the planning controller based on the acquired planning samples, the labels of the planning samples, and a pre-constructed two-stage training loss function includes:

[0034] S21, The planning sample is input into the network input layer to obtain the predicted trajectory output from the planning controller;

[0035] S22, calculate the planning loss between the predicted trajectory and the labels of the planned samples based on the pre-constructed two-stage training loss function, wherein the mathematical expression of the two-stage training loss function is:

[0036]

[0037] In the formula, Indicates planning losses, Indicates trajectory loss, Indicates a loss of comfort. The weight representing the loss of comfort. For collision damage, The weight representing the collision loss;

[0038] S23, Based on the backpropagation of the planning loss, the internal parameters of the planning controller are adjusted using the gradient descent method;

[0039] S24. Repeat steps S21-S23 until training is complete.

[0040] In one embodiment of this application, the perception network is jointly trained based on the joint training samples to obtain a perception model, including:

[0041] S31, the joint training samples are input into the network input layer to obtain the predicted trajectory output from the planning controller and the structured perception prediction result output from the structured perception detection head;

[0042] S32, calculate the joint loss based on the pre-constructed joint training loss function, wherein the mathematical expression of the two-stage training loss function is:

[0043]

[0044]

[0045] In the formula, Indicates joint loss, and All are equilibrium coefficients. This represents the uncertainty regularization loss. For the height of fusion features, For the width of the fused features, For BEV grid index, This is the implicit feature vector output by the implicit feature encoder. The target of the implicit feature vector. Indicates the variance of the prediction;

[0046] S33, Based on the backpropagation of the joint loss, the internal parameters of each sub-network of the perception model are adjusted by combining the gradient descent method;

[0047] S34. Repeat steps S31-S33 until training is complete.

[0048] This application also provides a perception feature output system for unmanned logistics vehicles oriented towards end-to-end regulatory control, including:

[0049] The acquisition module is used to acquire surround view images collected by the camera of the unmanned logistics vehicle, laser point cloud data collected by the lidar, vehicle status data collected by the vehicle's perception sensors, and high-precision map data.

[0050] The feature extraction module is used to extract multi-scale features of the panoramic image, extract point cloud features of the laser point cloud data, and fuse the multi-scale features and the point cloud features to obtain fused features, wherein the fused features are BEV features.

[0051] The perception module is used to input the fused features into a pre-built structured perception detection head to obtain structured perception results, wherein the structured perception results include obstacle perception results and lane line perception results; input the fused features into a pre-built implicit feature encoder to obtain implicit feature vectors; and input the fused features into a pre-built uncertainty estimation head to obtain uncertainty evaluation results.

[0052] The planning and control module is used to input the fused features, the implicit feature vector, the vehicle state data and the high-precision map data into a pre-built planning controller to obtain the trajectory distribution of multiple future time steps. The implicit feature encoder and the uncertainty estimation head are both connected to the planning controller through a differentiable interface.

[0053] The decision module is used to plan the path of the unmanned logistics vehicle based on the structured perception results and a pre-built rule-based conservative planner when the uncertainty assessment result exceeds a preset uncertainty threshold; otherwise, the trajectory distribution of the multiple future time steps is used as the path of the unmanned logistics vehicle.

[0054] This application also provides an electronic device, including: a processor and a memory;

[0055] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to cause the electronic device to perform the methods described above.

[0056] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0057] The beneficial effects of this application are as follows: This application provides a method and system for outputting perception features of unmanned logistics vehicles for end-to-end planning and control. By outputting continuous BEV fusion features and implicit feature vectors, it replaces discrete obstacle boxes and lane line point sets, preserving the semantic, spatial, motion, and texture details of the scene, avoiding information compression loss, and providing rich environmental representation for the planning and control module. Both the implicit feature encoder and the uncertainty estimation head are connected to the planning controller through a differentiable interface, enabling the planning loss to propagate back to the perception module, achieving true end-to-end joint training, thereby improving overall decision-making performance. The uncertainty estimation head outputs variance and other evaluation results for each BEV grid, allowing the planning and control module to perceive the reliability of the perception results. When the uncertainty exceeds a threshold, the system automatically switches to a conservative planner based on structured output to provide a safety net; when the uncertainty is low, high-performance end-to-end planning is adopted, balancing efficiency and safety. This application achieves rich information transmission, gradient integration, and risk-controllable decision-making while retaining modular interpretability. Attached Figure Description

[0058] The present application will be further described below with reference to the accompanying drawings and embodiments:

[0059] Figure 1 This is a system architecture diagram of a perception model shown in one embodiment of this application;

[0060] Figure 2This is a diagram showing the specific network structure of the perception model in one embodiment of this application;

[0061] Figure 3 This is a flowchart illustrating a method for outputting perception features of an unmanned logistics vehicle for end-to-end regulatory control, as shown in one embodiment of this application.

[0062] Figure 4 This is a schematic diagram of uncertainty graph output in one embodiment of this application;

[0063] Figure 5 This is a structural diagram of a perception feature output system for an unmanned logistics vehicle oriented towards end-to-end control, as shown in one embodiment of this application. Detailed Implementation

[0064] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0065] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the layers related to this application and are not drawn according to the actual number, shape and size of the layers in the actual implementation. In the actual implementation, the form, number and proportion of each layer can be arbitrarily changed, and the layer layout may also be more complex.

[0066] Numerous details are explored in the following description to provide a more thorough explanation of embodiments of this application; however, it will be apparent to those skilled in the art that embodiments of this application may be practiced without these specific details.

[0067] This application aims to provide a method for outputting perception features of unmanned logistics vehicles for end-to-end regulatory control, in order to solve the following technical problems existing in the prior art:

[0068] (1) Discrete sensing outputs result in severe information loss and cannot support refined planning;

[0069] (2) The output is not differentiable, which blocks the joint optimization of perception and control;

[0070] (3) Lack of uncertainty transmission makes it impossible for regulatory decisions to perceive risks.

[0071] To solve the above-mentioned technical problems, this application adopts the following technical solution.

[0072] Figure 1 This is a system architecture diagram of a perception model shown in one embodiment of this application, such as... Figure 1 As shown, this application includes:

[0073] Perception Feature Extraction Module: Converts sensor input into multi-level perception features;

[0074] Structured output module: Outputs explicit structured information (object boxes, lane lines, etc.) for interpretability requirements;

[0075] Implicit feature output module: Outputs implicit representations such as BEV feature map and object feature vector, which are directly input into the planning and control module;

[0076] Uncertainty estimation module: Estimates the uncertainty (variance, confidence level, probability distribution) for each output;

[0077] Differentiable interface design: ensures that all outputs are differentiable with respect to the inputs and supports gradient backpropagation.

[0078] Figure 2 Here is a specific network structure diagram of the perception model shown in one embodiment of this application, such as... Figure 2 As shown, this application includes:

[0079] The input layer is used to input images and laser point cloud data;

[0080] Backbone network extraction is used to extract multi-scale features from images;

[0081] Point cloud encoder is used to extract point cloud features from laser point cloud data;

[0082] The fusion module is used to fuse multi-scale features and point cloud features based on deformable attention to obtain BEV features (fused features).

[0083] The structured detection head is used to perceive 3D obstacles and lane lines based on fused features, and output discrete structured perception structures.

[0084] MLP network and planning controller: The MLP network is used to extract implicit features from the fused features and feed the implicit features into the planning controller to output the trajectory at multiple future time points;

[0085] An uncertainty estimation head is used to output uncertainty based on fused features to evaluate the reliability of the current predicted trajectory. If unreliable, the process switches to a rule-based conservative planner to perform path planning.

[0086] Figure 3 This is a flowchart illustrating a method for outputting perception features of unmanned logistics vehicles for end-to-end regulatory control, as shown in one embodiment of this application. Figure 3As shown, the method for outputting perception features of unmanned logistics vehicles for end-to-end control in this application mainly includes the following steps:

[0087] S310 acquires surround view images collected by the camera of the unmanned logistics vehicle, laser point cloud data collected by the lidar, vehicle status data collected by the vehicle's perception sensors, and high-precision map data.

[0088] This step involves synchronously collecting four types of data at a fixed frequency (e.g., 10Hz) during the operation of the unmanned logistics vehicle, providing input for subsequent perception and control.

[0089] (1) Surround view image acquisition

[0090] Sensor configuration: Six industrial-grade cameras are installed around the vehicle, specifically arranged as follows: two for front-view (covering long distance and wide field of view), one for rear-view, and one each for the left front / left rear / right front / right rear, ensuring 360° coverage without blind spots. All cameras use global shutter sensors with a resolution of 1920×1200, a frame rate of 30Hz, and a dynamic range of 120dB, suitable for nighttime and backlighting scenarios.

[0091] A hardware synchronization signal generator (PTP or GPS clock synchronization) provides the same trigger pulse to all cameras, ensuring that the timestamp deviation of multi-view images is less than 1ms.

[0092] Acquire raw RGB images and store them in JPEG or lossless PNG format according to camera number and timestamp. At the same time, record the camera's intrinsic parameters (focal length, principal point, distortion coefficient) and extrinsic parameters (rotation matrix and translation vector relative to the rear axle center of the vehicle body).

[0093] (2) LiDAR point cloud data acquisition

[0094] A 128-line rotating lidar (such as RoboSense RS-Ruby or Hesai AT128) is installed on the roof of the vehicle, with a vertical field of view of -25° to +15°, a horizontal field of view of 360°, a ranging range of 200m, a frame rate of 10Hz, and an output point cloud of approximately 1.5 million points per second.

[0095] Each point contains three-dimensional coordinates (x, y, z) relative to the radar center, reflection intensity, and a timestamp (in microseconds). The raw point cloud is recorded in binary format (such as .pcap or .bin).

[0096] The extrinsic parameters of the lidar and the vehicle coordinate system are obtained through offline calibration, and the point cloud is uniformly transformed into the vehicle coordinate system with the rear axis center as the origin (x forward, y left, z upward).

[0097] (3) Acquisition of vehicle status data

[0098] Use a combined navigation system (such as GNSS+RTK+IMU) and wheel speedometer.

[0099] Data content: Record the vehicle's state vector at each sampling time. ,in:

[0100] The position (in meters) of the rear axle center in the global coordinate system is provided by RTK-GNSS with an accuracy of ≤2cm.

[0101] The heading angle (in radians) is calculated by dual-antenna GNSS or fused IMU.

[0102] The longitudinal velocity (m / s) is obtained by fusing the wheel speed gauge with the IMU.

[0103] The longitudinal acceleration (m / s²) is measured directly by the IMU.

[0104] The integrated navigation system outputs data with GPS timestamps, which are consistent with the clock source of the camera and LiDAR (using PTP or GPS synchronization) to ensure that all data are aligned on the same time base.

[0105] (4) Acquisition of high-precision map data

[0106] Pre-made high-precision maps (such as those in NDS or OpenDrive format) are stored on the vehicle's solid-state drive and loaded on demand by the map engine.

[0107] Includes the following static environment information:

[0108] Lane markings: lane centerline (cubic spline curve), lane boundary lines, lane width, lane type (straight, left turn, etc.);

[0109] Curbs and guardrails: polygons or sets of points;

[0110] Traffic signs: location, type (speed limit, stop, etc.), and area of ​​impact;

[0111] Road topology: lane connections, intersection areas, and virtual reference lines.

[0112] Based on the current vehicle location ( and heading angle The system retrieves map elements within a 200m radius centered on the vehicle and converts them into vectorized representations (such as lane line sampling point sets) or rasterized feature maps in the BEV coordinate system.

[0113] (5) Data preprocessing and alignment

[0114] Timestamp alignment: All sensor data (images, point clouds, vehicle status) uses the same master clock (GPS time). For data that does not arrive strictly simultaneously (such as vehicle status within a LiDAR frame interval), a linear interpolation method is used to align it to the same time (e.g., based on the LiDAR frame time).

[0115] Spatial alignment: Using a pre-calibrated extrinsic matrix, the point cloud coordinates are transformed to the vehicle coordinate system, and the image pixels are associated with the point cloud through collinearity equations. When generating BEV features, multimodal data needs to be unified to a Cartesian coordinate system with the rear axle center of the vehicle body as the origin, the x-axis pointing forward, and the y-axis pointing left.

[0116] Data filtering: outlier removal (radius filtering) and ground filtering (optional, retaining non-ground points for obstacle detection) are performed on the point cloud; distortion correction and exposure compensation are performed on the image.

[0117] Through the above step S310, the system obtains a multimodal data stream that has undergone time synchronization, spatial alignment, and preprocessing, providing high-quality input for subsequent feature extraction and fusion.

[0118] S320, extract the multi-scale features of the panoramic image and the point cloud features of the laser point cloud data, and fuse the multi-scale features and the point cloud features to obtain the fused features, wherein the fused features are BEV features;

[0119] In this application, multi-scale features of a panoramic image are extracted based on a backbone network. Point cloud features are extracted using a point cloud encoder. The backbone network and the point cloud encoder are trained together during subsequent training, and finally fused using a deformable attention mechanism. The feature fusion process includes:

[0120] S321, Define the BEV query vector and transform the BEV mesh coordinates to the three-dimensional world coordinate system;

[0121] This step initializes the learnable query vector in the BEV space and generates corresponding 3D world coordinates for each BEV mesh, so that deformable sampling can be performed in the image and point cloud space later.

[0122] For example, creating a parameter tensor for a science department. This tensor is randomly initialized at the start of training and updated via gradient descent throughout the end-to-end training process. Each position vector This will serve as the initial query for the grid, and subsequent multimodal information will be aggregated through an attention mechanism.

[0123] For each BEV grid The three-dimensional points of the reference points are calculated in the vehicle coordinate system, and the three-dimensional reference points are projected onto the pixel coordinate system of each camera using the pre-calibrated camera intrinsic and extrinsic parameter matrix.

[0124] S322, Based on deformable attention, the BEV query vector is interacted with the multi-scale features and multiple point cloud features in multiple rounds to obtain fused features.

[0125] This step utilizes a multi-layer deformable attention module to adaptively aggregate information from multi-scale image features and point cloud features for each BEV query vector, progressively updating it to fused features. The steps include:

[0126] S322-1, Initialize BEV query: Generate an initial feature vector (learnable parameters) for each BEV grid as the "query representative" for that grid.

[0127] S322-2, Generate reference points: Transform the center coordinates of each BEV grid from the BEV plane to the real 3D world coordinates (e.g., ground height), and then project them onto the images of each camera to obtain reference pixels in the images; at the same time, find the laser points near the 3D location in the point cloud.

[0128] S322-3, Deformable Sampling: For each reference point, the network does not fixate on the features of that point, but instead learns an offset that allows the sampling point to adaptively move to more important surrounding locations (such as the edge or center of an object). Features are then extracted from the multi-scale feature map of the image at these offset locations. For point clouds, several points are selected from nearby laser points, and their coordinates and intensity features are extracted.

[0129] S322-4, Attention Aggregation: Calculate the importance weight (attention score) of each sampled feature, and sum all sampled image features and point cloud features according to their weights to obtain the fused information.

[0130] S322-5, Update BEV query: Add the fused information obtained from aggregation to the original query vector, and transform it through a feedforward network to generate the updated BEV query.

[0131] S322-6, Multi-round Interaction: Repeat the above "sampling-aggregation-update" process multiple times (e.g., 6 layers). Each layer can re-predict the sampling offset and attention weights based on the new query, gradually refining the features. After multiple rounds of interaction, the query vector of each BEV grid integrates rich information related to its own position from the image and point cloud.

[0132] S322-7, Output Fusion Features: The query vectors of all BEV grids are finally combined to form a complete BEV feature map, which contains both the texture and color information of the image and the precise geometric structure of the point cloud. The whole process is differentiable and supports end-to-end training.

[0133] S330, the fused features are input into a pre-built structured perception detection head to obtain structured perception results, wherein the structured perception results include obstacle perception results and lane line perception results; the fused features are input into a pre-built implicit feature encoder to obtain implicit feature vectors; and the fused features are input into a pre-built uncertainty estimation head to obtain uncertainty evaluation results;

[0134] This application employs dual-stream output, specifically including:

[0135] Stream A: Structured output (for interpretability and redundancy check), including 3D obstacle detection boxes and attributes, lane line key point sequences, drivable area polygons, traffic sign locations and categories;

[0136] Stream B: Implicit feature output (for end-to-end planning), including: (1) Dense BEV feature map: size H×W×C, containing semantic, spatial and motion information of the scene, which can be directly input into the learning-based planner; (2) Sparse object feature vector: each detected object corresponds to a feature vector, including appearance features, motion features, interaction features, etc., which can be interacted with the planning module through the attention mechanism; (3) Scene context vector: a compressed representation of the global scene, containing context information such as weather, lighting, and road type.

[0137] In addition, the uncertainty estimation header estimates the uncertainty for each output, including:

[0138] For structured output: output probability distribution parameters (such as the mean and variance of a Gaussian distribution);

[0139] For implicit features: the confidence mask or attention weight of the output feature;

[0140] It is uniformly represented as an uncertainty map U, which is output synchronously with the feature map F.

[0141] Figure 4 This is a schematic diagram of uncertainty graph output in one embodiment of this application. Figure 4 The uncertainty of each grid in the BEV feature map is shown in the form of a heatmap, with darker colors indicating higher uncertainty.

[0142] S340, the fused features, the implicit feature vector, the vehicle state data and the high-precision map data are input to the pre-built planning controller to obtain the trajectory distribution of multiple future time steps, wherein the implicit feature encoder and the uncertainty estimation head are both connected to the planning controller through a differentiable interface;

[0143] In this application, all outputs are encapsulated in a standardized tensor format, and each output node records the computation graph, supporting gradient backpropagation. A perception-control interface protocol is defined, including: data structure definition, coordinate system alignment, timestamp alignment, and gradient propagation switch.

[0144] S350, when the uncertainty assessment result exceeds the preset uncertainty threshold, the path of the unmanned logistics vehicle is planned based on the structured perception result and the pre-built rule-based conservative planner; otherwise, the trajectory distribution of the multiple future time steps is used as the path of the unmanned logistics vehicle.

[0145] The uncertainty estimation head outputs the variance for each grid cell in the BEV feature map. To obtain the overall reliability of the scenario, the system calculates the mean of global uncertainty. :

[0146]

[0147] In the formula, For the height of fusion features, The width of the fused feature;

[0148] like Less than or equal to the threshold: The system considers the current perception result reliable and the implicit feature quality high. In this case, the multimodal trajectory distribution output by the planning controller (end-to-end planner) is directly adopted. Usually, the trajectory with the highest probability or lowest cost is selected from K modalities as the final path, or multiple modalities are fused to generate a smooth trajectory.

[0149] like If the threshold is exceeded: The system determines that the perception uncertainty is too high (such as rainy or foggy weather, sensor occlusion, or off-site distribution), and the implicit features are unreliable. At this time, the end-to-end planner is automatically disabled and switched to a rule-based conservative planner, whose input is only the structured perception results (3D obstacle boxes, lane line point sets, and drivable area polygons), and does not rely on implicit features.

[0150] The conservative planner employs a classic hierarchical planning architecture, contains no learnable parameters, and is entirely based on geometry and rules.

[0151] Based on the current lane lines, obstacle boxes, and vehicle status, determine the appropriate action (e.g., lane keeping, following the vehicle in front, obstacle avoidance, or pulling over). Example rule:

[0152] If the distance to the nearest obstacle is less than the safe distance, the decision is to either "slow down and follow" or "change lanes to overtake".

[0153] If the lane markings disappear or uncertainty remains high, the decision is to "pull over".

[0154] Motion planning layer: Employing either optimization-based path planning or pure tracking control.

[0155] Path generation: Fit a cubic spline curve to the lane line point set as a reference line, and then use A* or Dijkstra's algorithm to search for a collision-free path within the drivable area.

[0156] Velocity planning: Use a PID controller or model predictive control (MPC) to track the reference velocity and take into account the expansion boundary of the obstacle box.

[0157] Control output: Generates a smooth sequence of trajectory points (position, velocity, acceleration) and sends it to the vehicle chassis actuator.

[0158] Smooth transition handling: To avoid trajectory jumps caused by mode switching, the system adopts a gradual transition strategy:

[0159] when When the threshold is exceeded from low to high, the system does not switch immediately. Instead, it triggers conservative mode only after three consecutive frames have exceeded the threshold.

[0160] At the switching time, the trajectory output by the current end-to-end planner is used as the initial reference for the conservative planner to achieve trajectory connection.

[0161] when Once the temperature recovers below the threshold and remains stable, switch back to end-to-end mode.

[0162] Based on the above principles and beneficial effects, step S350 realizes an intelligent decision-making mechanism that "pursues high performance under low uncertainty and ensures safety under high uncertainty," which is a key redundancy design for unmanned logistics vehicles to move towards large-scale application.

[0163] The above method relies on the perception model constructed in this application. The perception model in this application adopts a multi-detector and perception modality scheme and is trained using a three-stage joint training method, specifically including:

[0164] (1) First stage

[0165] The goal of the first stage is to enable the perception modules (backbone network, point cloud encoder, structured detection head, and implicit feature encoder) to learn to extract high-quality BEV features from multi-sensor inputs and output accurate structured perception results (3D obstacle boxes, lane lines) as well as usable implicit features. The output of this stage provides a reliable intermediate representation for subsequent traffic control modules.

[0166] (1-1) Obtain structured perception training samples and labels for the structured perception training samples, wherein the structured perception training samples include surround view image samples and lidar point cloud data samples; the labels for the structured perception training samples include 3D obstacle detection box labels and lane line labels.

[0167] (1-2) Based on the structured sensing training samples, the labels of the structured sensing training samples, and the pre-constructed one-stage training loss function, the backbone network extraction, the point cloud encoder, the structured sensing detection head, and the implicit feature encoder are trained. The specific training process includes:

[0168] S11, the structured perception training samples are input into the network input layer to obtain the structured perception prediction results output from the structured perception detection head;

[0169] S12, calculate the perceptual loss between the structured perception prediction result and the labels of the structured perception training samples based on the pre-constructed one-stage training loss function, wherein the mathematical expression of the one-stage training loss function is:

[0170]

[0171] In the formula, Indicates perceived loss. Indicates 3D detection loss. The weights represent the 3D detection loss. For lane line segmentation loss, The weights representing the lane line segmentation loss are: This represents the implicit characteristic distillation loss. The weights representing the implicit feature distillation loss;

[0172] Specifically, 3D detection loss It includes classification loss (Focal Loss) and regression loss, and the specific calculation formula is as follows:

[0173]

[0174]

[0175]

[0176] In the formula, Represents classification loss. Indicates regression loss, Indicates the total number of obstacle categories. For category indexing, Indicates the first Truth-like tags, Indicates the predicted first The probability of a class To focus parameters, The coordinates of the obstacle's center are... These represent the length, width, and height of the obstacle, respectively. Indicates the orientation angle of the obstacle. For regression parameters The predicted value of any one of them, Indicates predicted value The corresponding truth value.

[0177] In addition, to ensure the consistency of the dimensions of the calculation formula, the regression parameters It has been pre-processed by normalization.

[0178] The formula for calculating lane line segmentation loss is:

[0179]

[0180]

[0181]

[0182] In the formula, This indicates Dice's loss. The weights represent the Dice loss. This represents the binary cross-entropy loss. The weights represent the binary cross-entropy loss. Represents the grid for prediction The probability of belonging to a lane line, This is a true binary mask (1 indicates that the grid belongs to the lane line, and 0 indicates that it does not). This is a smoothing term.

[0183] Implicit characteristic distillation loss The formula for calculation is:

[0184]

[0185] In the formula, For the number of channels, For channel indexing, Location in the BEV feature map predicted for the student network No. The value of each channel, The corresponding feature values ​​output by the teacher network (pre-trained, with better performance) are used as the distillation target.

[0186] In this application, the teacher model can be a pre-trained large model (such as BEVFormer-large) or a self-maintained temporal EMA model. In actual deployment, the teacher model is only used during the training phase, and only the student network is retained during inference.

[0187] S13, Based on the backpropagation of the perceptual loss, the internal parameters of the backbone network extraction, the point cloud encoder, the structured perceptual detection head, and the implicit feature encoder are adjusted using the gradient descent method;

[0188] S14. Repeat steps S11-S13 until training is complete.

[0189] (2) Second stage

[0190] The second phase aims to enable the planning and control module (planning controller) to learn to output accurate future multimodal trajectory distributions based on fixed implicit perceptual features (BEV feature map, object feature vector, scene context), vehicle status, and high-precision map, while the perception module parameters are frozen. This phase ensures that the planning and control module can correctly utilize the perceptual features without being disturbed by changes in perception.

[0191] (2-1) When training the backbone network extraction, the point cloud encoder, the structured perception detection head and the implicit feature encoder is completed, planning samples and labels of planning samples are obtained, wherein the planning samples include fused feature samples, implicit feature samples, vehicle state samples and high-precision map samples, and the labels of planning samples include planning trajectory labels.

[0192] (2-2) Freeze the parameters of the backbone network extraction, the point cloud encoder, the structured perception detection head, and the implicit feature encoder, and train the planning controller based on the acquired planning samples, the labels of the planning samples, and the pre-constructed two-stage training loss function, specifically including:

[0193] S21, The planning sample is input into the network input layer to obtain the predicted trajectory output from the planning controller;

[0194] S22, calculate the planning loss between the predicted trajectory and the labels of the planned samples based on the pre-constructed two-stage training loss function, wherein the mathematical expression of the two-stage training loss function is:

[0195]

[0196] In the formula, Indicates planning losses, Indicates trajectory loss, Indicates a loss of comfort. The weight representing the loss of comfort. For collision damage, The weight representing the collision loss;

[0197] Planning losses The formula for calculation is:

[0198]

[0199] In the formula, Represents the optimal modal index, satisfying ;

[0200] Indicates the first Modal prediction probability;

[0201] Indicates the time step for future prediction;

[0202] Indicates scene scale;

[0203] For time step index;

[0204] Describes the th mode under optimal mode Predicted position of the step;

[0205] Indicates the first The truth value position of the step;

[0206] Loss of comfort The formula for calculation is:

[0207]

[0208] In the formula, Indicates the first The acceleration vector of the step. Indicates the first The acceleration vector of the step. Indicates the sampling time interval. The set maximum speed;

[0209] Collision loss The formula for calculation is:

[0210]

[0211] In the formula, As a preset safe distance, For the first The step predicts the distance from the location to the nearest obstacle.

[0212] S23, Based on the backpropagation of the planning loss, the internal parameters of the planning controller are adjusted using the gradient descent method;

[0213] S24. Repeat steps S21-S23 until training is complete.

[0214] (3) Third stage

[0215] The third stage aims to unfreeze all network parameters and backpropagate the planning loss to the perception module, optimizing the perception features in a direction favorable to the planning task. Simultaneously, an uncertainty regularization loss is introduced, allowing the perception module to learn to estimate the uncertainty of its own output. Ultimately, this achieves joint optimization of perception and control, improving overall driving performance and safety.

[0216] (3-1) When training the planning controller is completed, obtain joint training samples and labels of joint training samples, wherein the joint training samples include surround view image samples, lidar point cloud data samples, vehicle status data samples and high-precision map samples, and the labels of the joint training samples include 3D obstacle detection box labels, lane line labels and planning trajectory labels.

[0217] (3-2) Based on the joint training samples, the entire perception network is jointly trained to obtain the perception model, specifically including:

[0218] S31, the joint training samples are input into the network input layer to obtain the predicted trajectory output from the planning controller and the structured perception prediction result output from the structured perception detection head;

[0219] S32, calculate the joint loss based on the pre-constructed joint training loss function, wherein the mathematical expression of the two-stage training loss function is:

[0220]

[0221]

[0222] In the formula, Indicates joint loss, and All are equilibrium coefficients. This represents the uncertainty regularization loss. For the height of fusion features, For the width of the fused features, For BEV grid index, This is the implicit feature vector output by the implicit feature encoder. The target of the implicit feature vector. Indicates the variance of the prediction;

[0223] S33, Based on the backpropagation of the joint loss, the internal parameters of each sub-network of the perception model are adjusted by combining the gradient descent method;

[0224] S34. Repeat steps S31-S33 until training is complete.

[0225] The three-stage joint training process described above, through end-to-end joint optimization, specifically adjusts the perception features to serve the planning task. Compared to independently trained modular systems, this results in higher trajectory prediction accuracy and smoother driving. The model can output the uncertainty of each BEV grid, allowing the planning module to judge the reliability of the perception results and automatically switch to a conservative planner when the uncertainty is too high, providing a safety net. Continuous, high-dimensional implicit features (BEV feature maps, object feature vectors) are used instead of discrete structured outputs, preserving rich scene details and avoiding information compression loss. All implicit outputs remain differentiable, allowing the planning loss to be directly backpropagated to the perception module, achieving global joint optimization and overcoming the gradient blocking problem of traditional modular architectures. The three-stage training strategy (perception first, then planning, then joint) avoids training instability caused by end-to-end random initialization, while retaining the interpretability of structured outputs for debugging and redundancy verification. The third stage can use only planning labels (trace ground truth) and self-supervised uncertainty loss, reducing reliance on expensive perception labels (3D bounding boxes, lane lines) and facilitating continuous optimization using a large amount of unannotated real vehicle data. The final model supports both high-performance end-to-end planning and retains conservative rule backups based on structured output, meeting the stringent safety and robustness requirements of L4 autonomous logistics vehicles.

[0226] The following are specific embodiments of this application:

[0227] Example 1: Dense BEV Feature Output: In a logistics vehicle project, the perception module outputs a 200×200×64 BEV feature map, covering an 80m×80m area around the vehicle, with a resolution of 0.4m / pixel. This feature map is directly input into a CNN-based planning module, which generates the trajectory for the next 5 seconds. Through end-to-end joint training, the final trajectory prediction error is reduced by 15%.

[0228] Example 2: Sparse Object Feature Output: The perception module detects a slowly moving vehicle 30 meters ahead. In addition to outputting its 3D bounding box, it also outputs a 128-dimensional feature vector, encoding the vehicle's relative speed, acceleration, historical trajectory, and intention (such as turn signal status). The traffic control module interacts with this feature vector through an attention mechanism to predict the vehicle's intention and plan overtaking or following strategies accordingly.

[0229] Example 3: Uncertainty Transmission: In a rainy test, the perception module had low confidence in detecting distant objects (high uncertainty). The uncertainty map transmitted this information to the control module, which then adopted a conservative strategy (deceleration, increasing the safe distance) to avoid overreacting to objects with low confidence. Post-event analysis showed that this mechanism effectively avoided two potential collision risks.

[0230] The specific implementation process is as follows:

[0231] Taking the development of an end-to-end system for a Level 4 unmanned logistics vehicle as an example, the specific implementation process of this invention is as follows:

[0232] Sensing module design:

[0233] Backbone network: Swing Transformer + BEVFormer;

[0234] Structured output heads: 3D detection head, lane line head, segmentation head;

[0235] Implicit feature output head: BEV feature map output layer (200×200×64), object feature extraction layer (128 dimensions per object);

[0236] Uncertainty Header: Predicts uncertainty for both BEV feature map and object features.

[0237] Planning and control module design:

[0238] A Transformer-based trajectory prediction network is used;

[0239] Input: Perceptual features + High-precision map + Vehicle status;

[0240] Output: Probability distribution of multimodal trajectories.

[0241] Joint training:

[0242] Phase 1: Train the perception module separately using supervised learning;

[0243] Phase Two: Freeze the perception module and train the control module;

[0244] Phase 3: Unfreeze all parameters and perform end-to-end joint optimization. The loss function includes the planned trajectory error, comfort penalty, and collision penalty.

[0245] Real vehicle verification:

[0246] In a 5-kilometer urban open road test, the end-to-end jointly optimized system, compared with the independently trained version, reduced the average planning error by 22%, the number of emergency braking incidents by 35%, and the takeover rate by 40%.

[0247] The main features of this application include:

[0248] Lossless information transmission: Rich perceptual information is transmitted to the planning and control module through implicit feature output, avoiding information loss from discrete output.

[0249] End-to-end differentiability: Design a differentiable output interface to support gradient backpropagation from the control module to the sensing module, achieving true end-to-end joint optimization.

[0250] Uncertainty perception: To estimate the uncertainty of the output, the regulatory control module is equipped with risk perception capabilities, thereby improving the safety of decision-making.

[0251] Dual-stream architecture design: Simultaneously outputs structured information and implicit features, balancing interpretability and end-to-end performance, facilitating system debugging and redundancy verification.

[0252] Interface standardization: Define a standardized perception-control interface protocol to support modular development and flexible replacement.

[0253] This application presents a perception feature output method for unmanned logistics vehicles (UAVs) oriented towards end-to-end planning and control. By outputting continuous BEV fusion features and implicit feature vectors, it replaces discrete obstacle boxes and lane line point sets, preserving the semantic, spatial, motion, and texture details of the scene, avoiding information compression loss, and providing rich environmental representation for the planning and control module. Both the implicit feature encoder and the uncertainty estimation head are connected to the planning controller through a differentiable interface, enabling the planning loss to propagate back to the perception module, achieving true end-to-end joint training and thus improving overall decision-making performance. The uncertainty estimation head outputs variance and other evaluation results for each BEV grid, allowing the planning and control module to perceive the reliability of the perception results. When the uncertainty exceeds a threshold, the system automatically switches to a conservative planner based on structured output for a safety fallback; when uncertainty is low, high-performance end-to-end planning is adopted, balancing efficiency and safety. This application achieves rich information transmission, gradient integration, and controllable risk decision-making while retaining modular interpretability.

[0254] like Figure 5 As shown, this application also provides a perception feature output system for unmanned logistics vehicles oriented towards end-to-end control, including:

[0255] The acquisition module is used to acquire surround view images collected by the camera of the unmanned logistics vehicle, laser point cloud data collected by the lidar, vehicle status data collected by the vehicle's perception sensors, and high-precision map data.

[0256] The feature extraction module is used to extract multi-scale features of the panoramic image, extract point cloud features of the laser point cloud data, and fuse the multi-scale features and the point cloud features to obtain fused features, wherein the fused features are BEV features.

[0257] The perception module is used to input the fused features into a pre-built structured perception detection head to obtain structured perception results, wherein the structured perception results include obstacle perception results and lane line perception results; input the fused features into a pre-built implicit feature encoder to obtain implicit feature vectors; and input the fused features into a pre-built uncertainty estimation head to obtain uncertainty evaluation results.

[0258] The planning and control module is used to input the fused features, the implicit feature vector, the vehicle state data and the high-precision map data into a pre-built planning controller to obtain the trajectory distribution of multiple future time steps. The implicit feature encoder and the uncertainty estimation head are both connected to the planning controller through a differentiable interface.

[0259] The decision module is used to plan the path of the unmanned logistics vehicle based on the structured perception results and a pre-built rule-based conservative planner when the uncertainty assessment result exceeds a preset uncertainty threshold; otherwise, the trajectory distribution of the multiple future time steps is used as the path of the unmanned logistics vehicle.

[0260] This application presents a perception feature output system for unmanned logistics vehicles (UAVs) oriented towards end-to-end planning and control. By outputting continuous BEV fusion features and implicit feature vectors, it replaces discrete obstacle boxes and lane line point sets, preserving the semantic, spatial, motion, and texture details of the scene, avoiding information compression loss, and providing rich environmental representation for the planning and control module. Both the implicit feature encoder and the uncertainty estimation head are connected to the planning controller through a differentiable interface, enabling the planning loss to propagate back to the perception module, achieving true end-to-end joint training, thereby improving overall decision-making performance. The uncertainty estimation head outputs variance and other evaluation results for each BEV grid, allowing the planning and control module to perceive the reliability of the perception results. When the uncertainty exceeds a threshold, the system automatically switches to a conservative planner based on structured output for a safety fallback; when uncertainty is low, high-performance end-to-end planning is adopted, balancing efficiency and safety. This application achieves rich information transmission, gradient integration, and risk-controllable decision-making while retaining modular interpretability.

[0261] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0262] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory so that the terminal performs any of the methods in this embodiment.

[0263] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0264] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0265] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0266] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0267] In the above embodiments, although the present application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. The embodiments of the present application are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims.

[0268] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for outputting perception features of unmanned logistics vehicles for end-to-end planning and control, characterized in that, Performed by a pre-built perception model, including the following steps: Acquire surround view images captured by cameras of unmanned logistics vehicles, laser point cloud data collected by lidar, vehicle status data collected by vehicle perception sensors, and high-precision map data; Multi-scale features of the panoramic image are extracted, and point cloud features of the laser point cloud data are extracted. The multi-scale features and the point cloud features are then fused to obtain fused features, wherein the fused features are BEV features. The fused features are input into a pre-built structured perception detection head to obtain structured perception results, wherein the structured perception results include obstacle perception results and lane line perception results; the fused features are input into a pre-built implicit feature encoder to obtain implicit feature vectors; and the fused features are input into a pre-built uncertainty estimation head to obtain uncertainty evaluation results. The fused features, the implicit feature vector, the vehicle state data, and the high-precision map data are input into a pre-built planning controller to obtain the trajectory distribution for multiple future time steps. The implicit feature encoder and the uncertainty estimation head are both connected to the planning controller through a differentiable interface. When the uncertainty assessment result exceeds a preset uncertainty threshold, the path of the unmanned logistics vehicle is planned based on the structured perception result and a pre-built rule-based conservative planner; otherwise, the trajectory distribution of the multiple future time steps is used as the path of the unmanned logistics vehicle.

2. The method for outputting perception features of unmanned logistics vehicles for end-to-end regulatory control according to claim 1, characterized in that, The multi-scale features are extracted by a pre-built backbone network, and the point cloud features are extracted by a pre-built point cloud encoder; the structured perception detection head includes a 3D detection head, a lane line detection head, and a segmentation head; the implicit feature encoder is an MLP perception network; and the planning controller is a Transformer-based trajectory prediction network.

3. The method for outputting perception features of unmanned logistics vehicles for end-to-end control according to claim 1, characterized in that, The multi-scale features and the point cloud features are fused to obtain fused features, including: Define the BEV query vector and transform the BEV mesh coordinates to the 3D world coordinate system; Based on deformable attention, the BEV query vector is interacted with the multi-scale features and multiple point cloud features in multiple rounds to obtain fused features.

4. The method for outputting perception features of unmanned logistics vehicles for end-to-end regulatory control according to claim 2, characterized in that, The method for constructing the perception model includes: Obtain structured perception training samples and their labels, wherein the structured perception training samples include surround view image samples and LiDAR point cloud data samples; the labels of the structured perception training samples include 3D obstacle detection box labels and lane line labels. The backbone network extraction, the point cloud encoder, the structured perception detection head, and the implicit feature encoder are trained based on the structured perception training samples, the labels of the structured perception training samples, and the pre-constructed one-stage training loss function. When training the backbone network extraction, the point cloud encoder, the structured perception detection head, and the implicit feature encoder is completed, planning samples and labels of planning samples are obtained. The planning samples include fused feature samples, implicit feature samples, vehicle state samples, and high-precision map samples. The labels of the planning samples include planning trajectory labels. The parameters of the backbone network extraction, the point cloud encoder, the structured perception detection head, and the implicit feature encoder are frozen, and the planning controller is trained based on the acquired planning samples, the labels of the planning samples, and the pre-constructed two-stage training loss function. When training the planning controller is completed, joint training samples and labels of the joint training samples are obtained. The joint training samples include surround view image samples, lidar point cloud data samples, vehicle status data samples and high-precision map samples. The labels of the joint training samples include 3D obstacle detection box labels, lane line labels and planning trajectory labels. The perception network is jointly trained based on the joint training samples to obtain the perception model.

5. The method for outputting perception features of unmanned logistics vehicles for end-to-end regulatory control according to claim 4, characterized in that, The backbone network extraction, the point cloud encoder, the structured perception detection head, and the implicit feature encoder are trained based on the structured perception training samples, the labels of the structured perception training samples, and a pre-constructed one-stage training loss function, including: S11, the structured perception training samples are input into the network input layer to obtain the structured perception prediction results output from the structured perception detection head; S12, calculate the perceptual loss between the structured perception prediction result and the labels of the structured perception training samples based on the pre-constructed one-stage training loss function, wherein the mathematical expression of the one-stage training loss function is: In the formula, Indicates perceived loss. Indicates 3D detection loss. The weights represent the 3D detection loss. For lane line segmentation loss, The weights representing the lane line segmentation loss are: This represents the implicit characteristic distillation loss. The weights representing the implicit feature distillation loss; S13, Based on the backpropagation of the perceptual loss, the internal parameters of the backbone network extraction, the point cloud encoder, the structured perceptual detection head, and the implicit feature encoder are adjusted using the gradient descent method; S14. Repeat steps S11-S13 until training is complete.

6. The method for outputting perception features of unmanned logistics vehicles for end-to-end regulatory control according to claim 4, characterized in that, The planning controller is trained based on the acquired planning samples, the labels of the planning samples, and a pre-constructed two-stage training loss function, including: S21, The planning sample is input into the network input layer to obtain the predicted trajectory output from the planning controller; S22, calculate the planning loss between the predicted trajectory and the labels of the planned samples based on the pre-constructed two-stage training loss function, wherein the mathematical expression of the two-stage training loss function is: In the formula, Indicates planning losses, Indicates trajectory loss, Indicates a loss of comfort. The weight representing the loss of comfort. For collision damage, The weight representing the collision loss; S23, Based on the backpropagation of the planning loss, the internal parameters of the planning controller are adjusted using the gradient descent method; S24. Repeat steps S21-S23 until training is complete.

7. The method for outputting perception features of unmanned logistics vehicles for end-to-end regulatory control according to claim 4, characterized in that, The perception network is jointly trained based on the joint training samples to obtain a perception model, including: S31, the joint training samples are input into the network input layer to obtain the predicted trajectory output from the planning controller and the structured perception prediction result output from the structured perception detection head; S32, calculate the joint loss based on the pre-constructed joint training loss function, wherein the mathematical expression of the two-stage training loss function is: In the formula, Indicates joint loss, and All are equilibrium coefficients. This represents the uncertainty regularization loss. For the height of fusion features, For the width of the fused features, For BEV grid index, This is the implicit feature vector output by the implicit feature encoder. The target of the implicit feature vector. Indicates the variance of the prediction; S33, Based on the backpropagation of the joint loss, the internal parameters of each sub-network of the perception model are adjusted by combining the gradient descent method; S34. Repeat steps S31-S33 until training is complete.

8. A perception feature output system for unmanned logistics vehicles oriented towards end-to-end regulatory control, characterized in that, include: The acquisition module is used to acquire surround view images collected by the camera of the unmanned logistics vehicle, laser point cloud data collected by the lidar, vehicle status data collected by the vehicle's perception sensors, and high-precision map data. The feature extraction module is used to extract multi-scale features of the panoramic image, extract point cloud features of the laser point cloud data, and fuse the multi-scale features and the point cloud features to obtain fused features, wherein the fused features are BEV features. The perception module is used to input the fused features into a pre-built structured perception detection head to obtain structured perception results, wherein the structured perception results include obstacle perception results and lane line perception results; input the fused features into a pre-built implicit feature encoder to obtain implicit feature vectors; and input the fused features into a pre-built uncertainty estimation head to obtain uncertainty evaluation results. The planning and control module is used to input the fused features, the implicit feature vector, the vehicle state data and the high-precision map data into a pre-built planning controller to obtain the trajectory distribution of multiple future time steps. The implicit feature encoder and the uncertainty estimation head are both connected to the planning controller through a differentiable interface. The decision module is used to plan the path of the unmanned logistics vehicle based on the structured perception results and a pre-built rule-based conservative planner when the uncertainty assessment result exceeds a preset uncertainty threshold; otherwise, the trajectory distribution of the multiple future time steps is used as the path of the unmanned logistics vehicle.

9. An electronic device, characterized in that, include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images