Traffic participant behavior recognition collaborative perception method and control equipment

Abstract

The invention discloses a traffic participant behavior recognition collaborative perception method and control equipment, and the method comprises the steps: firstly, taking an extracted multi-modal feature as a diffusion target, and providing a basis for the subsequent feature optimization; secondly, the features are enhanced by injecting Gaussian noise, and the diversity and robustness of the features are improved; and finally, partial mask processing is performed on the features by adopting a progressive sensor discarding training (PSDT) method, a sensor fault scene is simulated, and the adaptability of the model to incomplete data is further enhanced. The processed features are input into a DuAT (check) module, the module adopts a global self-attention module (GLSA) as an encoder block, the features are denoised and refined, and the relevance between the global features and the local features is fully captured. Finally, optimized feature representation is transmitted to a specific task head, and high-precision three-dimensional target detection and aerial view map segmentation are realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent automatic driving car environment perception, and designs an automatic driving car vision cooperative driving environment perception method and control equipment based on map segmentation and target detection. BACKGROUND

[0002] With the rapid development of artificial intelligence, integrated circuits, sensor devices and intelligent transportation, automatic driving has attracted attention from the industry and academia. Environment perception plays an important role in automatic driving, providing important information for decision making, planning and control. In order to control the vehicle safely and effectively, the environment perception system should understand the driving scene and provide inference results to the decision system, including the position and category of objects, free space, motion state of objects, position information of objects, etc., to avoid collision and comply with traffic rules. Free space and motion state are also crucial in automatic driving route planning.

[0003] Single-mode three-dimensional perception technology is mainly divided into two types of methods based on laser radar and camera. The camera-based method has experienced evolution from the early dense detection process to complex architecture, among which representative methods such as DETR3D, BEVFormer and BEVDepth innovatively combine bird's eye view representation with transformer attention mechanism, realizing multi-view fusion. Laser radar-based methods mainly process point cloud data, and its development context gradually evolves from PointNet and other pioneering architectures that process raw point clouds to VoxelNet and other methods that convert point clouds into structured formats such as voxels, distance images and columnar bodies, fully embodying the technological progress in the field of target detection.

[0004] In the field of three-dimensional perception, multi-modal fusion methods can be divided into three categories according to the fusion stage: early fusion, late fusion and feature fusion. Early fusion methods enhance the original point cloud through image features or guide point cloud feature extraction, directly fuse the original data of the two sensors, but there are problems of system delay and error propagation. Late fusion methods operate in the prediction stage, and achieve information integration by fusing prediction results or sensor extracted features, although the efficiency is higher, but it is difficult to capture the feature correlation between sensors, resulting in noise and non-optimal fusion results.

[0005] The feature fusion method can obtain more expressive fusion representation and better performance by fusing deep features at multiple levels of the network, but the feature alignment still faces challenges due to the inherent differences between the laser radar and the camera features. At present, the fusion method based on the transformer captures the relationship between the features by using the encoder-decoder architecture, which has achieved advanced performance in three-dimensional target detection, but its computational complexity is large and it is difficult to adapt to other perception tasks such as bird's eye view map segmentation. Another method based on LSS backbone network can improve two-dimensional image features to three-dimensional space by constructing a unified bird's eye view representation, and apply a multi-task head, which has the advantages of high efficiency and scalability, but is still limited by the noise and misalignment between sensors, which affects the optimization of the fusion effect.

[0006] Three-dimensional target detection and bird's eye view map segmentation are two core tasks in the field of autonomous driving. Three-dimensional target detection aims to realize target recognition and positioning in three-dimensional space, while bird's eye view map segmentation focuses on dividing the bird's eye view into different semantic categories. The current technology development trend emphasizes multi-sensor feature fusion to fully exert the complementary advantages of each sensor. Specifically, although three-dimensional point cloud data can provide accurate geometric information, it lacks color information, which is the strength of image data. Combining the two can significantly improve the performance of BEV semantic segmentation.

[0007] In recent years, fusion methods based on transformers have achieved breakthroughs in three-dimensional target detection tasks by implementing effective learning of sensor feature mapping through cross-attention mechanisms. However, these methods have shown limitations in their adaptability when extended to other tasks such as bird's eye view map segmentation. Another method based on LSS attempts to achieve multi-sensor fusion by constructing a unified BEV representation, but its performance still has the following limitations: first, the architecture design of existing fusion modules is difficult to fully capture the complex relationships between sensors; second, the inherent noise characteristics of different modal data cause noise interference and accuracy problems in the fused features. These limitations restrict the performance of multi-modal fusion methods in practical applications.

[0008] The bird's eye view (BEV) features generated by the model of the present application can be shared and optimized with downstream tasks. In constructing an online vectorization mapping model based on historical grid maps, while mapping traffic participant behavior semantics to high-precision maps, a more comprehensive traffic participant behavior semantic reconstruction technology is realized. SUMMARY

[0009] To solve the above problems, the present application proposes a new framework, SD-FUSER, which is innovative in its three ways of processing radar and camera sensor features: first, the extracted features are taken as diffusion targets, second, Gaussian noise is injected for feature enhancement, and finally, partial mask processing is performed through PSDT. These processed features are then input into the DuAT(neck) module (using a global self-attention module GLSA as the encoder block), where denoising and refinement are performed. Finally, the optimized feature representation is delivered to the specific task head, achieving accurate three-dimensional target detection and bird's eye view map segmentation.

[0010] The structure of the SD-FUSER architecture framework is to first process the input data, including point clouds and images, through respective backbones to create initial latent features. Then these features are concatenated and fed into FUSER blocks. In these blocks, the concatenated features are used as conditions (partially masked) to iteratively denoise the features, improving the quality at each step. The output features are then used for downstream tasks.

[0011] SD-FUSER belongs to the Feature-fusion category. The goal of the present application is to improve the fusion architecture and take advantage of the denoising properties of generative models to mitigate noise and intrinsic differences between different sensors, thereby improving the performance of downstream tasks.

[0012] The encoder and decoder of the network of the present application are VoV99 network and DETR decoder, respectively.

[0013] The technical scheme of the traffic participant behavior recognition collaborative perception method proposed by the present application includes the following steps:

[0014] Step 1: design a feature encoder module; first input the pre-processed lidar point cloud and camera data, process through the backbone network, extract image features and point cloud features;

[0015] Step 2: design DuAT architecture and GLSA module; after processing through the backbone network, enter the diffusion module, which uses the DuAT architecture, specifically including a double aggregation module, namely the Global-to-Local Spatial Aggregation (GLSA) module and the Selective Boundary Aggregation (SBA) module;

[0016] Step 3: design a progressive sensor dropout training strategy; use the Probabilistic Self-Conditional Diffusion Transform (PSDT), the input of the entire fusion process Features including cameras and lidar are utilized in three ways: as training targets, as noisy inputs to a gated self-conditioning modulation (GSM) diffusion module, and as diffusion conditions with random dropout of camera or lidar features, progressively increasing the feature dropout rate of camera or lidar inputs from 0% to a predefined maximum during training.

[0017] Further, in step 2, the GLSA module is responsible for extracting and fusing local and global spatial features from the backbone network, adopting a separate channel, one channel for extracting global representation through a global context block, and the other channel for extracting local information through multiple deep convolutions.

[0018] The SBA module is used to simulate the biological visual perception process to distinguish objects from the background, which combines shallow and deep features to establish the relationship between the target region and the boundary, and enhances the boundary features.

[0019] Further, the shallow and deep features are complementary, the shallow features contain less semantic information, rich details and clear boundaries, and the deep features contain rich semantic information. The SBA module selectively aggregates the boundary information of shallow features and the semantic information of deep features to depict fine-grained target contours and recalibrate the position of the target. The module uses a new recalibration attention unit (RAU) block, which can adaptively extract mutual representations from two inputs (Fs, Fb), Fs refers to shallow features, which are feature maps extracted near the input layer in the neural network; Fb refers to boundary features, which are information related to the target boundary in deep features.

[0020] Further, the new recalibration attention unit (RAU) block specifically includes:

[0021] Boundary feature enhancement path (Path 1): input high-resolution, low-semantic shallow features Path 2): input low-resolution, high-semantic deep features , representing the height, width and channel number of the shallow features, the spatial details are enhanced by the following processing:

[0022] Local context extraction: adopt a dilated convolution to expand the receptive field and capture multi-scale boundary context information;

[0023] Spatial attention weighting: generate a spatial weight map to obtain the boundary high-gradient region;

[0024] Semantic feature refinement path (Path 2): input low-resolution, high-semantic deep features Path 2): input low-resolution, high-semantic deep features , The height, width and channel number of the deep feature are represented, the semantic information is compensated by channel attention, and channel recalibration is performed: a compression-excitation (SE) mechanism is introduced to dynamically adjust the channel importance.

[0025] Further, the channel separation of the GLSA module, specifically the feature map with 64 channels is evenly divided into two feature map groups and respectively input into the global spatial attention (GSA) module and the local spatial attention (LSA) module. The outputs of the two attention units are spliced and then processed by a 1x1 convolution layer. This process is represented as follows:

[0026]

[0027]

[0028] wherein represents the global spatial attention, represents the local spatial attention, is the output feature, represents the separation operation, represents the splicing operation, represents the 1x1 convolution operation.

[0029] Further, the GSA module emphasizes the long-distance relationship of each pixel in the space, which is complementary to the local spatial attention, to generate a global spatial attention map as input, as follows:

[0030]

[0031]

[0032] wherein is the attention operation, represents 1 convolution, represents a multi-layer perceptron composed of two fully connected layers with ReLU nonlinear activation function and normalization layer. The first layer of the MLP converts its input to a higher dimensional space with an expansion ratio of 2, while the second layer restores the dimension to the same as the input.

[0033] Further, the LSA module is used to extract local features of the region of interest in the spatial dimension of the given feature map, to calculate the local spatial attention response as input, as follows:

[0034]

[0035]

[0036] in Indicates three cascaded Convolutional layers and Depth-separable convolutional layers It is a local attention operation. It is the Sigmoid function. It involves multiplying each point individually.

[0037] Furthermore, the gated self-conditional modulation (GSM) diffusion module in step 3 is implemented as follows:

[0038] The gated self-conditional modulation (GSM) diffusion process is encapsulated by operations similar to characteristic linear modulation, and this modulation calculation is based on conditional parameters. The parameters for dynamically adjusting the diffusion trajectory are defined as follows:

[0039]

[0040]

[0041]

[0042]

[0043]

[0044] Here, It is the original sample The result of partial masking It is the output of gated self-conditional modulation (GSM). The gated self-conditional modulation (GSM) module uses parameters... and Noisy input features Dynamic modulation is performed, in which As a self-conditional element of the Sigmoid gate, Indicates the time step that guides the diffusion process. , , representing the weights and biases corresponding to each modulation, respectively.

[0045] Based on the above method, the present invention also proposes an autonomous vehicle control device that can execute the above perception method when in operation.

[0046] The beneficial effects of this invention are:

[0047] 1. First, the extracted multi-modal features are used as diffusion targets to provide a basis for subsequent feature optimization. Second, the features are enhanced by injecting Gaussian noise to improve their diversity and robustness. Finally, the features are partially masked using the Progressive Sensor Dropout Training (PSDT) method to simulate sensor failure scenarios and further enhance the model's ability to adapt to incomplete data. The processed features are input into the DuAT (neck) module, which uses a Global Self-Attention (GLSA) module as the encoder block to denoise and refine the features, fully capturing the relevance of global and local features. Finally, the optimized feature representation is delivered to the specific task head, enabling high-precision 3D target detection and bird's eye view map segmentation.

[0048] 2. The SD-FUSER framework, through the above innovative design, not only significantly improves the effectiveness of multi-modal feature fusion, but also enhances the model's robustness to sensor failures, providing an efficient and reliable solution for perception tasks in autonomous driving and other fields.

[0049] 3. The collaborative perception method proposed in this invention is a conditional generation model with a fusion-enhanced architecture designed for multi-modal and multi-task learning in the range of 3D target detection and bird's eye view (BEV) segmentation. The SD-FUSER of this invention processes the features extracted from millimeter wave radar and camera sensors in three ways: one as a diffusion target, one with Gaussian noise injection, and one with partial masking through Progressive Sensor Dropout Training (PSDT). These are then input into the conditional Global-to-Local Spatial Aggregation (GLSA) module as the encoder block, where they are denoised and refined. This enhanced representation is finally used by the specific task head for precise 3D target detection and bird's eye view segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 Invention implementation flowchart;

[0051] Figure 2 SD-FUSER network overall framework diagram;

[0052] Figure 3 Global-to-Local Spatial Aggregation (GLSA) module architecture diagram;

[0053] Figure 4 Conditional control (GSM) diffusion block diagram. DETAILED DESCRIPTION

[0054] The invention will be further described with reference to the accompanying drawings.

[0055] For the convenience of those skilled in the art to understand, the present application will be further described below in conjunction with the drawings, and the content mentioned in the embodiments is not a limitation on the present application.

[0056] The implementation process of the traffic participant behavior recognition collaborative perception method of the present application is shown in Figure 1 、 2 , and specifically includes the following steps:

[0057] Step one: design feature encoder module (backbone)

[0058] First, input the laser radar point cloud and camera data, and after data preprocessing, simultaneously pass through two backbone networks for feature extraction. The present application uses VoV99 as the backbone network.

[0059] VoV99 solves the low efficiency problem of DenseNet in target detection by introducing One-Shot Aggregation (OSA) technology. OSA retains the multi-receptive field features of DenseNet, while aggregating all features only on the last feature map, significantly reducing memory access cost and computational overhead. Compared with DenseNet and ResNet, the detection model based on VoV99 has a speed increase of 2 times and an energy consumption reduction of 1.6x to 4.1x, especially in small target detection tasks, the performance has been significantly improved. VoV99 optimizes memory access and improves GPU computing efficiency, which not only guarantees high precision, but also greatly improves inference speed and energy efficiency, and is particularly suitable for environments with limited computing resources.

[0060] Step two: design DuAT architecture and GLSA module

[0061] After processing by the backbone network, it enters the diffusion module, which contains the DuAT architecture. The DuAT architecture contains double aggregation modules, namely the Global-to-Local Spatial Aggregation (GLSA) module and the Selective Boundary Aggregation (SBA) module.

[0062] The GLSA module is responsible for extracting and fusing local and global spatial features from the backbone network. The present application separates the channels, one channel is used to extract global representation through a Global context (GC) block, and the other channel is used to extract local information through multiple deep convolutions. The module aims to simulate the biological visual perception process and distinguish objects from the background. Specifically, it combines shallow and deep features to establish the relationship between the target region and the boundary, enhancing the boundary features.

[0063] Shallow and deep features complement each other. Shallow features have less semantic meaning but are rich in detail, have clearer boundaries, and less distortion. Furthermore, deep features contain rich semantic information. Therefore, directly fusing low-level features with high-level features may lead to redundancy and inconsistencies. To address this issue, this invention proposes an SBA module that selectively aggregates boundary information from shallow features and semantic information from deep features to depict finer-grained target contours and recalibrate target positions. Unlike previous fusion methods, this invention designs a novel Recalibration Attention Unit (RAU) block that adaptively selects from both inputs (…) before fusion. The mutual representations are extracted from the layers. Fs refers to shallow features, which are feature maps extracted from shallower layers (closer to the input layer) in the neural network; Fb is the boundary feature, which is the information related to the target boundary in the deeper features. The new recalibrated attention unit (RAU) block is as follows:

[0064] Boundary feature enhancement path ( Path): Input shallow features (High resolution, low semantics) The height, width, and number of channels of shallow features are represented by the following steps to enhance spatial detail:

[0065] Local context extraction: Dilated Convolution is used. Expanding the receptive field to capture multi-scale boundary context ; To indicate dilated convolution, its subscript... Here, k is the original kernel size and d is the hole ratio.

[0066] Spatial attention weighting: Generates a spatial weight map, focusing on high-gradient regions at the boundary. ; ;here This refers to the feature map input to the spatial attention module.

[0067] Semantic feature refinement path ( Path): Input deep features (Low resolution, high semantics) The height, width, and number of channels represent deep features. Semantic information is compensated through channel attention, specifically through channel recalibration: a compression-excitation (SE) mechanism is introduced to dynamically adjust channel importance. ; ;here, It is the original deep feature Channel attention weight The refined characteristics obtained after recalibration. is a channel attention weight vector generated by the SE mechanism.

[0068] denotes channel concatenation, denotes element-wise multiplication.

[0069] Attention mechanisms enhance information relevant to the optimization objective and suppress irrelevant information. To capture both global and local spatial features, the present application proposes a GLSA module that fuses the results of two independent local and global attention units. As shown in Figure 3 , this dual-stream design effectively preserves local and non-local modeling capabilities. In addition, the present application uses split channels to balance accuracy and computational resources. Specifically, a feature map with 64 channels is evenly divided into two feature map groups The subscript denotes the feature map index at different levels in the neural network and is input into the global spatial attention (GSA) module and the local spatial attention (LSA) module, respectively. The outputs of the two attention units are concatenated and then passed through a 1x1 convolution layer. The present application represents this process as

[0070]

[0071]

[0072] where denotes global spatial attention, denotes local spatial attention. is the output feature, denotes a set of real spaces, and the superscript denotes the dimension size, and represent the height channel and the width channel, respectively, denotes the split operation, denotes the concatenation operation, denotes the 1x1 convolution operation. The present application will describe the LSA and GSA modules in detail below. This design ensures the integrity of feature extraction while optimizing the allocation of computational resources.

[0073] (1) GSA module: GSA emphasizes the long-distance relationship of each pixel in the space and can be used as a supplement to local spatial attention. Long-distance interaction can make features more powerful. The present application generates a global spatial attention map as follows:

[0074]

[0075] ​

[0076] wherein is an attention operation, denotes 1convolution. denotes a multi-layer perceptron consisting of two fully connected layers with ReLU nonlinear activation function and normalization layers. The first layer of the MLP transforms its input to a higher dimensional space with an expansion ratio of 2, while the second layer restores the dimension to the same as the input. Transpose stands for tensor dimension transposition; denotes a probability normalization function

[0077] (2) LSA module: The LSA module effectively extracts local features of regions of interest (such as small objects) in the spatial dimension of the given feature map. In short, the present application calculates the local spatial attention response as follows:

[0078]

[0079]

[0080] wherein denotes concatenating three convolutional layers and depthwise separable convolutional layers. The number of channels is adjusted to 32 in . is a local attention operation, is a Sigmoid function, is point-wise multiplication. This structure design can effectively aggregate local spatial information using fewer parameters.

[0081] Step three: design a progressive sensor dropout training method, which is implemented as follows

[0082] Specifically, in the probabilistic self-conditioned diffusion transform (PSDT), the input of the entire fusion process contains the features of the camera and the LiDAR (LiDAR), which is utilized in three key ways: as a training target, as a noisy input for the diffusion module, and as a diffusion condition with randomly dropped camera or LiDAR features. To simulate the situation of sensor loss or failure, the present application gradually increases the dropout rate of camera or LiDAR input from 0% to a predefined maximum value during training. This is expressed as:

[0083]

[0084] wherein,​​ represents the features that are occluded using the mask to simulate the case of sensor dropout or failure, represents the total number of features current epoch in the training epoch, which defines the dropout probability that determines the likelihood of each feature in the modality being dropped. This strategic process is implemented by applying binary masks generated from a Bernoulli distribution, which not only trains the model to effectively denoise and generate more expressive features but also maximizes the model's reliance on any single sensor, thereby enhancing its ability to handle incomplete sensor data with stronger recovery capabilities.

[0085] Step Four: Designing the Gated Self-Conditional Modulation (GSM) Diffusion Module

[0086] Unlike traditional diffusion models, which primarily rely on the time step t as the condition, the Gated Self-Conditional Modulation (GSM) module innovatively combines both the time information (time step ) and the sensor information contained in the partial mask latent sample to guide the diffusion process. This requires stronger conditional constraints on the noisy latent variable to make the diffusion trajectory develop towards generating high-quality features.

[0087] This dual-condition approach, combined with the Probabilistic Self-Conditional Diffusion Transform (PSDT) paradigm and task-specific loss functions, ensures that the diffusion process focuses on feature enhancement rather than simple replication, thereby facilitating the synthesis of more expressive features.

[0088] Specifically, in FUSER, the Gated Self-Conditional Modulation (GSM) module is used in the encoder block (see Figure 2 ). The Gated Self-Conditional Modulation (GSM) diffusion process is encapsulated by a feature-wise linear modulation (FiLM)-like operation. This modulation computation dynamically adjusts the parameters of the diffusion trajectory based on the condition . The formula is defined as:

[0089]

[0090]

[0091]

[0092]

[0093]

[0094] Here,​ is the original sample is the result that can be partially masked by PSDT, i.e., partially masked latent sample, is the output result of the gated self-conditioning modulation (GSM) process. represents the time step of the guided diffusion process. The gated self-conditioning modulation (GSM) module dynamically modulates the noisy input features and by parameters , , , respectively, representing the weights and biases in each dynamic modulation. Among them , as a sigmoid gate self-conditioning element, recalibrates the layer activation based on the features themselves, further enhancing the expression ability of the features. These parameters are derived from the conditional by a convolution block. The present application deliberately limits the gradient flow when introducing noise to prevent the model from memorizing. The overall architecture is shown in Figure 4 .

[0095] The above series of detailed descriptions are only specific descriptions of the feasible implementation of the present application, and are not intended to limit the protection scope of the present application. Any equivalent means or changes that do not deviate from the technology created by the present application shall be included in the protection scope of the present application.

Claims

1. A traffic participant behavior identification collaborative perception method, characterized in that, Firstly, the point cloud and camera data features are extracted and used as diffusion targets, secondly, Gaussian noise is injected for feature enhancement, and then partial mask processing is performed through progressive sensor dropout training; the processed features are input into the DuAT module, which uses a global self-attention module GLSA as an encoder block to perform denoising and refinement within the module, fully capturing the relevance of global and local features; Finally, the optimized feature representation is input into the task detection head to realize accurate three-dimensional target detection and bird's eye view map segmentation.

2. The method of claim 1, wherein, The method is realized by the following steps: Step 1: design a feature encoder module; first input the preprocessed lidar point cloud and camera data, and extract image features and point cloud features after backbone network processing; Step 2: design DuAT architecture and GLSA module; after backbone network processing, enter the diffusion module, which uses DuAT architecture, specifically including a double aggregation module, namely a global-to-local spatial aggregation (GLSA) module and a selective boundary aggregation (SBA) module; Step 3: design a progressive sensor dropout training strategy; The entire fusion process is inputted with probabilistic self-conditioned diffusion transforms (PSDT) Features from the camera and lidar are included and utilized in three ways: as training targets, as noisy input to the gated self-conditioned modulation (GSM) diffusion module, and as diffusion conditions with random dropout of camera or lidar features, with the feature dropout rate of the camera or lidar input gradually increased from 0% to a predefined maximum during training.

3. The method of claim 2, wherein, In step 2, the GLSA module is responsible for extracting and fusing local and global spatial features from the backbone network, using a separate channel, one channel for extracting global representation through a global context block, and the other channel for extracting local information through multiple deep convolutions; The SBA module is used to simulate the biological visual perception process to distinguish objects from the background, which combines shallow features and deep features to establish the relationship between the target area and the boundary, and enhances the boundary features.

4. The method of claim 3, wherein, The shallow features and deep features complement each other, the shallow features contain less semantic information, rich details and clear boundaries, and the deep features contain rich semantic information, the SBA module selectively aggregates the boundary information of the shallow features and the semantic information of the deep features to depict fine-grained target contours and recalibrate the target position, the module uses a new recalibration attention unit (RAU) block, which can adaptively extract mutual representations from two inputs (Fs, Fb), Fs refers to shallow features, feature maps extracted near the input layer in the neural network; Fb refers to boundary features, which are information related to the target boundary in deep features.

5. The method of claim 4, wherein, The new recalibration attention unit (RAU) block specifically includes: Boundary feature enhancement path Path): input high-resolution, low-semantic shallow features , represent the height, width, and channel number of the shallow features, and spatial details are enhanced by the following processing: Local context extraction: uses a dilated convolution to expand the receptive field and capture multi-scale boundary context information; Spatial attention weighting: generate a spatial weight map to obtain the boundary high gradient region; Semantic feature refinement path Path): input low-resolution, high semantic deep features , represent the height, width and channel number of deep features, compensate semantic information through channel attention, and re-label channels: introduce compression-excitation (SE) mechanism to dynamically adjust channel importance.

6. The method of claim 5, wherein, channel separation of the GLSA module, specifically separating the feature map with 64 channels into two groups of feature maps and input into a global spatial attention (GSA) module and a local spatial attention (LSA) module respectively, the outputs of the two attention units are spliced and then processed by a 1x1 convolution layer, and the process is represented as follows: wherein denotes global spatial attention, denotes local spatial attention, is the output feature, denotes a separation operation, denotes a concatenation operation, denotes a 1 x 1 convolution operation.

7. The method of claim 6, wherein, The GSA module emphasizes the long-range relationship of each pixel in the space as a supplement to the local spatial attention to generate a global spatial attention map as input , as follows: wherein is an attention operation, denotes 1 convolution, denotes a multi-layer perceptron consisting of two fully connected layers with ReLU nonlinear activation function and normalization layers, the first layer of the MLP transforms its input to a higher dimensional space with a scaling factor of 2, while the second layer restores the dimension to the same as the input.

8. The method of claim 7, wherein, The LSA module is configured to extract local features of a region of interest in a spatial dimension of a given feature map, so as to compute a local spatial attention response as an input as follows: wherein represents cascading three convolutional layers and depthwise separable convolutional layers, is a local attention operation, is a Sigmoid function, is point-wise multiplication.

9. The method of claim 8, wherein, The GSM diffusion module of step 3 realizes the following: The Gated Self-Conditioned Modulation (GSM) diffusion process is encapsulated by an operation similar to the characteristic linear modulation, which is computed based on the condition parameters The parameters of the diffusion trajectory are dynamically adjusted and defined as: Here, is the original sample is the result of partial masking, is the output result of gated self-conditioning modulation, the gated self-conditioning (GSM) module dynamically modulates the noisy input features and by parameters , as the self-conditioning elements of the Sigmoid gate, denotes the time step that guides the diffusion process, , represent the corresponding weights and biases of each modulation, respectively.

10. An automated vehicle control device, characterized by comprising: The control device can perform the perception method of any one of claims 1-9 when working.

Images