Automatic driving control method and system based on intelligent perception of driving environment

Abstract

The application relates to the technical field of intelligent driving and discloses an automatic driving vehicle control method and system based on intelligent sensing of a driving environment. The method comprises the following steps: acquiring an RGB image sequence and a corresponding depth image sequence in front of a vehicle at multiple continuous moments; performing multi-modal feature extraction on the RGB image sequence and the depth image sequence; generating a bird's-eye view semantic feature; combining the spatio-temporal feature map and the bird's-eye view semantic feature map through a multi-modal multi-view decision state generation module, and performing time series modeling through a gated recurrent unit to output a latent feature vector; and inputting the latent feature vector as a current environment state into a deep deterministic policy gradient network to output a control instruction. The application can not only accurately identify and classify various obstacles on a road, but also can analyze and predict the motion trajectory of the obstacles in real time to support decision making and safety control of the system in a complex traffic scene.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, specifically to an autonomous driving control method and system based on intelligent perception of the driving environment. Background Technology

[0002] Intelligent driving technology has developed rapidly in recent years, demonstrating enormous potential in improving traffic safety, reducing traffic congestion, and increasing travel efficiency. Traditional autonomous driving systems mainly rely on pre-programmed map data and rules; however, the complex environments in real-world driving often present challenges, such as ever-changing road conditions, unexpected obstacles, and sudden movements of pedestrians and other vehicles.

[0003] Existing autonomous driving systems still have limitations in environmental perception and real-time decision-making capabilities, which significantly affect the safety and adaptability of the systems in complex traffic scenarios. Therefore, it is particularly important to develop an autonomous driving control method and system that can efficiently perceive and quickly respond to changes in the surrounding environment. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an autonomous driving control method and system based on intelligent perception of the driving environment, which improves the high-precision perception and comprehensive understanding of the driving environment by unmanned vehicles, thereby enhancing the overall performance of the autonomous driving system.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides an autonomous driving control method based on intelligent perception of the driving environment, comprising the following steps:

[0007] S1, acquire the RGB image sequence of the front of the vehicle and the corresponding depth image sequence at multiple consecutive time points;

[0008] S2, performing multimodal feature extraction on the RGB image sequence and depth image sequence, including:

[0009] S21, using a ResNet50 network that introduces a channel attention mechanism, RGB spatial features and depth spatial features are extracted from the RGB image and depth image, respectively;

[0010] S22, the RGB image sequence and the depth image sequence are input into a long short-term memory network to extract RGB temporal features and depth temporal features;

[0011] S23, through the spatiotemporal context global association module, the RGB spatial features, depth spatial features, RGB temporal features, and depth temporal features are fused to generate a spatiotemporal feature map;

[0012] S3 generates semantic features of the bird's-eye view, including:

[0013] S31, Based on inverse perspective transformation, the RGB image is converted into a bird's-eye view;

[0014] S32, the bird's-eye view is processed using a semantic segmentation network based on an encoder-decoder structure, and semantic feature maps of the bird's-eye view are extracted from the semantic segmentation network;

[0015] S4, through the multi-mode multi-view decision state generation module, the spatiotemporal feature map and the bird's-eye view semantic feature map are combined, and temporal modeling is performed through the gated loop unit to output the latent feature vector;

[0016] S5, the potential feature vector is used as the current environment state and input into the deep deterministic policy gradient network to output control commands for controlling the steering wheel angle, throttle and braking of the vehicle.

[0017] In one embodiment, a channel attention module is added after each residual block of the ResNet50 network, and a dropout layer is set after each fully connected layer of the ResNet50 network; the channel attention module adopts a compressed excitation network, and the excitation operation of the compressed excitation network includes two fully connected layers.

[0018] In one embodiment, the step of fusing the RGB spatial features, depth spatial features, RGB temporal features, and depth temporal features through a spatiotemporal context global association module to generate a spatiotemporal feature map specifically includes:

[0019] The RGB spatial features, depth spatial features, RGB temporal features, and depth temporal features are combined along the channel dimension to form an input feature map.

[0020] The nonlocal operations of a nonlocal neural network are used to perform global association modeling on all pixels in the input feature map. For each pixel in the input feature map, the correlation degree between it and all other pixels is calculated, and the features of all other pixels are weighted and summed according to the correlation degree to generate a response that can represent global context information. The response is then used to update the features of the current pixel, thereby outputting the spatiotemporal feature map.

[0021] In one embodiment, the global correlation modeling of all pixels in the input feature map using the nonlocal operations of a nonlocal neural network is implemented through the following unit:

[0022] The first transformation unit is used to perform embedding transformation on the feature vectors in the input feature map;

[0023] A similarity calculation unit is used to calculate the similarity between transformed feature vectors at any two positions in the input feature map, so as to obtain the correlation degree.

[0024] The response aggregation unit is used to perform a weighted summation of all feature vectors transformed by the first transformation unit according to the correlation degree, and then normalize them to generate the response.

[0025] In one embodiment, the semantic segmentation network based on the encoder-decoder structure specifically includes:

[0026] The encoder of the semantic segmentation network uses the EfficientNetB3 network; the decoder of the semantic segmentation network consists of multiple convolutional blocks containing convolutional layers, batch normalization layers, ReLU activation functions, and upsampling operations; the encoder and decoder are combined through skip connections.

[0027] In one embodiment, the multi-modal, multi-view decision state generation module combines the spatiotemporal feature map and the bird's-eye view semantic feature map, and performs temporal modeling through a gated loop unit, specifically including:

[0028] At each time step, the spatiotemporal feature map and the bird's-eye view semantic feature map are flattened and stitched together to form a combined feature vector;

[0029] The combined feature vector is used as the current input of the gated loop unit, which performs time-series processing to output the latent feature vector.

[0030] In one embodiment, the step of inputting the latent feature vector as the current environment state into a deep deterministic policy gradient network and outputting control commands for controlling the vehicle's steering wheel angle, throttle, and braking specifically includes:

[0031] The latent feature vectors are used as the current environment state in the deep deterministic policy gradient algorithm. ;

[0032] The deep deterministic policy gradient algorithm employs a deep deterministic policy gradient network comprising an actor network and a critic network, wherein the actor network adjusts according to the current environmental state. Output a control command containing steering wheel angle, throttle, and brake as an action. ;

[0033] The critic network is based on the current environmental state. and the action Assess the corresponding value;

[0034] The deep deterministic policy gradient network learns through interaction with the driving environment and optimizes to maximize a predefined cumulative reward, which is an instantaneous reward calculated based on the vehicle's driving state. The driving status includes driving speed, lane keeping status, and whether a collision has occurred.

[0035] In a second aspect, the present invention provides a computer system including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method of any embodiment of the first aspect.

[0036] Compared with the prior art, the beneficial technical effects of the present invention are:

[0037] This invention not only accurately identifies and classifies various obstacles on roads, but also analyzes and predicts their trajectories in real time, supporting system decision-making and safety control in complex traffic scenarios. Furthermore, this invention improves the system's response speed and environmental adaptability by optimizing the selection and configuration of sensing sensors, as well as through innovative data processing and algorithm design. Therefore, this invention can effectively handle various road conditions, including urban and highway environments, and maintain stable and efficient operation under different weather conditions. Attached Figure Description

[0038] Figure 1 This is a flowchart of an autonomous driving control method based on intelligent perception of the driving environment in an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the spatiotemporal context global association module in an embodiment of the present invention.

[0040] Figure 3 This is a schematic diagram of the structure of the multi-mode, multi-view decision state generation module in an embodiment of the present invention.

[0041] Figure 4 This is a schematic diagram of the framework of the DDPG (Actor-Critic) algorithm in an embodiment of the present invention. Detailed Implementation

[0042] To further illustrate the technical solution of this invention, please refer to the following detailed description and accompanying drawings. The drawings are for reference and illustration only and do not limit the scope of protection of this invention.

[0043] like Figure 1 As shown in the figure, this embodiment discloses an autonomous driving control method based on intelligent perception of the driving environment, which specifically includes the following technical solutions.

[0044] 1. Data Acquisition and Preprocessing:

[0045] The first step of the method of this invention is to acquire and preprocess raw sensor data for environmental perception. The goal of this stage is to acquire a continuous data stream containing rich visual and three-dimensional structural information and convert it into a format suitable for deep neural network processing. The entire process mainly includes two core steps: the simultaneous acquisition of multimodal data sequences and the generation of a bird's-eye view.

[0046] 1.1 Synchronous acquisition of multimodal data sequences:

[0047] To achieve comprehensive perception of the dynamic driving environment, this system relies on two complementary visual sensor data sets: RGB images and depth images. Using an onboard sensor suite, the system simultaneously acquires visual information about the area in front of the vehicle within consecutive time steps t=1,2,...,T, forming two time series.

[0048] RGB image sequence: RGB image sequence is obtained by capturing images through a front-facing high-definition camera. Each frame of RGB image It provides rich color, texture, and lighting information, which is fundamental for recognizing objects such as traffic signs, lane lines, vehicles, and pedestrians. and These represent the height and width of the image, respectively.

[0049] Depth image sequence: Synchronized with RGB images, the system acquires a depth image sequence through a depth sensor (such as a stereo camera or a ToF camera). Each frame of depth image Each pixel value directly corresponds to the distance from that point in the scene to the camera. Depth information is crucial for determining object distance, avoiding collisions, and understanding the 3D spatial layout of a scene. In embodiments without a hardware depth sensor, known monocular depth estimation algorithms can also be used to estimate depth from RGB images. The corresponding pseudo-depth image is inferred in real time.

[0050] These two sequences are strictly aligned in time, ensuring that at any time t, and It describes two different modal views of the same scene, laying the data foundation for subsequent multimodal feature fusion.

[0051] 1.2, Bird's-eye view generation:

[0052] While forward-looking images are intuitive, they suffer from the perspective effect of "objects appearing larger when closer and smaller when farther away" and are easily obstructed by vehicles ahead, hindering path planning and global situational awareness. To overcome this deficiency, this invention introduces a bird's-eye view (BEV) as another key representation of the scene. The generation of the bird's-eye view is based on inverse perspective mapping (IPM), the core of which is establishing a mapping relationship between the image's planar coordinates and the ground coordinates in the world coordinate system. This mapping relationship is achieved through a 3×3 homography transformation matrix. This is achieved by... The derivation of this matrix depends on the camera's intrinsic parameter matrix. and extrinsic parameter matrix .

[0053] First, define the coordinate system:

[0054] World coordinate system The origin is usually defined as the projection of the vehicle's rear axle center onto the ground. The axis points to the right side of the vehicle. The axis points forward of the vehicle. The axis is perpendicular to the ground and points upwards. In this coordinate system, points on the ground satisfy... = 0.

[0055] Camera coordinate system With the camera's optical center as the origin, The axis points in the direction of the camera's optical axis.

[0056] Image coordinate system The origin is the top left corner of the image, and the unit is pixels.

[0057] A three-dimensional point in a world coordinate system To the pixel in the image coordinate system The projection relationship is as follows:

[0058] ;

[0059] in, It is a scale factor. It is an intrinsic parameter matrix. It is the extrinsic parameter matrix from the world coordinate system to the camera coordinate system. They are respectively x-axis coordinates, y-axis coordinates, z-axis coordinates. for The x and y coordinates.

[0060] Intrinsic parameter matrix : Describes the optical characteristics of a camera, in the form of ,in It's the focal length. This is the main point.

[0061] extrinsic parameter matrix : Describes the pose of the camera in the world coordinate system. It is a 3×3 rotation matrix that defines the rotation of the camera coordinate system relative to the world coordinate system; It is a 3×1 translation vector that defines the position of the camera's optical center in the world coordinate system. Assume the rotation matrix... The column vectors are respectively Then the extrinsic parameter matrix can be written in 3×4 form: .

[0062] To perform inverse perspective transformation, ground points in the world coordinate system need to be considered, i.e. At this point, the projection formula simplifies to:

[0063] ;

[0064] Ground points in the world coordinate system Points in the image coordinate system Between them, through a 3×3 matrix Related. This matrix It is the homography matrix from the world ground to the image plane.

[0065] The inverse perspective transformation required by this invention is from image points Mapped to world ground points Therefore, the homography matrix required by this invention... It is precisely the matrix mentioned above The reverse:

[0066] ;

[0067] Thus, we obtain the final mapping relationship of the inverse perspective transformation from image coordinates to world ground coordinates:

[0068] ;

[0069] in It is a new scaling factor.

[0070] In practical applications, camera intrinsic and extrinsic parameters can be obtained offline in one go using standard camera calibration procedures (such as using a checkerboard calibration board). Once... Once determined, the fixed homography matrix can be calculated. Subsequently, the system can utilize this matrix to efficiently process each frame of the real-time input RGB image by calling image processing libraries (such as OpenCV's warpPerspective function). Transform into the corresponding bird's-eye view image This process provides unobstructed, uniformly sized input for subsequent semantic segmentation and global feature extraction, greatly simplifying the analysis of drivable areas and obstacle layouts.

[0071] At this point, the data acquisition and preprocessing stage is complete. This invention prepares three key data streams for the subsequent feature extraction module: RGB image sequences... Depth image sequence And bird's-eye view image sequences derived from RGB image sequences. .

[0072] 2. Spatiotemporal feature extraction:

[0073] After data acquisition and preprocessing, this invention enters the core feature extraction stage. The goal of the spatiotemporal feature extraction module is to deeply mine and fuse features that comprehensively reflect the spatial structure and temporal dynamics of the driving scene from continuous RGB image sequences and depth image sequences. The final output of this module is a highly condensed spatiotemporal feature map, which will serve as one of the key inputs for subsequent decision state generation. The entire process is achieved through three closely linked steps: spatial feature extraction, temporal feature extraction, and fusion through a global spatiotemporal context association module.

[0074] 2.1 Spatial Feature Extraction:

[0075] To capture the rich spatial details of the scene at every moment, this invention employs a ResNet50 network with an incorporating channel attention mechanism as the basic spatial feature extractor. The system processes RGB and depth modal data in parallel to extract complementary spatial information.

[0076] Parallel processing architecture: The system instantiates two ResNet50 networks with identical structures but independent weights. One network receives an RGB image. As input, one network focuses on extracting visual appearance features such as color, texture, and contour; another network receives depth images. As input, we focus on extracting the geometric features of the object, such as its three-dimensional shape and relative distance.

[0077] Backbone network ResNet50: ResNet50 was chosen as the backbone network because of its deep residual learning capabilities. It can effectively train very deep networks while avoiding the gradient vanishing problem, thereby learning multi-level features from low-level edges to underlying abstract concepts.

[0078] Channel Attention Enhancement (SENet): To enable the network to more intelligently focus on feature channels with richer information, this invention integrates a Squeeze-and-Excitation Network (SENet) module after each residual block of ResNet50. This module achieves channel attention through the following two steps:

[0079] (1) Squeeze: Global average pooling compresses the spatial dimension of each feature map into a single value, which can be regarded as a global information descriptor of the feature channel.

[0080] (2) Excitation: The channel descriptor vector obtained by the Squeeze operation is input into a small neural network with two fully connected layers (usually a "bottleneck" structure that is reduced in dimension and then increased in dimension), and activated by the Sigmoid function. The network learns and outputs a weight vector with the same dimension as the number of input feature channels, where each weight value is between (0, 1) and represents the importance of the corresponding channel.

[0081] Finally, the learned channel weights are multiplied back onto the original feature map channel by channel, thereby achieving adaptive recalibration of the features. The channel compression ratio r in the activation operation can be set to 16 to achieve a balance between computational efficiency and performance.

[0082] After the above processing, for any time t, the system obtains the RGB spatial features respectively. and depth spatial features .

[0083] 2.2 Temporal Feature Extraction:

[0084] Driving decisions depend not only on the current situation but also on understanding the dynamic evolution of that situation. Therefore, this invention employs a Long Short-Term Memory (LSTM) network to model time-series information.

[0085] Similarly, the system uses a parallel LSTM network structure to process the RGB and depth data streams separately.

[0086] Input: A sequence of RGB images at T consecutive time points and depth image sequence The initial encoding is performed through a lightweight convolutional layer to reduce dimensionality, and then the data is fed into two separate LSTM networks.

[0087] How LSTM works: LSTM, through its ingenious internal gating mechanism—input gate, forget gate, and output gate—can effectively learn long-distance temporal dependencies. It can determine which historical information needs to be forgotten and which current information needs to be remembered, ultimately outputting a hidden state that contains the historical context.

[0088] Output: After processing the entire sequence of length T, the hidden state output by the LSTM network at the last time step T is considered as the RGB temporal features of the sequence. and deep temporal features These two feature vectors encapsulate the dynamic changes of the scene over the past T time points.

[0089] 2.3, Fusion is performed through the global spatiotemporal context association module:

[0090] After obtaining spatial and temporal features, the most crucial step is to deeply and effectively fuse these four heterogeneous features. Simple concatenation or addition cannot capture the complex relationships within and between features. To address this, this invention designs a spatiotemporal context global association module, which, based on the concept of non-local neural networks, can establish global associations for all locations in the feature map.

[0091] (1) Construction of input feature map:

[0092] First, combine the four features obtained in the first two steps. Then combine the RGB spatial features at time T. Depth spatial features RGB time-series features representing the dynamics of the entire sequence Deep temporal features (Typically, the temporal feature vectors are first broadcast along the spatial dimension to match the size of the spatial features) and then concatenated along the channel dimension. This constructs a unified input feature map that contains all spatiotemporal information. . for Width and height.

[0093] (2) Non-local operations:

[0094] The core of this module is the input feature map. Perform nonlocal operations to compute the response of each location's feature to features at all other locations. For the output feature map... For any position i on the vector, its eigenvector The calculation formula is as follows:

[0095] ;

[0096] and These are the input feature maps. The feature vectors at positions i and j are summed and iterated through all possible positions j, achieving global association. To output feature map The eigenvector at position i.

[0097] It is a univariate transformation function used to embed the feature vector at position j.

[0098] It is a similarity calculation function used to evaluate the correlation between positions i and j.

[0099] It is a normalization factor that ensures the sum of the weights at all positions is 1, similar to the Softmax operation.

[0100] This non-local operation process can be understood as being completed collaboratively by the following three core functional units:

[0101] The first transformation unit is responsible for performing embedding transformations on the feature vectors in the input feature map to extract representations for subsequent calculations. In this invention, this corresponds to the function in the formula. It is implemented through a 1x1 convolutional layer, i.e. ,in These are learnable weights.

[0102] Similarity Calculation Unit: This unit is responsible for calculating the correlation between any two locations in the input feature map. This corresponds to the formula... The function. The specific implementation is an embedded Gaussian function. ,in and It is also a linear transformation achieved through 1x1 convolution.

[0103] Response Aggregation Unit: This unit is responsible for weighted summation of all feature vectors processed by the first transformation unit based on the correlation degree obtained from the similarity calculation unit, and then normalizing them to generate the final response. This corresponds to the entire weighted summation and normalization part in the formula. .

[0104] (3) Residual connection:

[0105] To preserve original local information and simplify network optimization, the output of nonlocal operations is typically combined with the original input through a residual connection. This results in the final output being a spatiotemporal feature map. for:

[0106] ;

[0107] in It is a learnable weight matrix (which can also be implemented via 1x1 convolution) used to control the injection strength of global context information.

[0108] Through this module, the system not only integrates four types of features, but more importantly, it ensures that the final feature of each pixel contains global spatiotemporal context information of the entire scene. For example, the features of vehicles in the left lane are now also associated with the dynamic information of vehicles in the right lane.

[0109] The final spatiotemporal feature map output by this module It is a highly structured feature representation that demonstrates a deep understanding of the spatial layout and temporal dynamics of the current driving environment.

[0110] 3. Extraction of semantic features from bird's-eye view:

[0111] A sequence of distortion-free and occluded bird's-eye view (BEV) images was obtained through inverse perspective transformation (IPM). Subsequently, this invention enters the stage of advanced semantic understanding of the scene. Parallel to the extraction of spatiotemporal features from the forward-looking perspective, the goal of this module is to extract structured semantic features from the bird's-eye view that are instructive for driving decisions. These features mainly include lane layout, drivable area range, obstacle location and category, etc. This process is achieved through an advanced semantic segmentation network.

[0112] 3.1 Semantic segmentation based on encoder-decoder architecture:

[0113] This invention employs a deep neural network based on an encoder-decoder architecture to process each frame of the bird's-eye view image. Perform pixel-level semantic segmentation. This architecture is a standard paradigm in the field of image segmentation, effectively balancing the depth of feature extraction with the accuracy of spatial localization.

[0114] Encoder: The encoder's function is feature extraction. It typically consists of a powerful convolutional neural network (CNN) pre-trained on a large image dataset (such as ImageNet). In a preferred embodiment of the invention, the encoder uses the EfficientNetB3 network. EfficientNetB3 was chosen because it jointly optimizes the network depth, width, and resolution through compound scaling, maintaining extremely high accuracy while keeping computational and parameter counts low, making it ideal for real-time automotive applications. The encoder receives a bird's-eye view image. As input, a series of convolution and downsampling operations are used to progressively extract multi-level feature maps, ranging from low-level textures to high-level abstract concepts. As the network depth increases, the spatial resolution of the feature maps gradually decreases, but their semantic information becomes increasingly rich. The final feature map output by the encoder contains a highly condensed understanding of the entire bird's-eye view scene.

[0115] Decoder: The decoder's role is spatial reconstruction and pixel classification. It receives the low-resolution, high-semantic feature map output from the encoder and gradually restores its spatial resolution to that of the original input image through a series of upsampling operations (such as bilinear interpolation or transposed convolution) and convolution operations. The same size. To improve segmentation accuracy, especially the sharpness of object edges, skip connections are established between the decoder and encoder. This means that at each upsampling stage, the decoder concatenates its feature map with the corresponding layer's feature map from the encoder, which has a higher spatial resolution. This design (also known as the U-Net structure) allows the decoder to utilize both deep semantic information and shallow detail information from the encoder for precise localization.

[0116] 3.2 Generation of semantic segmentation maps and semantic feature maps:

[0117] After the complete encoder-decoder network processing, the network usually has a Softmax or Sigmoid activation function in the last layer to output a probability map of each pixel belonging to various predefined categories (such as lane lines, drivable areas, vehicles, pedestrians, background, etc.).

[0118] Assuming this invention predefines K semantic categories, for input The network outputs a probability graph. ,in Represents coordinates in a bird's-eye view The probability that a pixel belongs to the k-th class. Final bird's-eye view semantic segmentation map. This can be obtained by taking the category with the highest probability for each pixel:

[0119] ;

[0120] This semantic segmentation graph This is an intuitive visualization that clearly depicts the static layout of the driving environment. However, for subsequent neural network processing, the segmentation map composed of category labels is used directly. This is not the optimal choice because it loses probabilistic information and has a discrete form. Therefore, this invention directly extracts features from the intermediate layers of the semantic segmentation network as the final output. Specifically, this invention selects the highest-level feature map output by the encoder, or a feature map from an intermediate layer in the decoder that incorporates multi-scale information, as the final output of this module. This feature map is referred to as the bird's-eye view semantic feature map in this invention. .

[0121] choose Instead of the final segmentation image The advantages are: It is a high-dimensional, continuous floating-point tensor. Compared to discrete category labels, it contains richer and more ambiguous semantic information and uncertainty between categories, which is crucial for subsequent fusion and decision-making.

[0122] At this point, the bird's-eye view semantic feature extraction process is complete. It runs parallel to the spatiotemporal feature extraction process, providing the system with a high-level understanding of the global layout and static structure of the driving environment. Its output is a bird's-eye view semantic feature map. Will be combined with spatiotemporal feature maps Together, they will be integrated into the decision state generation module for the next stage.

[0123] 4. Decision state generation:

[0124] Spatiotemporal feature maps representing scene dynamics and local details were extracted in parallel. semantic feature map of bird's-eye view representing the static structure and global layout of the scene. Next, the invention enters the decision state generation stage. The core task of this stage is to integrate these two heterogeneous feature streams from different modalities and perspectives into a unified latent feature vector that can comprehensively characterize the current driving environment state. This vector will serve as the sole input to the final decision module.

[0125] In this invention, the fusion and encoding process is executed through a "multi-modal, multi-view decision state generation module". The core of this module is a temporal modeling method based on a gated recurrent unit (GRU), which can effectively process and fuse the spliced ​​multi-source features.

[0126] 4.1 Multi-source feature combination:

[0127] Before feeding the GRU for temporal modeling, the two feature maps need to be combined into a unified input vector.

[0128] At each time step t, the system will generate a spatiotemporal feature map. and bird's-eye view semantic feature map Flatten The operation transforms the two-dimensional feature map into a one-dimensional feature vector. Then, these two feature vectors are concatenated. This forms a longer combined feature vector that contains all available information. This combined feature vector is then used as the input to the gated recurrent unit (GRU) at the current time t. .

[0129] .

[0130] 4.2 Timing Modeling and State Coding Based on Gated Recurrent Units (GRUs):

[0131] After completing the multi-source feature combination, it is necessary to process the combined feature vector. The final integration and encoding are performed in the time dimension. This invention uses a Gated Recurrent Unit (GRU) to accomplish this task. GRU is a variant of LSTM, with a simpler structure, higher computational efficiency, and comparable performance to LSTM on many tasks.

[0132] Input: At each time step t, the combined feature vector generated in the previous stage is the current input of the GRU. As the current input to the GRU.

[0133] How GRUs work and how they update their status: GRUs use a sophisticated internal reset gate. And the Update Gate It intelligently controls the flow of information. It combines the combined feature inputs of the current moment. and the hidden state of the previous moment It determines which historical information needs to be retained and which current information needs to be integrated, thereby updating its internal state and ultimately outputting a new hidden state. The detailed update formula is as follows:

[0134] (1) Reset the door The hidden state of the previous moment was determined. How much information needs to be "forgotten":

[0135] ;

[0136] in, It is the output of the reset door. It is a weight matrix. It is the hidden state from the previous moment; It is the input at the current moment, by and It is pieced together. It is the Sigmoid activation function.

[0137] (2) Update the door The candidate hidden state is determined at the current moment. How much information needs to be updated to the final hidden state? This also determines how much of the past hidden state should be retained. :

[0138] ;

[0139] in, It updates the output of the gate. It is a weight matrix.

[0140] (3) Candidate hidden state The calculation includes the current input. And some historical information that has been "reset":

[0141] ;

[0142] in, These are candidate hidden states, and W is the weight matrix. This indicates element-wise multiplication.

[0143] (4) Final hidden state The update is achieved by updating the gate. The old hidden state was linearly interpolated. and new candidate hidden states :

[0144] ;

[0145] in, It is the hidden state at the current moment, which serves as the latent feature vector output by the gated recurrent unit.

[0146] Output: The hidden state of the output of the GRU network after processing the input at the current time step. This is the final output of this module—the latent feature vector. , .

[0147] This latent feature vector It is the best numerical representation of the current complete driving environment state that can be directly used by subsequent reinforcement learning algorithms. Through the gating mechanism of GRU, it has implicitly learned how to fuse and weigh information from spatiotemporal features and bird's-eye view semantic features.

[0148] At this point, the decision state generation module is complete. It successfully transforms high-dimensional, complex perceptual features into a low-dimensional state vector suitable for processing by modern reinforcement learning algorithms, laying a solid foundation for the next stage of intelligent decision control.

[0149] 5. Strengthen learning decision control:

[0150] In the decision state generation process, high-dimensional, multimodal perceptual information is successfully encoded into a compact latent feature vector. Then, the invention enters the final decision-making and control stage. The goal of this stage is to, based on the current comprehensive understanding of the environment (i.e., the state vector), This invention generates optimal, continuous control commands that enable safe and smooth vehicle driving. To this end, the invention employs an advanced deep reinforcement learning algorithm—the Deep Deterministic Policy Gradient (DDPG) algorithm.

[0151] The DDPG algorithm is a model-free reinforcement learning algorithm based on the actor-critic architecture, specifically designed to solve problems in continuous action spaces. It enables an agent (i.e., an autonomous driving system) to autonomously learn an optimal driving strategy through continuous interaction and trial and error with the driving environment.

[0152] 5.1 Core components of the DDPG algorithm and their specific definitions in this invention:

[0153] The DDPG algorithm mainly consists of four core network components, whose roles and definitions in this invention are as follows:

[0154] (1) Actor Network: also known as policy network, denoted as Responsible for determining the current status Directly output a deterministic action .

[0155] In this invention, the input (state) of the actor network ( ) represents the latent feature vector output by the decision state generation process. .Right now .

[0156] Actor network output (motion) () is: a vector containing three consecutive values These correspond to the steering wheel angle, throttle opening (0~1), and braking force (0~1), respectively, and control commands are obtained.

[0157] The network structure of an actor's network typically consists of several layers of fully connected neural networks (MLPs), with the last layer using... Activation functions normalize the output value to a specific range (e.g., steering wheel angle between -1 and 1).

[0158] The network parameters of the actor network are denoted as .

[0159] (2) Critic Network: also known as a value network, denoted as Responsible for assessing the status. Next action The quality of output is determined by the long-term expected reward (Q value) of the state-action pair.

[0160] In this invention, the input to the critic network is a state. (i.e., latent feature vector) ) and actions (i.e., the control commands output by the actor's network).

[0161] The output of the critic network is a scalar Q-value, used to evaluate the merits of the current driving decision.

[0162] The network structure of the critic network is usually also an MLP, but its input layer receives a concatenation of states and actions.

[0163] The network parameters of the critic network are denoted as .

[0164] (3) Target Networks: To make the training process more stable and avoid drastic fluctuations in Q-value estimation, DDPG creates a "shadow" network with the same structure but slower parameter updates for both the actor and critic networks, namely the target actor network. and the target critic network .

[0165] Replay Buffer: A large-capacity memory used to store a large number of experience tuples generated during the agent's interaction with the environment. .in It is to perform an action The instant reward obtained afterward It is the next state to be reached.

[0166] 5.2 Design of the Reward Function:

[0167] The design of the reward function is crucial to the success of reinforcement learning, as it directly defines the criteria for "good driving behavior." In this invention, immediate reward... It is at each time step An optimal reward function is calculated based on the vehicle's driving performance. The design is as follows:

[0168] ;

[0169] in:

[0170] This is the current vehicle speed. It is the angle between the vehicle's orientation and the direction of the lane. It indicates the effective speed of a vehicle along the lane direction, encouraging vehicles to move forward efficiently and correctly.

[0171] It is the lateral offset distance between the vehicle's centerline and the lane's centerline, encouraging vehicles to stay in the center of the lane.

[0172] It is the change in steering wheel angle between two consecutive moments, used to penalize overly aggressive steering maneuvers and encourage smooth driving.

[0173] It is a huge negative penalty that is triggered when a vehicle collides, goes off the road, or violates traffic rules; otherwise, it is 0.

[0174] These are the weighting coefficients for each item, used to balance different driving objectives.

[0175] 5.3 Network Training and Update Process:

[0176] DDPG is trained by iteratively updating network parameters by randomly sampling mini-batch empirical tuples from the empirical replay pool.

[0177] (1) Update the critic network:

[0178] The goal of updating the critic network is to make its Q-value estimate as close as possible to the "true" long-term return. First, the target Q-value is calculated using the target network. :

[0179] ;

[0180] in, It is a reward discount factor (e.g., 0.99), which indicates the degree of importance attached to future rewards.

[0181] Then, by minimizing the loss function To update the parameters of the critic network The loss function is the mean squared error (MSE) between the predicted Q-value and the target Q-value:

[0182] ;

[0183] It is expected that E will be approximated by mini-batch sampling.

[0184] (2) Update the actor network:

[0185] The goal of updating the actor network is to adjust its policy so that its output actions receive a higher Q-value from the critic network. This is achieved through policy gradient ascent. The formula for calculating the policy gradient is as follows:

[0186] ;

[0187] This gradient guides the actor network parameters. The direction of the update is to maximize the Q value.

[0188] (3) Update the target network:

[0189] The parameters of the target network are not updated directly through backpropagation, but rather through soft updates, slowly "converging" with the parameters of the main network.

[0190] ;

[0191] ;

[0192] in, It is a very small update rate (e.g., 0.001), which ensures the stability of the target network.

[0193] Through the above iterative training process, the actor network eventually learned an intelligent mapping strategy from complex environmental states (latent feature vectors) to precise and smooth continuous driving control commands, thus achieving end-to-end autonomous driving control.

[0194] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0195] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or stages, which are not necessarily completed at the same time, but may be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0196] In one embodiment, a computer system is provided, which may be a server. The computer system includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data used in the methods described above. The network interface communicates with external terminals via a network connection. The computer program is executed by the processor to implement the methods described above.

[0197] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0198] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0199] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. An automatic driving control method based on intelligent perception of a driving environment, characterized by, The method comprises the following steps: S1, acquiring a sequence of RGB images and a corresponding sequence of depth images in front of a vehicle at multiple continuous time points; S2, performing multi-modal feature extraction on the sequence of RGB images and the sequence of depth images, comprising: S21, extracting RGB spatial features and depth spatial features from the RGB images and the depth images respectively by introducing a ResNet50 network with a channel attention mechanism; S22, inputting the sequence of RGB images and the sequence of depth images into a long short-term memory network to extract RGB time sequence features and depth time sequence features; S23, fusing the RGB spatial features, the depth spatial features, the RGB time sequence features and the depth time sequence features by a spatio-temporal context global correlation module to generate a spatio-temporal feature map; S3, generating a bird's eye view semantic feature, comprising: S31, converting the RGB images into a bird's eye view based on inverse perspective transformation; S32, processing the bird's eye view using a semantic segmentation network based on an encoder-decoder structure, and extracting a bird's eye view semantic feature map from the semantic segmentation network; S4, combining the spatio-temporal feature map and the bird's eye view semantic feature map by a multi-modal multi-view decision state generation module, and performing time sequence modeling by a gated recurrent unit to output a latent feature vector; S5, inputting the latent feature vector as a current environment state into a deep deterministic policy gradient network to output control instructions for controlling a steering wheel angle, an accelerator and a brake of the vehicle. 2.The automatic driving control method based on intelligent perception of a driving environment according to claim 1, wherein, A channel attention module is added after each residual block of the ResNet50 network, and a dropout layer is set after each fully connected layer of the ResNet50 network; the channel attention module adopts a squeeze-and-excitation network, and an excitation operation of the squeeze-and-excitation network comprises two fully connected layers. 3.The automatic driving control method based on intelligent perception of a driving environment according to claim 1, wherein, The spatio-temporal feature map is generated by fusing the RGB spatial features, the depth spatial features, the RGB time sequence features and the depth time sequence features by the spatio-temporal context global correlation module, specifically comprising: combining the RGB spatial features, the depth spatial features, the RGB time sequence features and the depth time sequence features in a channel dimension to form an input feature map; performing global correlation modeling on all pixel points of the input feature map by a non-local operation of a non-local neural network; for each pixel point in the input feature map, the correlation degrees between the pixel point and all other pixel points are calculated, and features of all other pixel points are weighted and summed according to the correlation degrees to generate a response capable of representing global context information, and the features of the current pixel point are updated by the response to output the spatio-temporal feature map. 4.The automatic driving control method based on intelligent perception of a driving environment according to claim 3, wherein, The global correlation modeling on all pixel points of the input feature map by the non-local operation of the non-local neural network is implemented by the following unit: a first transformation unit for embedding transformation on feature vectors in the input feature map; a similarity calculation unit for calculating the similarity between the transformed feature vectors at any two positions in the input feature map to obtain the correlation degrees. In response to the aggregation unit, all feature vectors transformed by the first transformation unit are weighted and summed according to the correlation degree, and normalized to generate the response. 5.The automatic driving control method based on intelligent perception of a driving environment according to claim 1, wherein, The semantic segmentation network based on the encoder-decoder structure specifically comprises: The encoder of the semantic segmentation network adopts an EfficientNetB3 network; the decoder of the semantic segmentation network is composed of multiple convolutional blocks containing convolutional layers, batch normalization layers, ReLU activation functions and up-sampling operations; the encoder and the decoder are combined through a skip connection. 6.The automatic driving control method based on intelligent perception of a driving environment according to claim 1, wherein, The multi-mode multi-view decision state generation module combines the spatio-temporal feature map and the bird's eye view semantic feature map, and performs time series modeling through a gated recurrent unit, specifically comprising: At each time step, the spatio-temporal feature map and the bird's eye view semantic feature map are flattened and spliced to form a combined feature vector; The combined feature vector is input as the current time input of the gated recurrent unit, and the gated recurrent unit performs time series processing to output the latent feature vector. 7.The automatic driving control method based on intelligent perception of a driving environment according to claim 1, wherein, The latent feature vector is input as the current environment state to a deep deterministic policy gradient network to output control instructions for controlling the steering wheel angle, throttle and brake of the vehicle, specifically comprising: using the potential feature vector as a current environment state in a deep deterministic policy gradient algorithm ; The deep deterministic policy gradient algorithm adopts a deep deterministic policy gradient network including an actor network and a critic network, the actor network outputs a control instruction including a steering wheel rotation angle, an accelerator and a brake as an action according to the current environment state Output a control instruction including a steering wheel rotation angle, an accelerator and a brake as an action ; The critic network evaluates a corresponding value based on the current environment state and the action ; The deep deterministic policy gradient network is learned by interacting with a driving environment, and is optimized to maximize a predefined cumulative reward, which is an instantaneous reward calculated according to a driving state of the vehicle The driving state includes a driving speed, a lane keeping situation, and whether a collision occurs.

8. A computer system comprising a memory and a processor, said memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 7. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 7.

Images