Environment sensing and path planning method and system based on vehicle controller

Abstract

The invention provides an environment perception and path planning method and system based on a vehicle controller, and the method comprises the steps: processing the data of an environment perception sensor, and obtaining a standardized image and a two-dimensional event counting graph; respectively processing the standardized image and the event counting graph to obtain an image multi-scale feature graph set and an event multi-scale feature graph set; carrying out attention mechanism fusion on the image multi-scale feature image set and the event multi-scale feature image set to obtain a fusion feature pyramid; performing feature enhancement processing according to the fused feature pyramid to obtain a line segment descriptor set; performing time sequence matching on the line segment descriptor set to obtain optimized line segment parameters; performing topology analysis and plane segmentation processing according to the optimized line segment parameters to obtain an environment structure model with a confidence score; and according to an output result of the environment structure model and a preset path planning algorithm, outputting a path indication identifier for assisting vehicle driving.

Description

Technical Field

[0001] The present application relates to the field of vehicle navigation technology, and in particular to an environment perception and path planning method and system based on a vehicle controller. Background Art

[0002] With the increase in the number of vehicles and the acceleration of urbanization, road conditions have become increasingly complex. Drivers are faced with information input from all directions during driving, including but not limited to the behavior of other road users, traffic signals, road signs, and emergencies. In this case, human drivers may not be able to fully notice all the obstacles around them, especially those in the visual blind spot or objects that suddenly appear, which increases the risk of traffic accidents and easily leads to vehicle collisions. Summary of the invention

[0003] The present application provides an environmental perception and path planning method and system based on a vehicle controller, which are used to assist driving, improve driving safety, and reduce traffic accidents.

[0004] In a first aspect, an embodiment of the present application provides an environment perception and path planning method based on a vehicle controller, the method comprising: Acquire RGB images and event stream data through an environmental perception sensor, perform pixel normalization processing on the RGB image to obtain a standardized image, and perform cumulative projection on the event stream data to obtain a two-dimensional event count map; Performing feature extraction on the standardized image to obtain a multi-scale feature atlas of the image, and performing spatiotemporal feature compression on the event count image through a preset 3D convolution layer and a preset 2D convolution layer to obtain a multi-scale feature atlas of the event; Performing an attention mechanism fusion on the image multi-scale feature atlas and the event multi-scale feature atlas to obtain a fused feature pyramid; Perform feature enhancement processing according to the fused feature pyramid to obtain a line segment descriptor set; Performing time sequence matching on the line segment descriptor set to obtain optimized line segment parameters; Performing topological analysis and plane segmentation processing according to the optimized line segment parameters to obtain an environmental structure model with a confidence score; According to the output result of the environmental structure model and the preset path planning algorithm, a path indication mark for assisting vehicle driving is output.

[0005] In a second aspect, an embodiment of the present application provides an environment perception and path planning system based on a vehicle controller, the system comprising: An image processing module is used to obtain an RGB image and event stream data through an environment perception sensor, perform pixel normalization processing on the RGB image to obtain a standardized image, and perform cumulative projection on the event stream data to obtain a two-dimensional event count map; A feature conversion module is used to extract features from the standardized image to obtain a multi-scale feature atlas of the image, and to compress the spatiotemporal features of the event count image through a preset 3D convolution layer and a preset 2D convolution layer to obtain a multi-scale feature atlas of the event; A feature fusion module, used for fusing the image multi-scale feature atlas and the event multi-scale feature atlas by an attention mechanism to obtain a fused feature pyramid; A feature enhancement module, used for performing feature enhancement processing according to the fused feature pyramid to obtain a line segment descriptor set; A timing matching module, used for performing timing matching on the line segment descriptor set to obtain optimized line segment parameters; A model building module, used to perform topological analysis and plane segmentation processing according to the optimized line segment parameters to obtain an environmental structure model with a confidence score; The identification generation module is used to output a path indication identification for assisting vehicle driving according to the output result of the environmental structure model and a preset path planning algorithm.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the vehicle controller-based environment perception and path planning method as described in any one of the embodiments of the present application when executing the computer program.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the vehicle controller-based environment perception and path planning method as described in any one of the embodiments of the present application.

[0008] The embodiment of the present application provides an environment perception and path planning method based on a vehicle controller, characterized in that the method includes: acquiring RGB images and event stream data through an environment perception sensor, performing pixel normalization processing on the RGB images to obtain standardized images, and performing cumulative projection on the event stream data to obtain a two-dimensional event count map; performing feature extraction on the standardized image to obtain an image multi-scale feature atlas, performing spatiotemporal feature compression on the event count map through a preset 3D convolution layer and a preset 2D convolution layer to obtain an event multi-scale feature atlas; performing attention mechanism fusion on the image multi-scale feature atlas and the event multi-scale feature atlas to obtain a fused feature pyramid; performing feature enhancement processing according to the fused feature pyramid to obtain a line segment descriptor set; performing time sequence matching on the line segment descriptor set to obtain optimized line segment parameters; performing topological analysis and plane segmentation processing according to the optimized line segment parameters to obtain an environment structure model with a confidence score; and outputting a path indication mark for assisting vehicle driving according to the output result of the environment structure model and a preset path planning algorithm. Through the above method, a unified expression of data is performed by combining pixel standardization processing of RGB images and cumulative projection processing of event stream data. Feature extraction and spatiotemporal feature compression are performed on the processed data respectively to capture the spatial features of RGB images and the temporal dynamic features of event stream data. When the two are fused by the attention mechanism, different weight coefficients can be given in combination with the scene to obtain a fused feature pyramid. Feature enhancement processing is performed based on the fused feature pyramid to obtain a set of line segment descriptors, which retains the multi-scale line segment information of the scene. By performing temporal matching processing on the line segment descriptor set, optimized line segment parameters covering the line segment correspondence between consecutive frames are established. According to the optimized line segment parameters, an environmental structure model with geometric constraints is constructed, which provides a structured understanding of the scene and can combine map information with the path planning algorithm to output path indication marks. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0010] Figure 1 A schematic flow chart of an environment perception and path planning method based on a vehicle controller provided in an embodiment of the present application; Figure 2 A schematic block diagram of an environment perception and path planning system based on a vehicle controller provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0013] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0014] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0015] The embodiment of the present application provides an environment perception and path planning method based on a vehicle controller. Based on this method, a path prompt mark can be output on a visual component on the vehicle to assist the driver in identifying obstacles in the environment. The path prompt mark includes: arrows, path curves, and collision prompt marks, etc. The arrows and path curves can be changed according to the real-time position of the vehicle.

[0016] See also Figure 1 , Figure 1 FIG. 1 is a schematic flow chart of an environment perception and path planning method based on a vehicle controller provided in an embodiment of the present application. Figure 1 As shown, the specific steps of the vehicle controller-based environment perception and path planning method include: S101-S107.

[0017] S101. Acquire RGB images and event stream data through an environmental perception sensor, perform pixel normalization processing on the RGB image to obtain a standardized image, and perform cumulative projection on the event stream data to obtain a two-dimensional event count map.

[0018] Exemplarily, the environmental perception sensor includes a video camera and an event camera, the video camera is used to obtain RGB images, and the event camera is used to obtain event stream data. The event stream data contains a large number of discrete event points. The standardization of RGB images includes two sub-steps: first, scale normalization, scaling the image to a uniform size; second, pixel value normalization, mapping the pixel value to the [0, 1] interval. The time window mechanism is used to process event stream data, and event points are collected in the time interval [t-△t / 2, t+△t / 2]. The event count map V(t) is generated by two-dimensional plane projection, where each pixel value represents the number of events accumulated at that location.

[0019] S102, extracting features from the standardized image to obtain a multi-scale feature atlas of the image, and compressing the spatiotemporal features of the event count image through a preset 3D convolution layer and a preset 2D convolution layer to obtain a multi-scale feature atlas of the event.

[0020] For example, the processing of the standardized image adopts an improved ResNet backbone network, which includes 4 residual blocks with skip connections, and each residual block consists of multiple convolutional layers. A feature pyramid module is added after each residual block to generate feature maps with different spatial resolutions {F I 1 , F I 2 , F I 3 , F I 4 The processing of the event count map first uses a 3D convolution layer to extract spatiotemporal features to capture the temporal dynamic information of the event data, and compresses the features into a two-dimensional space through a 2D convolution layer to generate a multi-scale feature map {F E 1 , F E 2 , F E 3 , F E 4}, ensuring the same spatial resolution as the image branch features.

[0021] S103, performing an attention mechanism fusion on the image multi-scale feature atlas and the event multi-scale feature atlas to obtain a fused feature pyramid.

[0022] Exemplarily, in the channel dimension, the channel attention weight α is calculated by global average pooling (GAP) and multi-layer perceptron (MLP) I i and α E i , evaluate the importance of different feature channels. In the spatial dimension, a spatial self-attention module is designed to capture long-range dependencies and generate an attention map Ai Through the attention map A i Weighted sum F fused i =α I i ⊙F I i +α E i ⊙F E i +A i ⊙(F I i +F E i ) realizes feature fusion, and ⊙ represents element-by-element multiplication. The fusion process is performed independently at all scale levels to generate a multi-scale fusion feature pyramid.

[0023] S104, performing feature enhancement processing according to the fused feature pyramid to obtain a line segment descriptor set.

[0024] Exemplarily, the fused feature pyramid is enhanced by a double hourglass network structure. The double hourglass network structure includes: a first branch and a second branch. The first branch extracts coarse-grained line segment contours through downsampling and upsampling operations to generate a response map S coarse The second branch maintains high spatial resolution and adds residual connections to extract detailed features and generate a response map S fine The two response maps are fused by adaptive weight λ: S = λS coarse +(1-λ)S fine The non-maximum suppression (NMS) algorithm is used to extract candidate line segments, and the confidence and geometric attributes of each line segment are calculated to form a line segment descriptor set L = {l i}.

[0025] S105 . Perform time sequence matching on the line segment descriptor set to obtain optimized line segment parameters.

[0026] For example, in the line segment descriptor set, the matching relationship between the line segments at adjacent moments is calculated, and a comprehensive score is given based on the geometric similarity and appearance similarity: Score(l i(t) , l j(t-1) ), where i is the adjacent time. The Kalman filter is used to smooth and predict the parameters of the matching line segment, and the temporal consistency loss ||l(t)-l is introduced. pred(t) ||2 constrains the changes of line segments at adjacent moments. This can suppress detection jitter and improve the timing stability of line segment detection.

[0027] S106. Perform topological analysis and plane segmentation processing according to the optimized line segment parameters to obtain an environmental structure model with a confidence score.

[0028] For example, the topological relationship (parallelism, perpendicularity, intersection) between line segments is analyzed to construct a wireframe structure graph G = (V, E). For the wireframe structure graph G, the RANSAC algorithm is used to perform plane segmentation and merging of the wireframe structure, and the structure is registered and fused in combination with the time series information, and a confidence score is assigned to the reconstruction result to reflect the structural reliability. This model can be directly used for subsequent navigation and decision-making tasks, providing reliable environmental perception results for autonomous driving.

[0029] S107: Output a path indication mark for assisting vehicle driving according to the output result of the environment structure model and a preset path planning algorithm.

[0030] For example, the output results of the environmental structure model are combined to construct rich surrounding environment point cloud data, and the optimal driving path is calculated in combination with the preset path planning algorithm (such as A*, RRT*, etc.). Path planning takes into account factors such as road structure, obstacle location, driving rules, and generates a smooth trajectory based on vehicle dynamics constraints. The planning results are converted into intuitive path indication signs to assist the vehicle to complete the autonomous driving task safely and efficiently.

[0031] The embodiment of the present application provides an environment perception and path planning method based on a vehicle controller, characterized in that the method includes: acquiring RGB images and event stream data through an environment perception sensor, performing pixel normalization processing on the RGB images to obtain standardized images, and performing cumulative projection on the event stream data to obtain a two-dimensional event count map; performing feature extraction on the standardized image to obtain an image multi-scale feature atlas, performing spatiotemporal feature compression on the event count map through a preset 3D convolution layer and a preset 2D convolution layer to obtain an event multi-scale feature atlas; performing attention mechanism fusion on the image multi-scale feature atlas and the event multi-scale feature atlas to obtain a fused feature pyramid; performing feature enhancement processing according to the fused feature pyramid to obtain a line segment descriptor set; performing time sequence matching on the line segment descriptor set to obtain optimized line segment parameters; performing topological analysis and plane segmentation processing according to the optimized line segment parameters to obtain an environment structure model with a confidence score; and outputting a path indication mark for assisting vehicle driving according to the output result of the environment structure model and a preset path planning algorithm. Through the above method, a unified expression of data is performed by combining pixel standardization processing of RGB images and cumulative projection processing of event stream data. Feature extraction and spatiotemporal feature compression are performed on the processed data respectively to capture the spatial features of RGB images and the temporal dynamic features of event stream data. When the two are fused by the attention mechanism, different weight coefficients can be given in combination with the scene to obtain a fused feature pyramid. Feature enhancement processing is performed based on the fused feature pyramid to obtain a set of line segment descriptors, which retains the multi-scale line segment information of the scene. By performing temporal matching processing on the line segment descriptor set, optimized line segment parameters covering the line segment correspondence between consecutive frames are established. According to the optimized line segment parameters, an environmental structure model with geometric constraints is constructed, which provides a structured understanding of the scene and can combine map information with the path planning algorithm to output path indication marks.

[0032] In order to more clearly introduce the technical solution of the present application, the technical solution of the present application will be introduced through specific embodiments below. It should be noted that the specific embodiments are used to expand the technical solution of the present application, but are not intended to limit the present application.

[0033] In some embodiments, cumulative projection is performed on the event stream data to obtain a two-dimensional event count graph, including: S1011-S1014.

[0034] S1011. Perform time window division processing on event points in the event stream data according to timestamps to obtain event point subsets within multiple time intervals.

[0035] For example, the time window size △t is set, and the entire event stream sequence is segmented at fixed intervals. For each time window [t-△t / 2, t+△t / 2], all event points e(x, y, t, p) within the time range are collected. In order to deal with the uneven distribution of event points, an exponential decay weighting mechanism is used to assign different weights to event points in the time window. Specifically, for the time window center time t and the event point timestamp t i , the weight is calculated as w i =exp(-|tt i | / σ), where σ is the time attenuation coefficient. In this way, event points closer to the central moment receive higher weights, thereby better preserving the timing information. At the same time, in order to reduce the impact of noise events, the minimum event number threshold n is set min , when the number of events in a time window is less than the threshold, the event points of the window are merged into the adjacent window for processing.

[0036] S1012. Perform spatial grid quantization processing on the event points according to the pixel coordinates and polarity values ​​of each event point in the event point subset to obtain a grid index matrix and a corresponding polarity accumulation value matrix.

[0037] For example, after completing the time window division, it is necessary to perform spatial quantization processing on the event points in each time window. Define the spatial grid size g×g and divide the image plane into regular grids. For each event point e(x, y, t, p), calculate its grid coordinates (i, j): i=x / g, j=y / g. Construct the grid index matrix M idx , where M idx (i, j) records the event point index set contained in the grid (i, j). At the same time, the polarity accumulation matrix M is constructed pol , where M pol (i, j) is the weighted sum of the polarity values ​​(+1 or -1) of all event points in the grid. In order to handle event points at the grid boundary, a soft assignment mechanism is introduced to assign event points close to the grid boundary to adjacent grids at the same time, and the assignment weight is calculated based on the inverse distance. In addition, for each grid, the spatial distribution characteristics of the event points are calculated, including the direction histogram and polarity consistency measure.

[0038] S1013. Up-sample the grid data using a bilinear interpolation algorithm according to the grid index matrix and the polarity accumulation value matrix to obtain a high-resolution feature map having the same resolution as the corresponding RGB image.

[0039] Exemplarily, for each pixel position (x', y') at the target resolution, determine its four nearest neighbor grid points (i1, j1), (i1, j2), (i2, j1), (i2, j2) in the grid feature map. Calculate the distance weights w11, w12, w21, w22 from the pixel to these four grid points, satisfying ∑w=1. In addition to considering the spatial distance, the weight calculation also introduces the event density and polarity consistency within the grid as modulation factors. Specifically, the weight w=d weight *e weight *p weight , where d weight is the distance weight, e weight is the event density weight, p weight is the polarity consistency weight. Through this adaptive weighting scheme, the clarity of edge and motion structures can be better maintained.

[0040] S1014. According to the high-resolution feature map, the feature values ​​are normalized by an adaptive threshold segmentation algorithm to obtain a two-dimensional event count map, wherein each pixel value of the two-dimensional event count map represents the event intensity at the corresponding position.

[0041] Exemplarily, a local adaptive threshold algorithm is used to calculate the threshold of each area. Specifically, in a w×w window centered on each pixel, the weighted average μ and standard deviation σ are calculated, and the threshold T=μ+k*σ, where k is an adjustable parameter. For pixels whose eigenvalues ​​are higher than the threshold, nonlinear mapping is performed through the sigmoid function to obtain a normalized event intensity value. In order to enhance the contrast, local histogram equalization processing is introduced. At the same time, in order to suppress isolated noise points, the feature map is morphologically processed, including opening and closing operations, and the output two-dimensional event count map not only maintains the spatial distribution characteristics of the event, but also enhances the local contrast through adaptive normalization.

[0042] In some embodiments, the preset 3D convolution layer includes: a 3×3×3 three-dimensional convolution kernel, the preset 2D convolution layer includes a 2×2 maximum pooling layer and a 1×1 convolution layer, and the event count graph is compressed in time and space through the preset 3D convolution layer and the preset 2D convolution layer to obtain a multi-scale feature atlas of events, including: S1021-S1026.

[0043] S1021. Perform time series stacking processing on the two-dimensional event count graph through a sliding time window to obtain a T-frame event feature volume, where T is a preset time window length.

[0044] Exemplarily, a sliding time window of fixed size W is used to sample the two-dimensional event count map, and the window step size is W / 2, ensuring that adjacent windows have a 50% overlap. In each time window, the event count map is arranged and stacked in chronological order to form a three-dimensional feature volume of size H×W×T, where H and W are the spatial height and width, respectively, and T is the preset time window length. In order to maintain the continuity of the timing information, the linear interpolation method is used to smooth the transition of the event count map at the window boundary. At the same time, the event count map of each frame is normalized so that its value is distributed in the range of [0, 1] to avoid feature instability caused by excessive differences in the number of events in different time windows.

[0045] S1022. Extract spatiotemporal features of the event feature volume through a 3×3×3 three-dimensional convolution kernel to obtain an initial feature tensor. The number of channels of the initial feature tensor is C.

[0046] Exemplarily, in the embodiment of the present application, a 3×3×3 three-dimensional convolution kernel is used to process the event feature volume. The convolution kernel slides simultaneously along the time dimension and the space dimension, and the stride is set to (1, 2, 2). Each convolution kernel contains 27 learnable parameters for capturing feature patterns in local spatiotemporal regions. By setting padding='SAME', the spatial size of the feature map is kept unchanged. In order to enhance the feature extraction capability, multiple sets of parallel 3D convolution kernels are used to output feature maps of C channels. Each channel captures different feature patterns, such as edges, corners, motion, etc., to obtain an initial feature tensor. A BatchNormalization layer and a ReLU activation function are added after the convolution operation to accelerate training convergence and introduce nonlinear characteristics.

[0047] S1023. Perform multi-scale feature decomposition processing on the initial feature tensor through a preset spatial pyramid pooling network to obtain four sets of feature atlases with different spatial resolutions.

[0048] Exemplarily, the preset spatial pyramid pooling network in the embodiment of the present application adopts a four-level pyramid structure, and the pooling ratios are 1×1, 2×2, 3×3 and 6×6 respectively. The initial feature tensor is adaptively averaged and pooled at each pyramid level to generate feature maps of different spatial resolutions. In order to maintain the discriminability of the features, a 1×1 convolution layer is added after each pyramid level to perform channel dimensionality reduction to make the feature dimensions at all levels consistent. The feature maps of different levels are then adjusted to a uniform spatial size through bilinear interpolation and spliced ​​in the channel dimension to form a multi-scale feature representation, and then the features of different scales are adaptively weighted through the attention mechanism to obtain four sets of feature maps with different spatial resolutions.

[0049] S1024. Perform feature dimension reduction on each group of feature maps in the feature map set through a 2×2 maximum pooling layer and a 1×1 convolution layer to obtain a compressed feature map, where the number of channels of the compressed feature map is C / 2.

[0050] Exemplarily, the feature map is downsampled through a 2×2 maximum pooling layer, effectively reducing the spatial dimension while retaining significant features. Then, a 1×1 point-by-point convolution is used to reorganize and reduce the dimension of the feature channels, reducing the number of channels from C to C / 2. In order to reduce information loss, a channel attention mechanism is introduced before the point-by-point convolution to calculate the importance weight of each channel. Specifically, the channel descriptor is obtained through global average pooling, and the channel weight is generated through a two-layer fully connected network to weight the feature map. The feature distribution is stabilized by a batch normalization layer, and PReLU is used as the activation function to provide nonlinear capabilities.

[0051] S1025. Perform feature enhancement on the compressed feature map through a residual connection module to obtain an enhanced feature map, wherein the residual connection module includes two 3×3 convolutional layers and a skip connection path.

[0052] Exemplarily, the residual connection module uses two 3×3 convolutional layers in series for feature transformation. The first convolutional layer uses stride=2 for downsampling, and the second convolutional layer keeps the spatial resolution unchanged. On the main path, each convolutional layer is equipped with BatchNormalization and ReLU activation functions. The skip connection path uses 1×1 convolution for dimension matching and uses the same stride to ensure consistent spatial size to obtain an enhanced feature map. The residual learning structure is formed by adding the features of the main path and the skip path.

[0053] S1026. According to the enhanced feature map, feature upsampling is performed through a deconvolution layer, and a cross-layer connection is performed with a compressed feature map of a corresponding scale to obtain a multi-scale feature atlas of the event.

[0054] Exemplarily, in the embodiment of the present application, the deconvolution layer uses a 4×4 learnable convolution kernel, and the stride is set to 2 to achieve 2 times feature map upsampling. In order to reduce the checkerboard effect in the upsampling process, nearest neighbor interpolation is performed before deconvolution. The upsampled feature map is spliced ​​with the compressed feature map of the corresponding scale in the channel dimension to construct a cross-layer feature connection. In order to balance the contribution of features in different layers, a feature selection gating mechanism is introduced to dynamically adjust the feature fusion ratio through learnable weight parameters. The number of channels is adjusted through 1×1 convolution to generate a multi-scale feature atlas, which realizes the effective fusion of features with different spatial resolutions.

[0055] In some embodiments, an attention mechanism is performed on the image multi-scale feature atlas and the event multi-scale feature atlas to obtain a fused feature pyramid, including: S1031-S1037.

[0056] S1031, performing feature statistics on the first feature map at each scale level in the image multi-scale feature map set to obtain an image feature channel weight vector, and performing feature statistics on the second feature map at the corresponding scale level in the event multi-scale feature map set to obtain an event feature channel weight vector.

[0057] Exemplarily, for the first feature map F at each scale level I i Perform a global average pooling operation to compress the C×H×W feature map into a C×1×1 vector v I Then, a two-layer fully connected network is used for nonlinear transformation: the first layer reduces the number of channels from C to C / r (r is the compression ratio, usually 16), using the ReLU activation function; the second layer restores the number of channels from C / r to C, and normalizes it with the sigmoid function to obtain the channel weight vector α I For event feature F E i Using the same calculation process, we get the event feature channel weight vector α E This channel attention mechanism can adaptively adjust the importance of different feature channels and highlight channels with large amounts of information.

[0058] S1032. Recalibrate the weights of the first feature map and the second feature map according to the image feature channel weight vector and the event feature channel weight vector to obtain a first weighted feature map and a second weighted feature map.

[0059] Exemplarily, the weight recalibration process uses a channel-level attention mechanism to convert the weight vector α obtained in S1031 I and α E Respectively with the original feature map F I i and F E i Perform channel-by-channel multiplication. Specifically, α I and α E They are expanded to the same spatial dimensions as the feature map (C×H×W), and then element-wise multiplication is performed: F I i-weighted =α I ⊗F I i , F E i-weighted =α E ⊗F E i , where ⊗ represents a broadcast multiplication operation, F I i-weighted is the first weighted feature map, F E i-weightedis the second weighted feature map.

[0060] S1033: perform feature concatenation on the first weighted feature map and the second weighted feature map to obtain a multimodal feature map.

[0061] Exemplarily, the first weighted feature map F I i-weighted and the second weighted feature map F E i-weighted Connect in channel dimension: F concat =[F I i-weighted ||F E i-weighted ], and a multimodal feature map with 2C channels is obtained. In order to maintain the spatial alignment of the features, the two feature maps are ensured to have the same spatial resolution before splicing. In addition, a 1×1 convolutional layer is introduced after splicing for feature adjustment to learn the complementary relationship between the two modal features and enhance the feature expression ability.

[0062] S1034. Perform feature mapping on the multimodal feature map to obtain a query matrix, a key matrix and a value matrix. Perform attention map calculation and processing based on the query matrix and the key matrix through matrix multiplication and softmax normalization operations to obtain a spatial attention weight matrix.

[0063] For example, the feature mapping process uses three independent 1×1 convolutional layers to transform the multimodal feature map F concat They are mapped to query matrix Q, key matrix K and value matrix V respectively, and their dimensions are all C'×(H×W). Then the attention score is calculated: , where d is the square root of the feature dimension, which is used to scale the dot product result. This self-attention mechanism can capture long-range dependencies in the feature map and establish global context associations at the pixel level. At the same time, a multi-head attention mechanism is introduced to divide the features into h heads, each head calculates attention independently, and then merges the results.

[0064] S1035. Weight the value matrix according to the spatial attention weight matrix to obtain an attention feature map, and enhance the features of the attention feature map through a residual connection module to obtain a spatial enhanced feature map.

[0065] Exemplarily, the value matrix V is weighted according to the attention weight matrix A: F att =A·V, and get the attention feature map. Then the feature enhancement is performed through the residual connection module: F res =F att +F concatThe residual connection module consists of two 3×3 convolutional layers, with BatchNorm and ReLU activation functions in between. The first convolutional layer reduces the number of channels to C' / 2, and the second layer restores it to the original number of channels C'.

[0066] S1036. Perform feature selection and channel compression on the spatial enhancement feature map and the multimodal feature map to obtain a compressed feature map.

[0067] Exemplarily, the SE (Squeeze-and-Excitation) module is used to evaluate the channel importance of the spatial enhancement feature map and generate a channel weight vector β. Then, the top k channels in importance are selected according to β, where k is determined by an adaptive threshold. For the selected channels, a 1×1 convolutional layer is used to perform channel compression, reducing the number of channels from C' to C" / 2. At the same time, in order to maintain information integrity, a channel shuffle operation is used during the compression process to disperse highly correlated channels into different groups to avoid information redundancy.

[0068] S1037. Determine a fused feature pyramid according to the compressed feature map and a preset feature pyramid network.

[0069] Exemplarily, the compressed feature map is gradually downsampled through a maximum pooling layer with a step size of 2 to generate feature maps of different scales {P1, P2, P3, P4}. Starting from the highest layer, features are fused layer by layer through 1×1 convolution and upsampling operations, and each layer contains a lateral connection and feature recalibration module. The lateral connection adopts a weighted summation method, and the weight is dynamically adjusted through learnable parameters. According to the multi-scale fusion feature pyramid obtained by the above process, the feature map of each scale contains semantic information and detail information at different levels.

[0070] In some embodiments, feature enhancement is performed based on the fused feature pyramid to obtain a line segment descriptor set, including: S1041-S1049.

[0071] S1041. Perform feature denoising and downsampling encoding on the fused feature pyramid to obtain an encoded feature map.

[0072] Exemplarily, when performing feature denoising and downsampling encoding on the fused feature pyramid, a 3×3 Gaussian filter is used to suppress noise on each layer of the feature map, and the standard deviation σ of the filter is adaptively adjusted according to the scale of the feature map. The filtered features are then encoded using a residual coding unit, and each coding unit contains two parallel branches: the main branch consists of two 3×3 convolutional layers and a BatchNorm layer, and the secondary branch is a 1×1 convolutional layer. The outputs of the two branches are fused by addition, and the encoded features are obtained using the PReLU activation function. A 2×2 maximum pooling operation is performed on the encoded features to achieve feature downsampling, generating a set of encoded feature maps with different spatial resolutions.

[0073] S1042. Based on the encoded feature map, feature downsampling is performed through the coarse branch of the preset dual-path hourglass network to obtain four sets of coarse-grained feature atlases with different resolutions, and residual connection is performed through the fine branch of the dual-path hourglass network to obtain four sets of fine-grained feature atlases.

[0074] Exemplarily, the coarse branch of the dual-path hourglass network adopts an encoder-decoder structure. The encoding stage contains 4 downsampling blocks, each of which consists of a 3×3 convolutional layer, an instance normalization layer, and a LeakyReLU activation function, which gradually reduces the feature map size to 1 / 16 of the original. The decoding stage gradually upsamples through the deconvolution layer and establishes a jump connection with the encoded features at each scale. The fine branch maintains a high spatial resolution and extracts features through multiple residual blocks. Each residual block contains two 3×3 convolutional layers with dilated convolutions, and the dilation rates are 1, 2, 4, and 8, respectively, to achieve feature extraction of different receptive fields.

[0075] S1043. Perform channel weighting on the coarse-grained feature atlas and the fine-grained feature atlas to obtain a weighted feature atlas, perform feature mapping on the weighted feature atlas, and obtain a line segment response map.

[0076] Exemplarily, the channel attention weights are calculated through the Squeeze-and-Excitation module (SE module) to weight the coarse and fine branch feature maps. The SE module first performs global average pooling on the feature map to obtain the channel descriptor, and then learns the dependencies between channels through two fully connected layers to generate a channel weight vector. The weight vector is multiplied by the original feature map to obtain the weighted feature. Then, a 1×1 convolutional layer is used to map the weighted features to a unified channel dimension, and feature transformation is performed through depthwise separable convolution. The sigmoid function is used to normalize the feature values ​​to the [0, 1] interval to obtain the segment response map.

[0077] S1044, extracting the local maximum value of the line segment response graph to obtain a candidate point set, and performing dynamic threshold screening on the candidate point set to obtain a key point set.

[0078] Exemplarily, an adaptive local maximum detection operator is designed to scan the response map in an n×n window. If the central pixel value is greater than the other pixel values ​​in the window and exceeds the preset threshold τ, it is marked as a candidate point. The threshold τ is dynamically determined by the OTSU algorithm based on the histogram distribution of the response map. The second-order directional derivative of the response value is calculated for each candidate point, and the direction of the line segment is determined based on the main direction of the derivative value. If the derivative value meets the predetermined anisotropy condition, the point is retained as a key point.

[0079] S1045. Perform neighborhood expansion on each key point in the key point set to obtain a line segment candidate region set A, perform direction calculation on the line segment candidate region set to obtain a line segment direction vector set.

[0080] Exemplarily, each key point is taken as the center and region growing is performed along its main direction. During the growing process, the similarity and direction consistency of the response values ​​of the neighborhood pixels and the center point are calculated. If the similarity is greater than the threshold σ and the direction difference is less than the threshold θ, the pixel is included in the candidate region. Principal component analysis (PCA) is applied to each candidate region, and the direction vector of the first principal component is the initial direction estimate of the line segment. At the same time, the ratio of eigenvalues ​​is calculated to evaluate the reliability of the direction estimate.

[0081] S1046. According to the line segment candidate region set and the line segment direction vector set, estimate the line parameters of each candidate region using the RANSAC line fitting algorithm to obtain a line segment parameter set.

[0082] Exemplarily, the pixels in each candidate area are sampled, and the RANSAC algorithm is used to iteratively estimate the line parameters. In each iteration, two points are randomly selected to determine the line hypothesis, and the distances from other points to the line are calculated. If the number of inliers exceeds the threshold and the ratio of inliers is greater than the preset value, the line hypothesis is accepted. The qualified line hypothesis is optimized by least squares to obtain accurate line parameters, including slope and intercept.

[0083] S1047. Perform endpoint positioning on each line segment of the line segment parameter set according to the line segment endpoint detection algorithm to obtain an endpoint coordinate set, and perform validity verification on the endpoint coordinate set to obtain a valid line segment set.

[0084] For example, based on the line parameters, an endpoint detection operator is designed. The response graph is scanned in one dimension along the line direction to detect the significant change points of the response value. The change points are screened using an adaptive threshold to determine the potential endpoint positions. For each endpoint candidate position, local optimization is performed in the direction perpendicular to the line to accurately locate the endpoint coordinates. By calculating the geometric constraints and responses between the endpoint pairs Figure 1 Verify the validity of the consistency.

[0085] S1048. Perform feature encoding according to the valid line segment set to obtain a line segment descriptor vector set V.

[0086] For example, a multi-scale feature descriptor is constructed for each valid line segment. N equally spaced sampling points are taken at both ends of the line segment, and local feature vectors are extracted at each sampling point, including gradient direction histogram, color moment, and local binary pattern. The attention mechanism is used to perform weighted fusion of features of different scales to generate a descriptor vector of fixed dimension. The descriptor is rotationally invariant and scale invariant.

[0087] S1049. Merge overlapping line segments of the line segment descriptor vector set to obtain a line segment descriptor set.

[0088] For example, if the overlap rate of two line segments exceeds the threshold α and the cosine similarity of the descriptors is greater than the threshold β, they are considered to correspond to the same physical line segment. The overlapping line segments are merged using the non-maximum suppression algorithm, and the line segment with the largest response value is retained. The merged line segment descriptor is updated by weighted averaging, and the weight is proportional to the line segment response value.

[0089] In some embodiments, time matching is performed on the line segment descriptor set to obtain optimized line segment parameters, including: S1051-S1053.

[0090] S1051. Based on the line segment descriptor set, obtain a first line segment set at a first moment, and obtain a second line segment set at a second moment.

[0091] Exemplarily, based on the line segment descriptor set L, the line segment sets L(t-1) and L(t) at the first time t-1 and the second time t are extracted respectively. For each line segment set, each line segment descriptor contains the starting point coordinates (x start ,y start )、end point coordinates (x end ,y end ), line segment direction angle θ, line segment length len and line segment confidence confidence. At the same time, in order to ensure the stability of subsequent matching, the local feature descriptor is also extracted for each line segment, including the image block feature vector f centered at the midpoint of the line segment appearance And the gradient histogram feature h along the line segment direction gradient In addition, in order to handle the scale changes of line segments, a multi-scale representation is established for each line segment, and the corresponding line segment parameters and feature descriptions are retained at different feature map scales.

[0092] S1052: Calculate geometric similarity and appearance similarity based on the first line segment set and the second line segment set to obtain a line segment matching score.

[0093] Exemplarily, for each line segment li in the first line segment set L(t-1) and each line segment l in the second line segment set L(t) j , calculate the geometric similarity score between them geo and appearance similarity score app The geometric similarity is calculated by the following indicators: the midpoint distance dcenter of the line segment, the direction angle difference △θ, and the length ratio ratio len , specifically score geo =exp(-w1·dcenter -w2·△θ-w3·|1-ratio len |), where w1, w2, and w3 are weight coefficients. The appearance similarity is calculated by calculating the cosine distance score of the feature vector app =cos(f appearance-i , f appearance-j ) and the Bhattacharyya distance score of the gradient histogram hist =bhattacharyya(h gradient-i ,h gradient-j ) is obtained by weighted combination of . The calculation formula of matching score is: Score = α score geo +β·score app , where α and β are adaptive weights.

[0094] S1053: Perform parameter smoothing processing on the line segment group whose line segment matching score is higher than a preset score threshold to obtain optimized line segment parameters.

[0095] For example, a score threshold threshold is set, and line segment pairs with a matching score greater than the threshold are selected as valid matching pairs. i , l j ), and use the Kalman filter for parameter smoothing. The state vector of the Kalman filter contains the endpoint coordinates, direction angle, and length of the line segment, and the observation vector is the line segment parameter at the current moment. Through the prediction step and the update step, the optimal estimated line segment parameter l is obtained. opt At the same time, in order to maintain the consistency of timing, the change of parameters before and after optimization is calculated as △l=||l opt -l original ||, when △l exceeds the preset threshold, the soft constraint method is used to adjust the parameters to ensure smooth parameter changes. For the line segments that fail to find a match, their original parameters are retained and their confidence scores are reduced.

[0096] In some embodiments, topological analysis and plane segmentation processing are performed based on the optimized line segment parameters to obtain an environmental structure model with a confidence score, including: S1061-S1066.

[0097] S1061. Group the optimized line segment parameters by line segment direction to obtain three main direction line segment sets, perform parallelism and perpendicularity constraint calculations on the line segments in the three main direction line segment sets, and obtain a geometric constraint matrix.

[0098] For example, the line segment directions are clustered by the gradient direction histogram, and the K-means algorithm is used to cluster the line segment directions into three main directions. For each main direction, the angle θ between the line segment and the direction is calculated. When |θ|<θthreshold When , the line segment is classified into the direction set. For each pair of line segments, the inner product cosθ of their direction vectors and the distance d are calculated to construct the parallelism metric P score =exp(-|cosθ-1| / σ p ) and the verticality measure V score =exp(-|cosθ| / σ v ), where σ p and σ v is the smoothing factor. The constraint relationships of all line segment pairs are organized into a geometric constraint matrix M. The matrix elements M ij Represents the constraint strength between segments i and j.

[0099] S1062. Perform a topological connection analysis on the line segment intersections according to the geometric constraint matrix to obtain a line segment intersection network graph, divide the line segment intersection network graph into connected domains, and obtain a set of local structure subgraphs.

[0100] Exemplarily, the strong constraint relationships in the geometric constraint matrix M are traversed (M ij >threshold), extract the intersection points of the line segment pairs that meet the constraints. For each intersection point, calculate its position uncertainty covariance matrix Σ, and propagate the positioning error based on the line segment endpoints. Construct a weighted undirected graph G=(V, E), where the vertex set V is the set of intersection points, the edge set E represents the connection relationship between the intersection points, and the edge weight w ij Based on the Euclidean distance between intersection points and the strength of geometric constraints, the spectral clustering algorithm is used to segment the graph G, and the optimal number of segmentations k is determined through eigenvalue analysis to obtain k local structure subgraphs.

[0101] S1063. Calculate the edge weight of each subgraph in the local structure subgraph set by using the minimum spanning tree algorithm to obtain an edge weight matrix, perform loop detection on the local structure subgraph according to the edge weight matrix, and obtain a set of plane candidate regions.

[0102] For example, for each local structure subgraph, a complete graph G'=(V', E') is constructed, and the edge weight is defined as w' ij =d ij *(1-cos(θ ij )), where d ij is the Euclidean distance between vertices i and j, θ ij is the angle between the corresponding line segments. Apply the Kruskal algorithm to solve the minimum spanning tree T and record the edge weight matrix W. Design a loop detection algorithm: Starting from each edge e in T, search for the shortest path P that does not contain e in the original graph G'. If the total weight of P is less than the threshold τ, then e and P constitute a plane candidate region. Merge and filter the detected loops to remove candidate regions with high overlap.

[0103] S1064. According to the plane candidate region set, plane parameters are fitted for the line segments in each region by using the RANSAC algorithm to obtain a plane equation parameter set, and normal vector consistency check is performed on adjacent planes in the plane equation parameter set to obtain a plane fusion mark.

[0104] For example, for each candidate plane region, all line segment endpoints in the region are collected as the initial point set S for plane fitting. The RANSAC algorithm is used to iteratively perform: randomly sample 3 non-collinear points, fit the plane equation ax+by+cz+d=0, calculate the distance from all points to the plane, and count the number of inliers. The plane parameter with the largest number of inliers is selected as the plane equation of the region. i and p j , calculate its normal vector n i and n j The angle α and the plane distance d, when α<α threshold And d <d threshold When , the pair of marked planes can be fused to generate the plane fusion marking matrix F.

[0105] S1065. Optimize the parameters of adjacent planes with plane fusion marks by using the least square method to obtain an optimized plane parameter set, and perform boundary correction on the plane area according to the optimized plane parameter set to obtain a corrected plane model.

[0106] Exemplarily, for a plane pair with a fusion marker (p i , p j ), merge its support point set S i and S j , construct the least squares optimization problem: min∑(ax k +by k +cz k +d)², a²+b²+c²=1. The Lagrange multiplier method is used to solve the constrained optimization problem and obtain the optimized plane parameters (a*, b*, c*, d*). According to the optimized plane parameters, the plane boundary points are recalculated and the plane contour is extracted using the convex hull algorithm. At the same time, the projection constraints of the line segment endpoints are considered to ensure that the corrected boundary is consistent with the original line segment.

[0107] S1066, the corrected plane model is multi-viewed Figure 1 The consistency test is used to calculate the confidence and obtain the plane confidence score. The modified plane model is fused in time series according to the plane confidence score to obtain the environmental structure model with confidence score.

[0108] Exemplary, multi-view design Figure 1 Consistency scoring function: score=w1C texture +w2Cgeometry +w3*C temporal , where C texture Measuring the texture consistency of a planar area, C geometry Evaluate the regularity of the plane geometry, C temporal Calculate the time series projection error. Specifically, C texture It is evaluated by calculating the variance of the planar projected texture, C geometry Based on the orthogonality and closure measures of the plane boundary, C temporal The change rate of the plane parameters of the previous and next frames is used to measure the time series tracking and parameter fusion of the plane with high confidence (score>threshold) and update the environment structure model.

[0109] See also Figure 2 , Figure 2 1 is a schematic block diagram of an environment perception and path planning system based on a vehicle controller provided in an embodiment of the present application, wherein the environment perception and path planning system based on a vehicle controller 200 is used to execute the aforementioned environment perception and path planning method based on a vehicle controller. The environment perception and path planning system based on a vehicle controller 200 can be configured in a server.

[0110] Among them, the server can be an independent server or a server cluster, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN) and big data and artificial intelligence platforms.

[0111] like Figure 2 As shown, the vehicle controller-based environment perception and path planning system 200 includes: an image processing module 201, a feature conversion module 202, a feature fusion module 203, a feature enhancement module 204, a timing matching module 205, a model building module 206 and an identification generation module 207.

[0112] The image processing module 201 is used to obtain RGB images and event stream data through an environmental perception sensor, perform pixel normalization processing on the RGB image to obtain a standardized image, and perform cumulative projection on the event stream data to obtain a two-dimensional event counting map.

[0113] The feature conversion module 202 is used to extract features from the standardized image to obtain a multi-scale feature atlas of the image, and to compress the spatiotemporal features of the event count image through a preset 3D convolution layer and a preset 2D convolution layer to obtain a multi-scale feature atlas of the event.

[0114] The feature fusion module 203 is used to fuse the image multi-scale feature atlas and the event multi-scale feature atlas using an attention mechanism to obtain a fused feature pyramid.

[0115] The feature enhancement module 204 is used to perform feature enhancement processing according to the fused feature pyramid to obtain a line segment descriptor set.

[0116] The timing matching module 205 is used to perform timing matching on the line segment descriptor set to obtain optimized line segment parameters.

[0117] The model building module 206 is used to perform topological analysis and plane segmentation processing according to the optimized line segment parameters to obtain an environmental structure model with a confidence score.

[0118] The identification generating module 207 is used to output a path indication identification for assisting vehicle driving according to the output result of the environment structure model and a preset path planning algorithm.

[0119] An embodiment of the present application provides an electronic device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement an environmental perception and path planning method based on a vehicle controller as any one of the embodiments of the present application when executing the computer program.

[0120] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements an environmental perception and path planning method based on a vehicle controller as any one of the embodiments of the present application.

[0121] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for environmental perception and path planning based on a vehicle controller, characterized in that: The method comprises: Acquire RGB images and event stream data through an environmental perception sensor, perform pixel normalization processing on the RGB image to obtain a standardized image, and perform cumulative projection on the event stream data to obtain a two-dimensional event count map; Performing feature extraction on the standardized image to obtain a multi-scale feature atlas of the image, and performing spatiotemporal feature compression on the event count image through a preset 3D convolution layer and a preset 2D convolution layer to obtain a multi-scale feature atlas of the event; Performing an attention mechanism fusion on the image multi-scale feature atlas and the event multi-scale feature atlas to obtain a fused feature pyramid; Perform feature enhancement processing according to the fused feature pyramid to obtain a line segment descriptor set; Performing time sequence matching on the line segment descriptor set to obtain optimized line segment parameters; Performing topological analysis and plane segmentation processing according to the optimized line segment parameters to obtain an environmental structure model with a confidence score; According to the output result of the environmental structure model and the preset path planning algorithm, a path indication mark for assisting vehicle driving is output.

2. The vehicle controller-based environment perception and path planning method according to claim 1, characterized in that: The cumulative projection of the event stream data to obtain a two-dimensional event count graph includes: Performing time window division processing on the event points in the event stream data according to the timestamps to obtain event point subsets within multiple time intervals; According to the pixel coordinates and polarity values ​​of each event point in the event point subset, spatial grid quantization processing is performed on the event point to obtain a grid index matrix and a corresponding polarity accumulation value matrix; According to the grid index matrix and the polarity accumulation value matrix, upsampling the grid data by a bilinear interpolation algorithm to obtain a high-resolution feature map having the same resolution as the corresponding RGB image; According to the high-resolution feature map, the feature values ​​are normalized by an adaptive threshold segmentation algorithm to obtain a two-dimensional event count map, wherein each pixel value of the two-dimensional event count map represents the event intensity at the corresponding position.

3. The vehicle controller-based environment perception and path planning method according to claim 2, characterized in that: The preset 3D convolution layer includes: a 3×3×3 three-dimensional convolution kernel, the preset 2D convolution layer includes a 2×2 maximum pooling layer and a 1×1 convolution layer, and the event count graph is compressed in time and space by the preset 3D convolution layer and the preset 2D convolution layer to obtain a multi-scale feature atlas of events, including: Performing time series stacking processing on the two-dimensional event count graph through a sliding time window to obtain a T-frame event feature volume, where T is a preset time window length; Extracting spatiotemporal features of the event feature volume through the 3×3×3 three-dimensional convolution kernel to obtain an initial feature tensor, wherein the number of channels of the initial feature tensor is C; The initial feature tensor is subjected to multi-scale feature decomposition processing through a preset spatial pyramid pooling network to obtain four sets of feature atlases with different spatial resolutions; Performing feature dimension reduction on each group of feature maps in the feature map set through the 2×2 maximum pooling layer and the 1×1 convolution layer to obtain a compressed feature map, wherein the number of channels of the compressed feature map is C / 2; Performing feature enhancement on the compressed feature map through a residual connection module to obtain an enhanced feature map, wherein the residual connection module includes two 3×3 convolutional layers and a skip connection path; According to the enhanced feature map, feature upsampling processing is performed through a deconvolution layer, and a cross-layer connection is performed with a compressed feature map of a corresponding scale to obtain a multi-scale feature atlas of the event.

4. The vehicle controller-based environment perception and path planning method according to claim 1, characterized in that: The performing attention mechanism fusion on the image multi-scale feature atlas and the event multi-scale feature atlas to obtain a fused feature pyramid includes: Performing feature statistics on the first feature map at each scale level in the image multi-scale feature map set to obtain an image feature channel weight vector, and performing feature statistics on the second feature map at the corresponding scale level in the event multi-scale feature map set to obtain an event feature channel weight vector; Recalibrating the first feature map and the second feature map according to the image feature channel weight vector and the event feature channel weight vector to obtain a first weighted feature map and a second weighted feature map; Performing feature concatenation on the first weighted feature map and the second weighted feature map to obtain a multimodal feature map; Performing feature mapping on the multimodal feature map to obtain a query matrix, a key matrix and a value matrix, and performing attention map calculation processing through matrix multiplication and softmax normalization operation according to the query matrix and the key matrix to obtain a spatial attention weight matrix; Weighting the value matrix according to the spatial attention weight matrix to obtain an attention feature map, and performing feature enhancement on the attention feature map through a residual connection module to obtain a spatial enhanced feature map; Performing feature selection and channel compression on the spatial enhancement feature map and the multimodal feature map to obtain a compressed feature map; A fused feature pyramid is determined according to the compressed feature map and a preset feature pyramid network.

5. The vehicle controller-based environment perception and path planning method according to claim 1, characterized in that: The step of performing feature enhancement according to the fused feature pyramid to obtain a line segment descriptor set includes: Performing feature denoising and downsampling encoding on the fused feature pyramid to obtain an encoded feature map; Based on the encoded feature map, feature downsampling is performed through the coarse branch of the preset dual-path hourglass network to obtain four sets of coarse-grained feature atlases with different resolutions, and residual connection is performed through the fine branch of the dual-path hourglass network to obtain four sets of fine-grained feature atlases; Performing channel weighting on the coarse-grained feature atlas and the fine-grained feature atlas to obtain a weighted feature atlas, and performing feature mapping on the weighted feature atlas to obtain a line segment response map; Performing local maximum extraction on the line segment response graph to obtain a candidate point set, and performing dynamic threshold screening on the candidate point set to obtain a key point set; Performing neighborhood expansion on each key point of the key point set to obtain a line segment candidate region set A, and performing direction calculation on the line segment candidate region set to obtain a line segment direction vector set; According to the line segment candidate region set and the line segment direction vector set, a line parameter estimation is performed on each candidate region by using a RANSAC line fitting algorithm to obtain a line segment parameter set; Performing endpoint positioning on each line segment of the line segment parameter set according to a line segment endpoint detection algorithm to obtain an endpoint coordinate set, and performing validity verification on the endpoint coordinate set to obtain a valid line segment set; Perform feature encoding according to the valid line segment set to obtain a line segment descriptor vector set V; Overlapping line segments are merged on the line segment descriptor vector set to obtain the line segment descriptor set.

6. The vehicle controller-based environment perception and path planning method according to claim 1, characterized in that: The performing time sequence matching on the line segment descriptor set to obtain optimized line segment parameters includes: Based on the line segment descriptor set, obtaining a first line segment set at a first moment, and obtaining a second line segment set at a second moment; Calculating geometric similarity and appearance similarity based on the first line segment set and the second line segment set to obtain a line segment matching score; The line segment groups whose line segment matching scores are higher than a preset score threshold are subjected to parameter smoothing processing to obtain optimized line segment parameters.

7. The vehicle controller-based environment perception and path planning method according to claim 1, characterized in that: The topological analysis and plane segmentation processing are performed according to the optimized line segment parameters to obtain an environmental structure model with a confidence score, including: The optimized line segment parameters are grouped by line segment direction to obtain three main direction line segment sets, and the line segments in the three main direction line segment sets are calculated for parallelism and perpendicularity constraints to obtain a geometric constraint matrix; Performing topological connection analysis on line segment intersections according to the geometric constraint matrix to obtain a line segment intersection network graph, and performing connected domain division on the line segment intersection network graph to obtain a local structure subgraph set; Calculating edge weights for each subgraph in the local structure subgraph set by using a minimum spanning tree algorithm to obtain an edge weight matrix, and performing loop detection on the local structure subgraph according to the edge weight matrix to obtain a set of plane candidate regions; According to the plane candidate region set, plane parameters are fitted for line segments in each region by using the RANSAC algorithm to obtain a plane equation parameter set, and normal vector consistency check is performed on adjacent planes in the plane equation parameter set to obtain a plane fusion mark; Optimizing parameters of adjacent planes with the plane fusion mark by using the least square method to obtain an optimized plane parameter set, and correcting the boundary of the plane area according to the optimized plane parameter set to obtain a corrected plane model; The modified plane model is subjected to confidence calculation through multi-view consistency check to obtain a plane confidence score, and the modified plane model is subjected to time series fusion according to the plane confidence score to obtain an environmental structure model with a confidence score.

8. An environment perception and path planning system based on a vehicle controller, characterized in that: The vehicle controller-based environment perception and path planning system is used to execute the vehicle controller-based environment perception and path planning method according to any one of claims 1 to 7, and the vehicle controller-based environment perception and path planning system includes: An image processing module is used to obtain an RGB image and event stream data through an environment perception sensor, perform pixel normalization processing on the RGB image to obtain a standardized image, and perform cumulative projection on the event stream data to obtain a two-dimensional event count map; A feature conversion module is used to extract features from the standardized image to obtain a multi-scale feature atlas of the image, and to compress the spatiotemporal features of the event count image through a preset 3D convolution layer and a preset 2D convolution layer to obtain a multi-scale feature atlas of the event; A feature fusion module, used for fusing the image multi-scale feature atlas and the event multi-scale feature atlas by an attention mechanism to obtain a fused feature pyramid; A feature enhancement module, used for performing feature enhancement processing according to the fused feature pyramid to obtain a line segment descriptor set; A timing matching module, used for performing timing matching on the line segment descriptor set to obtain optimized line segment parameters; A model building module, used to perform topological analysis and plane segmentation processing according to the optimized line segment parameters to obtain an environmental structure model with a confidence score; The identification generation module is used to output a path indication identification for assisting vehicle driving according to the output result of the environmental structure model and a preset path planning algorithm.

Images