A parking environment mapping method and system based on computer vision

Abstract

This invention discloses a parking environment mapping method based on computer vision, belonging to the field of parking environment mapping technology. It obtains a corrected multi-view image sequence of the parking environment; generates a set of temporal feature maps of the bird's-eye view in the bird's-eye view coordinate system using a keyframe caching mechanism; generates a FlashOcc coarse-grained 3D occupancy tensor; obtains a set of boundary voxels; extracts the corresponding high-frequency feature information set; generates fine-grained boundary occupancy probabilities, obtaining a PointRend fine-grained 3D occupancy tensor; and obtains a spatiotemporally consistent 3D occupancy tensor to generate the final 3D occupancy map of the parking environment. This invention effectively eliminates the jagged effect and boundary blurring problems, making the geometric contours of key structures in the parking environment smoother and more consistent with real physical dimensions, improving the usability and safety of the mapping results in the planning and control system.

Description

Technical Field

[0001] This invention relates to the field of parking environment mapping technology, and in particular to a parking environment mapping method based on computer vision. Background Technology

[0002] As autonomous driving technology advances further into low-speed, complex scenarios for automated valet parking, vision-based 3D environment mapping technology is gradually becoming an important component of vehicle-side perception systems. In existing technologies, most methods rely on 3D occupancy networks to perform voxel-level modeling of multi-view images. This involves constructing dense 3D voxel meshes and employing 3D convolutional networks for feature extraction and spatial reasoning to obtain the 3D distribution of obstacles and drivable areas in the environment. However, in parking scenarios, the computing power and storage resources of onboard computing platforms are limited. The computational complexity of traditional 3D convolutional mapping methods increases exponentially with spatial resolution, making it difficult to meet the real-time requirements of vehicles and hindering the deployment of related algorithms in mass-production systems.

[0003] To reduce computational complexity, some existing technologies employ spatial dimensionality reduction or low-resolution voxel partitioning strategies, simplifying the 3D mapping process into 2D bird's-eye view modeling or sparse voxel representation. However, these methods inevitably introduce detail loss while compressing spatial information, resulting in noticeable jagged edges and blurred boundaries at object boundaries in the generated 3D occupancy results. In parking scenarios, key structures such as pillar edges, parking lock outlines, and parking space corner points have small dimensions and require high positioning accuracy. Existing methods struggle to accurately characterize these fine-grained structures, easily leading to path planning misjudgments or control deviations.

[0004] Existing boundary refinement methods for 2D image segmentation are difficult to apply directly to 3D occupancy mapping scenarios. They lack effective uncertainty modeling and adaptive optimization mechanisms for boundary regions in 3D voxel space, making it difficult for lightweight mapping schemes to achieve a balance between accuracy and efficiency. Summary of the Invention

[0005] One objective of this invention is to propose a computer vision-based method for mapping parking environments. This invention effectively eliminates the jagged edges and boundary blurring problems, making the geometric contours of key structures in the parking environment smoother and more consistent with their actual physical dimensions, thereby improving the usability and safety of the mapping results in the planning and control system.

[0006] A parking environment mapping method based on computer vision according to an embodiment of the present invention includes: A multi-view image sequence of the parking environment is acquired and preprocessed to obtain a corrected multi-view image sequence of the parking environment. The corrected parking environment multi-view image sequence is input into the temporal variable resolution bird's-eye view feature encoder. The resolution is dynamically adjusted according to the vehicle motion state and temporal two-dimensional features are extracted. Combined with the key frame caching mechanism, a set of bird's-eye view temporal feature maps is generated in the bird's-eye view coordinate system. An improved FlashOcc channel-height folding mechanism is invoked on the bird's-eye view temporal feature map set to rearrange the channel dimension into the height dimension, generating a FlashOcc coarse-grained three-dimensional occupancy tensor. Uncertainty assessment is performed on the FlashOcc coarse-grained 3D occupancy tensor to obtain the boundary voxel set; The boundary voxel set is input into the PointRend adaptive point sampling module, which performs sparse sampling and resampling according to the voxel distribution density and spatial location. The three-dimensional coordinates of the boundary voxels are used to back-index the bird's-eye view temporal feature map set and the original multi-scale image features to extract the corresponding high-frequency feature information set. The high-frequency feature information set is input into the PointRend dual-path multilayer perceptron rendering module. Spatial feature fusion and appearance feature fusion prediction are performed on each boundary voxel to generate fine-grained boundary occupancy probability. The fine-grained boundary occupancy probability is then used to update the FlashOcc coarse-grained 3D occupancy tensor to obtain the PointRend fine-grained 3D occupancy tensor. Spatiotemporal consistency fusion is performed on the PointRend fine-grained 3D occupancy tensor. The 3D occupancy tensor of the previous frame is aligned to the current coordinate system after vehicle attitude compensation. It is then fused with the PointRend fine-grained 3D occupancy tensor of the current frame at the voxel level to obtain a spatiotemporally consistent 3D occupancy tensor. Morphological optimization processing is performed on the spatiotemporally consistent 3D occupancy tensor to generate the final 3D occupancy map of the parking environment.

[0007] Optionally, the preprocessing includes time synchronization, intrinsic and extrinsic parameter correction, and distortion correction.

[0008] Optionally, the step of inputting the corrected parking environment multi-view image sequence into a time-varying resolution bird's-eye view feature encoder includes: Based on the corrected multi-view image sequence of the parking environment, the resolution adjustment intensity corresponding to the t-th frame is calculated. The resolution adjustment intensity is then subjected to nonlinear normalization mapping with the preset minimum resolution adjustment coefficient and the preset maximum resolution adjustment coefficient to obtain the resolution adjustment coefficient corresponding to the t-th frame. Based on the resolution adjustment coefficient, the corrected multi-view images of the parking environment corresponding to each camera are resampled to obtain the number of pixels for target input height and the number of pixels for target input width. A spatial interpolation algorithm is used to resample the corrected multi-view images of the parking environment to obtain the resampled corrected multi-view images of the parking environment. The images are then input into a temporally variable resolution bird's-eye view feature encoder. Within the spatial range defined by the number of pixels in the target input height and the number of pixels in the target input width, the feature extraction network extracts the features of each spatial location to generate a set of multi-scale two-dimensional feature maps corresponding to the number of pixels in the target input height and the number of pixels in the target input width. Temporal two-dimensional feature fusion processing is performed on the multi-scale two-dimensional feature map set to obtain a temporal two-dimensional feature map set; Mapping the temporal two-dimensional feature map set to the bird's-eye view coordinate system yields the current frame's bird's-eye view feature map set; Compare the resolution adjustment intensity with the resolution adjustment intensity threshold, the Euclidean distance between the current frame and the previous keyframe in the planar position with the planar displacement threshold, and the difference in heading angle between the current frame and the previous keyframe with the heading angle change threshold. When any of these conditions are met, the current frame is determined to be a keyframe. When the current frame is determined to be a key frame, the bird's-eye view feature map set of the current frame and the index of the current frame are written together into the key frame cache to form a key frame cache set; Align the bird's-eye view feature map sets of each key frame in the key frame cache set to the bird's-eye view coordinate system of the current frame, and merge them with the bird's-eye view feature map set of the current frame to obtain the bird's-eye view temporal feature map set.

[0009] Optionally, the step of invoking the improved FlashOcc channel-height folding mechanism on the bird's-eye view temporal feature map set to rearrange the channel dimensions into height dimensions includes: Based on the differences in local feature changes of each bird's-eye view grid position in the horizontal and vertical extension directions in the bird's-eye view time series feature map, the parking direction sensitivity coefficient corresponding to each bird's-eye view grid position is calculated. The target bird's-eye view temporal feature map is expanded by two-dimensional convolutional channels in the horizontal and vertical directions. The results of the two-dimensional convolutional channel expansion are then weighted and fused according to the parking direction sensitivity coefficient to obtain an expanded channel feature map that is adaptive to the parking direction. Based on the prior physical distribution of each occupancy category in the height direction in the parking environment, the category-height coupling modulation processing is performed on the extended channel feature map that is adaptive to the parking direction to obtain the category-height coupling extended feature set; Perform adjacent height discrete layer coupling folding processing on the category-height coupled extended feature set, rearrange the channel dimension into the height dimension within the occupied category dimension, and generate a FlashOcc coarse-grained three-dimensional occupied log tensor with adjacent height association constraints; Based on the continuity differences of each occupancy category in the height direction in the parking environment, the category continuity correction process is performed on the FlashOcc coarse-grained three-dimensional occupancy log tensor with adjacent height association constraints to obtain the continuity-corrected FlashOcc coarse-grained three-dimensional occupancy log tensor. Normalization is performed on the occupancy class dimension of the continuous correction FlashOcc coarse-grained 3D occupancy logarithm tensor to generate the FlashOcc coarse-grained 3D occupancy tensor.

[0010] Optionally, the uncertainty assessment of the FlashOcc coarse-grained three-dimensional occupancy tensor includes: Based on the class probability distribution of each voxel in the FlashOcc coarse-grained 3D occupancy tensor along the occupancy class dimension, a parking-sensitive class probability distribution for key targets in the parking environment is constructed, and the class distribution uncertainty corresponding to each voxel is calculated based on the parking-sensitive class probability distribution. Based on the local probability changes of the FlashOcc coarse-grained 3D occupancy tensor in the horizontal, vertical, and height directions of the bird's-eye view, the parking-sensitive spatial gradient corresponding to each voxel is calculated. The uncertainty of category distribution is fused with the parking-sensitive spatial gradient to obtain the boundary confidence of each voxel; Voxels in the FlashOcc coarse-grained 3D occupancy tensor are filtered based on boundary confidence to obtain boundary voxel indication results, and boundary voxel sets are extracted based on the boundary voxel indication results.

[0011] Optionally, the step of inputting the boundary voxel set into the PointRend adaptive point sampling module and performing sparse sampling and resampling according to the voxel distribution density and spatial location includes: Calculate the PointRend sampling priority value for each boundary voxel based on its local distribution density and spatial location in the boundary voxel set; Sparse sampling is performed on the boundary voxel set according to the PointRend sampling priority value to obtain the PointRend sparse sampling voxel set; Resampling is performed based on the voxel distribution density and spatial location of each sparse sampled voxel in the PointRend sparse sampled voxel set to obtain the PointRend resampling point set; Using the three-dimensional coordinates in the PointRend resampling point set as a reverse index, the bird's-eye view temporal feature map set is extracted to extract the corresponding high-frequency feature vectors of the bird's-eye view. The original multi-scale image features are extracted by using the three-dimensional coordinates in the PointRend resampling point set to inversely index the original multi-scale image features; The high-frequency feature vectors from the bird's-eye view and the high-frequency features of the original image are summarized to obtain the high-frequency feature information set of the corresponding boundary voxel set.

[0012] Optionally, the step of inputting the high-frequency feature information set into the PointRend dual-path multilayer perceptron rendering module, and performing spatial feature fusion and appearance feature fusion prediction for each boundary voxel, includes: The high-frequency features of all resampled points belonging to the same boundary voxel in the high-frequency feature information set are aggregated to obtain the bird's-eye view high-frequency aggregated feature vector and the original image high-frequency aggregated feature vector corresponding to each boundary voxel. The high-frequency aggregated feature vector of the bird's-eye view is input into the spatial feature fusion path in the PointRend dual-path multilayer perceptron rendering module, and the high-frequency aggregated feature vector of the original image is input into the appearance feature fusion path in the PointRend dual-path multilayer perceptron rendering module to obtain the spatial fusion feature vector and appearance fusion feature vector corresponding to each boundary voxel. Perform dual-path fusion prediction on the spatial fusion feature vector and the appearance fusion feature vector to obtain the fine-grained boundary occupancy probability of each boundary voxel in each occupancy category; The occupancy value of the corresponding boundary voxel position in the FlashOcc coarse-grained 3D occupancy tensor is updated based on the fine-grained boundary occupancy probability to obtain the PointRend fine-grained 3D occupancy tensor.

[0013] Optionally, the spatiotemporal consistency fusion of the PointRend fine-grained 3D occupancy tensor, which aligns the previous frame's 3D occupancy tensor to the current coordinate system after vehicle pose compensation, and performs voxel-level voting fusion with the current frame's PointRend fine-grained 3D occupancy tensor, includes: Align the PointRend fine-grained 3D occupancy tensor of the previous frame to the coordinate system of the current frame to obtain the pose-compensated aligned 3D occupancy tensor of the previous frame. The previous frame aligned 3D occupancy tensor after attitude compensation is fused with the current frame PointRend fine-grained 3D occupancy tensor at the voxel level to obtain a spatiotemporally consistent 3D occupancy tensor. Perform morphological optimization on the spatiotemporally consistent 3D occupancy tensor to obtain the morphologically optimized 3D occupancy tensor of the parking environment. The final 3D occupancy map of the parking environment is generated based on the morphologically optimized 3D occupancy tensor of the parking environment.

[0014] A computer vision-based parking environment mapping system is used to execute a computer vision-based parking environment mapping method, including: The data acquisition and preprocessing module is used to acquire and preprocess multi-view image sequences of the parking environment to obtain corrected multi-view image sequences of the parking environment. The temporal bird's-eye view feature encoding module is used to input the corrected multi-view image sequence of the parking environment into the temporal variable resolution bird's-eye view feature encoder to generate a set of temporal feature maps of bird's-eye view. The FlashOcc 3D occupancy generation module is used to call the improved FlashOcc channel-height folding mechanism on the bird's-eye view temporal feature map set to rearrange the channel dimension into the height dimension and generate the FlashOcc coarse-grained 3D occupancy tensor. The boundary voxel extraction module is used to perform uncertainty assessment on the FlashOcc coarse-grained 3D occupancy tensor to obtain the boundary voxel set; The PointRend point sampling feature extraction module is used to input the boundary voxel set into the PointRend adaptive point sampling module to extract the corresponding high-frequency feature information set. The PointRend boundary rendering module is used to input the high-frequency feature information set into the PointRend dual-path multilayer perceptron rendering module to obtain the PointRend fine-grained 3D occupancy tensor. The spatiotemporal fusion and map generation module is used to perform spatiotemporal consistency fusion on the PointRend fine-grained 3D occupancy tensor to obtain a spatiotemporally consistent 3D occupancy tensor, and perform morphological optimization processing on the spatiotemporally consistent 3D occupancy tensor to generate the final 3D occupancy map of the parking environment.

[0015] The beneficial effects of this invention are: This invention introduces an improved FlashOcc channel-height folding mechanism based on parking direction sensitivity coefficients onto a bird's-eye view temporal feature map set. During the 2D convolutional channel expansion stage, it integrates the differences in lateral and vertical directional features and combines category-height coupled prior weights and adjacent height discrete layer coupling folding strategies. This allows the originally independent channel and height dimensions to form a structural expression with physical semantic constraints during the folding process. Compared to the traditional FlashOcc method that only performs simple channel rearrangement, the introduction of direction awareness and height distribution modeling before folding makes the generated coarse-grained 3D occupancy tensor more expressive of targets with clear directional and height distribution features in parking scenarios, such as pillars, walls, and parking lines. While maintaining a completely 2D computational flow, it achieves enhanced consistency in 3D structural semantics, thereby reducing computational power consumption and avoiding disordered folding of spatial structural information.

[0016] This invention constructs a joint uncertainty assessment mechanism based on the probability distribution of parking-sensitive categories and the spatial gradient of parking sensitivity. This expands voxel-level uncertainty from a single probability dimension to a coupled measure of category distribution and spatial variation. Combined with boundary confidence screening and vehicle-side maximum rendering point quota constraints, it achieves precise localization of key boundary voxels in the 3D occupied space. By introducing parking-sensitive category weights, the boundary screening process assigns higher priority to high-risk targets such as parking locks, bollards, and parking lines. Simultaneously, through three-directional joint modeling of spatial gradients, the stability and continuity of boundary region detection are improved, achieving efficient focusing on key areas within a limited computational budget.

[0017] This invention extends the PointRend mechanism from two-dimensional image segmentation to three-dimensional voxel space, constructing a fine-grained boundary reconstruction method based on high-frequency feature aggregation of resampled points and dual-path multilayer perceptron rendering. It introduces spatial position coding and FlashOcc coarse-grained occupancy prior vectors into the spatial feature fusion path, and incorporates high-frequency information from multi-view original images into the appearance feature fusion path. A gating mechanism is used to achieve dual-path adaptive fusion prediction, performing point-by-point rendering repair only on boundary voxels. With approximately unchanged computational complexity, it achieves sub-voxel-level fine reconstruction of three-dimensional occupancy boundaries, effectively eliminating jagged edges and boundary blurring. This makes the geometric contours of key structures in parking environments smoother and more consistent with actual physical dimensions, improving the usability and safety of the mapping results in the planning and control system. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a parking environment mapping method based on computer vision proposed in this invention; Figure 2 This is a block diagram of the PointRend dual-path multilayer perceptron rendering module, which is part of a computer vision-based parking environment mapping method proposed in this invention. Detailed Implementation

[0019] Example 1: Reference Figures 1-2 A computer vision-based method for mapping parking environments includes: A multi-view image sequence of the parking environment is acquired and preprocessed to obtain a corrected multi-view image sequence of the parking environment. In this embodiment, preprocessing includes time synchronization, intrinsic and extrinsic parameter correction, and distortion correction.

[0020] The corrected parking environment multi-view image sequence is input into the temporal variable resolution bird's-eye view feature encoder. The resolution is dynamically adjusted according to the vehicle motion state and temporal two-dimensional features are extracted. Combined with the key frame caching mechanism, a set of bird's-eye view temporal feature maps is generated in the bird's-eye view coordinate system. In this embodiment, the resolution adjustment intensity corresponding to the t-th frame is calculated based on the corrected multi-view image sequence of the parking environment. The resolution adjustment intensity is then subjected to non-linear normalization mapping with the preset minimum resolution adjustment coefficient and the preset maximum resolution adjustment coefficient to obtain the resolution adjustment coefficient corresponding to the t-th frame. In Example 1, the corrected parking environment multi-view image sequence is represented as an image set consisting of multiple time frames and multiple cameras. Each time frame corresponds to each camera and contains one corrected parking environment multi-view image.

[0021] The vehicle's current kinematic state data is acquired, including the vehicle's longitudinal velocity, longitudinal acceleration, and yaw rate. The vehicle's longitudinal velocity, longitudinal acceleration, and yaw rate are weighted and fused to obtain the resolution adjustment intensity corresponding to the t-th frame. The kinematic state data is estimated by matching the environmental features of adjacent time frames of the corrected parking environment multi-view image sequence.

[0022] The preset minimum resolution adjustment coefficient is determined by the minimum image input scale corresponding to the vehicle computing platform under the condition of meeting the real-time inference frame rate constraint, and the preset maximum resolution adjustment coefficient is determined by the maximum image input scale corresponding to the vehicle computing platform within the allowable range of video memory and computing resources.

[0023] Based on the resolution adjustment coefficient, the corrected multi-view images of the parking environment corresponding to each camera are resampled to obtain the number of pixels for target input height and the number of pixels for target input width. In Example 1, the target input height in pixels is obtained by multiplying the preset reference input height in pixels with the resolution adjustment coefficient and then rounding the result. The target input width in pixels is obtained by multiplying the preset reference input width in pixels with the resolution adjustment coefficient and then rounding the result. The target input height in pixels and the target input width in pixels are used as the spatial dimensions of the resampled and corrected multi-view image of the parking environment, so that the resampled and corrected multi-view image of the parking environment is consistent with the resolution adjustment coefficient in spatial dimension.

[0024] A spatial interpolation algorithm is used to resample the corrected multi-view images of the parking environment to obtain the resampled corrected multi-view images of the parking environment. The images are then input into a temporally variable resolution bird's-eye view feature encoder. Within the spatial range defined by the number of pixels in the target input height and the number of pixels in the target input width, the feature extraction network extracts the features of each spatial location to generate a set of multi-scale two-dimensional feature maps corresponding to the number of pixels in the target input height and the number of pixels in the target input width. In Example 1, for each target pixel position in the corrected multi-view image of the parking environment corresponding to the m-th camera in the t-th frame, the target pixel position is reverse-mapped to the original pixel coordinate position, and the pixel value of the pixel in the neighborhood of the original pixel coordinate position is selected based on a preset interpolation strategy. The pixel value is then weighted and summed according to the corresponding interpolation ratio to obtain the resampled pixel value, thus obtaining the corrected multi-view image of the parking environment after resampling.

[0025] The resampled and corrected multi-view images of the parking environment are input into a temporal variable resolution bird's-eye view feature encoder. Layer-by-layer feature calculations are performed on each spatial location within the spatial range defined by the number of pixels in the target input height and the number of pixels in the target input width. In the first scale layer, multiple pixels are selected within the local neighborhood corresponding to the spatial location. The pixel values ​​of these pixels are then multiplied element-wise with preset weights. The summation of all product results is followed by the addition of a corresponding bias term. A nonlinear transformation is applied to the superimposed result to obtain the two-dimensional feature value of the corresponding spatial location at the first scale layer. After the first scale layer, for any spatial location, multiple two-dimensional feature values ​​are selected within the local neighborhood of the corresponding spatial location in the previous scale layer. These two-dimensional feature values ​​are then multiplied element-wise with preset weights. The summation of all product results is followed by the addition of a corresponding bias term. A nonlinear transformation is applied to the superimposed result to obtain the two-dimensional feature value of the corresponding spatial location at the current scale layer. A multi-scale two-dimensional feature map set consistent with the number of pixels in the target input height and the number of pixels in the target input width is generated layer by layer. The temporal two-dimensional feature map set is then mapped to the bird's-eye view coordinate system.

[0026] Temporal two-dimensional feature fusion processing is performed on the multi-scale two-dimensional feature map set to obtain a temporal two-dimensional feature map set; In Example 1, for frame t, within a preset number of historical frames, multi-scale two-dimensional feature maps of the corresponding historical frames are obtained, and corresponding temporal fusion weights are assigned to each historical frame. By weighting and summing the feature values ​​of the current frame and each historical frame in the same spatial position and the same channel according to the corresponding temporal fusion weights, a set of temporal two-dimensional feature maps is obtained.

[0027] Mapping the temporal two-dimensional feature map set to the bird's-eye view coordinate system yields the current frame's bird's-eye view feature map set; In Example 1, for each bird's-eye view grid position in the current frame's bird's-eye view coordinate system, based on the bird's-eye view horizontal grid resolution, the bird's-eye view vertical grid resolution, and the preset height sampling layer, multiple three-dimensional spatial sampling points corresponding to the bird's-eye view grid position in the vehicle coordinate system are determined. Based on the intrinsic and extrinsic parameters of each camera, each three-dimensional spatial sampling point is projected onto the pixel position in the corresponding camera image plane. For each pixel position, the corresponding temporal two-dimensional feature value is extracted from the temporal two-dimensional feature map set. The temporal two-dimensional feature values ​​extracted from multiple three-dimensional spatial sampling points corresponding to the same bird's-eye view grid position under multiple cameras are fused. The fused feature value is used as the bird's-eye view feature value of that bird's-eye view grid position. After traversing all bird's-eye view grid positions, the current frame's bird's-eye view feature map set is obtained.

[0028] Compare the resolution adjustment intensity with the resolution adjustment intensity threshold, the Euclidean distance between the current frame and the previous keyframe in the planar position with the planar displacement threshold, and the difference in heading angle between the current frame and the previous keyframe with the heading angle change threshold. When any of these conditions are met, the current frame is determined to be a keyframe. In Example 1, the current position coordinates of the vehicle in the current frame in the vehicle coordinate reference system and the current heading angle of the vehicle in the current frame are obtained. The position coordinates of the vehicle in the previous key frame in the vehicle coordinate reference system and the heading angle of the vehicle in the previous key frame are obtained. The Euclidean distance between the current frame and the previous key frame in the planar position is calculated using the current position coordinates of the vehicle in the current frame and the position coordinates of the previous key frame. The difference in heading angle between the current frame and the previous key frame is calculated using the current heading angle and the heading angle of the previous key frame.

[0029] When the current frame is determined to be a key frame, the bird's-eye view feature map set of the current frame and the index of the current frame are written together into the key frame cache to form a key frame cache set; Align the bird's-eye view feature map sets of each key frame in the key frame cache set to the bird's-eye view coordinate system of the current frame, and merge them with the bird's-eye view feature map set of the current frame to obtain the bird's-eye view temporal feature map set.

[0030] In Example 1, by calculating the change in heading angle from the key frame to the current frame, as well as the lateral and longitudinal displacements on the bird's-eye view plane, a planar rigid transformation relationship is constructed. Each grid position in the bird's-eye view feature map of the key frame is then mapped to the current frame's bird's-eye view grid position through the planar rigid transformation relationship.

[0031] The aligned grid coordinates are determined by rounding and the corresponding feature values ​​are read. The current frame bird's-eye view feature map set is fused with the bird's-eye view feature map sets of all aligned key frames. The current frame is assigned a current frame fusion weight and each key frame is assigned a key frame fusion weight based on time interval decay. The bird's-eye view temporal feature map set is obtained by weighted summing of the current frame feature values ​​and the feature values ​​of each key frame according to the corresponding fusion weights.

[0032] An improved FlashOcc channel-height folding mechanism is invoked on the bird's-eye view temporal feature map set to rearrange the channel dimension into the height dimension, generating a FlashOcc coarse-grained three-dimensional occupancy tensor. In this embodiment, based on the differences in local feature changes of each bird's-eye view grid position in the horizontal extension direction and the vertical extension direction in the bird's-eye view time series feature map, the parking direction sensitivity coefficient corresponding to each bird's-eye view grid position is calculated. In Example 1, for each bird's-eye view grid position, a local smoothing operator is used to reduce high-frequency noise interference before extracting the local feature change. Within a preset local direction perception radius, feature values ​​at symmetrical positions on both sides of the current grid position are selected along the horizontal direction of the bird's-eye view. The absolute value of the channel-by-channel difference of the feature values ​​at the symmetrical positions on both sides is calculated, and the absolute values ​​of the differences of all channels and all offset distances are accumulated to obtain the local feature change of the corresponding bird's-eye view grid position along the horizontal direction of the bird's-eye view. Within the same local direction perception radius, the local feature change of the corresponding bird's-eye view grid position along the vertical direction of the bird's-eye view is calculated in the same way.

[0033] In practical applications, the horizontal and vertical directions along the bird's-eye view are not limited to fixed orthogonal coordinate axes. They can also include multi-directional feature gradient calculations that extend along multiple preset oblique angles. By extracting the direction of maximum response as the main extension direction, it can adapt to non-orthogonal oblique parking spaces or vehicle turning conditions in underground parking garages.

[0034] The normalized ratio of the local feature variation along the horizontal direction and the local feature variation along the vertical direction of the bird's-eye view is calculated to obtain the parking direction sensitivity coefficient of the corresponding grid position in the bird's-eye view. This coefficient is used to represent the degree of dominance of the feature at the corresponding position in the horizontal and vertical extension directions, and to adapt to the directional features of parking lines, wall edges and vehicle outlines in the parking environment.

[0035] The target bird's-eye view temporal feature map is expanded by two-dimensional convolutional channels in the horizontal and vertical directions. The results of the two-dimensional convolutional channel expansion are then weighted and fused according to the parking direction sensitivity coefficient to obtain an expanded channel feature map that is adaptive to the parking direction. In Example 1, the preset number of height discrete layers is used to represent the total number of discrete layers divided along the height direction, and the preset number of occupancy categories is used to represent the total number of categories participating in the three-dimensional occupancy determination in the parking environment. The number of occupancy categories includes at least one idle state category to absorb the background probability that does not belong to any physical obstacle or sign. The number of extended channels is determined according to the product of the number of height discrete layers and the number of occupancy categories.

[0036] In the horizontal two-dimensional convolution channel expansion process, for each grid position in the bird's-eye view, multiple neighborhood feature values ​​are selected within its horizontal neighborhood range. The selected neighborhood feature values ​​are then multiplied element-wise with the horizontal convolution weights. The product results are accumulated and superimposed with the corresponding bias term before being passed through a nonlinear activation function to obtain the horizontal two-dimensional convolution channel expansion feature values. In the vertical two-dimensional convolution channel expansion process, the same calculation method as the horizontal two-dimensional convolution channel expansion is used, but the neighborhood selection range is along the vertical direction of the bird's-eye view to obtain the vertical two-dimensional convolution channel expansion feature values.

[0037] The horizontal and vertical two-dimensional convolutional channel extension feature values ​​are weighted and fused according to the parking direction sensitivity coefficient of the corresponding bird's-eye view grid position, so that the direction with a larger parking direction sensitivity coefficient has a higher proportion in the fusion result, thus obtaining an extended channel feature map that is adaptive to parking direction.

[0038] Based on the prior physical distribution of each occupancy category in the height direction in the parking environment, the category-height coupling modulation processing is performed on the extended channel feature map that is adaptive to the parking direction to obtain the category-height coupling extended feature set; The category-highly coupled modulation processing specifically includes mapping the height discrete layers to the actual physical height range of the parking environment according to a preset height interval, with each height discrete layer corresponding to a physical height value.

[0039] To eliminate relative height errors caused by ramps or uneven ground, the pitch and roll angles of the current vehicle are obtained or a local ground height reference plane is extracted. The physical height value is dynamically offset and compensated based on the local ground height reference plane. For each occupancy category, a height center prior value and a height discrete prior value are preset in the parking environment. The height center prior value is used to represent the height position where the occupancy category mainly appears, and the height discrete prior value is used to represent the distribution range of the occupancy category in the height direction.

[0040] For each occupancy category and each height discrete layer, based on the degree of deviation between the dynamically compensated physical height value corresponding to the height discrete layer and the height center prior value of the occupancy category, and in conjunction with the height discrete prior value of the occupancy category, calculate the category-height prior weight of the occupancy category in the height discrete layer: The extended channels in the extended channel feature map of parking direction adaptation are renumbered according to occupancy category and height discrete layer, and the feature value of each extended channel is multiplied with the corresponding category-height prior weight to obtain the category-height coupled extended feature value:

[0041] Perform adjacent height discrete layer coupling folding processing on the category-height coupled extended feature set, rearrange the channel dimension into the height dimension within the occupied category dimension, and generate a FlashOcc coarse-grained three-dimensional occupied log tensor with adjacent height association constraints; In Example 1, the extended channels in the category-highly coupled extended features are grouped according to the order of occupying categories, and arranged according to the order of highly discrete layers within each occupying category, so that each extended channel uniquely corresponds to one occupying category and one highly discrete layer.

[0042] For any grid position in a bird's-eye view, after determining the current occupied category and the current height discrete layer, the category-height coupling extended feature value corresponding to the current height discrete layer is read from the extended channel group corresponding to the occupied category. At the same time, the category-height coupling extended feature values ​​corresponding to the adjacent height discrete layers below the current height discrete layer and the adjacent height discrete layers above the current height discrete layer are read: when the current height discrete layer is located at the lowest height boundary, the category-height coupling extended feature value corresponding to the adjacent height discrete layer below is set to zero; when the current height discrete layer is located at the highest height boundary, the category-height coupling extended feature value corresponding to the adjacent height discrete layer above is set to zero.

[0043] Based on the category-height prior weights of the currently occupied category in the current height discrete layer, the adjacent height discrete layer below, and the adjacent height discrete layer above, calculate the downward coupling coefficient, central coupling coefficient, and upward coupling coefficient of the current height discrete layer respectively. The larger the category-height prior weight of the adjacent height discrete layer below, the larger the downward coupling coefficient; the larger the category-height prior weight of the current height discrete layer, the larger the central coupling coefficient; and the larger the category-height prior weight of the adjacent height discrete layer above, the larger the upward coupling coefficient.

[0044] The category-height coupling extended feature value corresponding to the adjacent height discrete layer below is multiplied with the downward coupling coefficient. The category-height coupling extended feature value corresponding to the current height discrete layer is multiplied with the center coupling coefficient. The category-height coupling extended feature value corresponding to the adjacent height discrete layer above is multiplied with the upward coupling coefficient. The three product results are summed to obtain the FlashOcc coarse-grained 3D occupancy logarithm value of the current occupancy category in the current height discrete layer with adjacent height association constraints at the current bird's-eye view grid position.

[0045] Based on the continuity differences of each occupancy category in the height direction in the parking environment, the category continuity correction process is performed on the FlashOcc coarse-grained three-dimensional occupancy log tensor with adjacent height association constraints to obtain the continuity-corrected FlashOcc coarse-grained three-dimensional occupancy log tensor. In Example 1, a height continuity coefficient of the occupancy category in the parking environment is preset to represent the degree of continuous extension of the occupancy category in the height direction.

[0046] For each bird's-eye view grid location and each occupation category, on the corresponding height discretization layer, obtain the 3D occupation logarithmic value of the occupation category at the height discretization layer, and execute a two-way support strategy based on the physical properties of the occupation category: For the wall and column grounding categories, obtain the three-dimensional occupancy logarithmic value of the occupancy category at all height discrete layers below the height discrete layer. Then, accumulate the three-dimensional occupancy logarithmic value of the occupancy category at all height discrete layers below the height discrete layer layer, and divide the accumulated result by the number of height discrete layers participating in the accumulation to obtain the cumulative logarithmic support value of the occupancy category below that height discrete layer.

[0047] For ventilation ducts and suspended signs, the three-dimensional occupancy logarithmic values ​​of all height discrete layers above the height discrete layer are obtained and accumulated to obtain the cumulative logarithmic support value above.

[0048] The 3D occupancy logarithmic value at the highly discrete layer is weighted and fused with the corresponding lower or upper cumulative logarithmic support value. The weight of the support value is determined by the height continuity coefficient corresponding to the occupancy category, so that the occupancy category with a larger height continuity coefficient has a higher dependence on the support value during the fusion process, thus obtaining the continuity-corrected 3D occupancy logarithmic value of the occupancy category at the highly discrete layer.

[0049] Normalization is performed on the occupancy class dimension of the continuous correction FlashOcc coarse-grained 3D occupancy logarithm tensor to generate the FlashOcc coarse-grained 3D occupancy tensor.

[0050] The improved FlashOcc architecture in this embodiment includes a parking direction-sensitive dual-branch convolution module, a category-height prior modulation layer, an adjacent height coupled folding layer, and a physical bidirectional continuity correction module. Compared with the traditional FlashOcc, which uses isotropic convolution and purely data-driven blind channel folding, the substantial improvement of the improved FlashOcc lies in the design of an anisotropic convolution extraction mechanism based on local feature gradients and the explicit injection of a Gaussian prior of physical height distribution combined with attitude compensation. At the same time, it constructs an asymmetric spatiotemporal continuity correction logic that conforms to the constraints of gravity and suspension in the real world, making up for the loss of physical information in the process of upgrading from 2D to 3D in the traditional FlashOcc. It not only adapts to the geometric features of the inclined parking spaces and slender pillars in the underground parking garage, but also eliminates the high distortion blind spots caused by cross-level ramps, eliminates ghost obstacles and voxel discontinuity floating phenomena, and achieves high-fidelity parking space reconstruction that meets the strict physical coherence principle while still maintaining the constraint of extremely low computational cost of pure 2D.

[0051] Uncertainty assessment is performed on the FlashOcc coarse-grained 3D occupancy tensor to obtain the boundary voxel set; In this embodiment, based on the class probability distribution of each voxel in the FlashOcc coarse-grained three-dimensional occupancy tensor in the occupancy class dimension, a parking-sensitive class probability distribution for key targets in the parking environment is constructed, and the class distribution uncertainty corresponding to each voxel is calculated based on the parking-sensitive class probability distribution. In Example 1, considering the different impacts of pillars, walls, parked vehicles, ground locks, parking lines, and vacant areas on parking safety in the parking environment, a parking sensitivity category weight is preset for each occupation category. The occupation probability of a voxel in each occupation category is multiplied by the corresponding parking sensitivity category weight, and the product results of all occupation categories are normalized to obtain the parking sensitivity category probability distribution.

[0052] The parking sensitivity entropy is obtained by multiplying the parking sensitivity class probability of a voxel in each occupied class with its logarithmic value, summing the product results for all occupied classes and taking the negative, and then normalizing it using the total number of occupied classes. This entropy is used to measure the uncertainty of a voxel in determining its occupied class.

[0053] Meanwhile, for each voxel, the maximum and second-largest probability values ​​are extracted from the parking sensitivity category probability distribution. The category probability separation degree is obtained by subtracting the second-largest probability value from the maximum probability value. The parking sensitivity entropy and the category probability separation degree are weighted and fused to obtain the category distribution uncertainty.

[0054] Based on the local probability changes of the FlashOcc coarse-grained 3D occupancy tensor in the horizontal, vertical, and height directions of the bird's-eye view, the parking-sensitive spatial gradient corresponding to each voxel is calculated. In Example 1, adjacent voxels to the left and right of a voxel are selected in the horizontal direction from a bird's-eye view, adjacent voxels before and after a voxel are selected in the vertical direction from a bird's-eye view, and adjacent voxels above and below a voxel are selected in the vertical direction from a voxel. Before selecting adjacent voxels for spatial gradient calculation, edge feature copying processing is performed on the FlashOcc coarse-grained 3D occupancy tensor. When a voxel is located at the boundary of the 3D spatial grid and is missing an adjacent voxel in a certain direction, the occupancy probability of the current voxel itself is used as the occupancy probability of the adjacent voxel on the missing side for difference calculation, ensuring the completeness and continuity of spatial gradient calculation in the entire space and avoiding the bottom-level tensor from going out of bounds.

[0055] In the horizontal direction of the bird's-eye view, the occupancy probabilities of adjacent voxels in each occupancy category are calculated by difference and the absolute value is taken. The absolute values ​​of all occupancy categories are weighted and summed and then normalized. The weights are the parking sensitivity category weights, and the parking sensitivity spatial gradient of the voxel in the horizontal direction of the bird's-eye view is obtained.

[0056] In the longitudinal and height directions of the bird's-eye view, the same calculation method as in the lateral direction of the bird's-eye view is used to obtain the parking-sensitive spatial gradient of the voxel in the longitudinal direction and the parking-sensitive spatial gradient of the voxel in the height direction. The parking-sensitive spatial gradients in the three directions are weighted and summed to obtain the parking-sensitive spatial gradient of the voxel.

[0057] The uncertainty of category distribution is fused with the parking-sensitive spatial gradient to obtain the boundary confidence of each voxel; In Example 1, for each voxel, the category distribution uncertainty is multiplied by its corresponding fusion weight, and the parking sensitivity spatial gradient is multiplied by its corresponding fusion weight. The two product results are summed to obtain the boundary uncertainty of the voxel. The boundary confidence of the voxel is obtained by subtracting the boundary uncertainty from 1.

[0058] Voxels in the FlashOcc coarse-grained 3D occupancy tensor are filtered based on boundary confidence to obtain boundary voxel indication results, and boundary voxel sets are extracted based on the boundary voxel indication results.

[0059] In Example 1, for each voxel, the boundary confidence score of the voxel is compared with a preset boundary confidence score threshold: When the boundary confidence is less than or equal to the preset boundary confidence threshold, the corresponding voxel is marked as a boundary voxel.

[0060] When the boundary confidence score is greater than the preset boundary confidence score threshold, the corresponding voxel is marked as a non-boundary voxel, and the boundary voxel indication results of all voxels are obtained.

[0061] For all voxels, the set of voxels marked as boundary voxels is selected based on the boundary voxel indication results.

[0062] For each selected boundary voxel, its spatial position in the parking environment physical coordinate system is calculated based on the bird's-eye view grid position and the height discrete layer index. The corresponding positions of all boundary voxels in the bird's-eye view grid coordinate system and the parking environment physical coordinate system are summarized to obtain the boundary voxel set.

[0063] To ensure the static allocation of video memory and the real-time inference frame rate of the vehicle computing platform, when the number of selected boundary voxels exceeds the preset maximum number of rendering points on the vehicle, the selected boundary voxels are sorted from low to high according to the boundary confidence, and the top K most uncertain voxels are selected as the final set of boundary voxels. If the number of selected voxels is lower than the quota, zero padding is used to keep the output tensor size fixed.

[0064] The boundary voxel set is input into the PointRend adaptive point sampling module, which performs sparse sampling and resampling according to the voxel distribution density and spatial location. The three-dimensional coordinates of the boundary voxels are used to back-index the bird's-eye view temporal feature map set and the original multi-scale image features to extract the corresponding high-frequency feature information set. In this embodiment, the PointRend sampling priority value corresponding to each boundary voxel is calculated based on the local distribution density and spatial position of each boundary voxel in the boundary voxel set. In Example 1, the boundary voxel set is represented as a set consisting of multiple effective boundary voxels. For each effective boundary voxel, the sum of squared three-dimensional spatial distances between the current effective boundary voxel and all other effective boundary voxels is calculated in the physical coordinate system of the parking environment. The sum of squared three-dimensional spatial distances is then substituted into an exponential decay function and accumulated and averaged on all effective boundary voxels to obtain the local distribution density of the boundary voxels.

[0065] Based on the planar distance and height relative to the vehicle origin in the physical coordinate system of the parking environment for the effective boundary voxel, the planar distance and height position terms are calculated using an exponential decay function. These terms are then weighted and summed according to a preset fusion weight to obtain the spatial position priority value of the boundary voxel corresponding to that effective boundary voxel. The local distribution density of boundary voxels and the spatial location priority value of boundary voxels are weighted and summed according to the preset fusion weight to obtain the PointRend sampling priority value corresponding to each effective boundary voxel.

[0066] Sparse sampling is performed on the boundary voxel set according to the PointRend sampling priority value to obtain the PointRend sparse sampling voxel set; In Example 1, the number of valid boundary voxels in the current frame boundary voxel set is compared with the vehicle-end maximum sparse sampling voxel quota, and the smaller of the two is taken as the actual sparse sampling voxel number. The vehicle-end maximum sparse sampling voxel quota is used to limit the maximum number of voxels input to the PointRend adaptive point sampling module.

[0067] The PointRend sampling priority values ​​corresponding to all valid boundary voxels are sorted in descending order. The valid boundary voxels corresponding to the number of actual sparse sampling voxels in the sorting results are selected as the PointRend sparse sampling voxel set, so that boundary voxels with higher sampling priority values ​​are retained first.

[0068] Resampling is performed based on the voxel distribution density and spatial location of each sparse sampled voxel in the PointRend sparse sampled voxel set to obtain the PointRend resampling point set; In Example 1, based on the local distribution density of the boundary voxels of the PointRend sparse sampling voxels, the preset maximum number of resampling points is reduced by one and then multiplied with the sampling priority value corresponding to the local distribution density of the boundary voxels. The product is then rounded down and then added by one to obtain the number of resampling points, so that the region with higher voxel distribution density generates more resampling points.

[0069] A local three-dimensional neighborhood space is constructed with the horizontal, vertical, and height physical coordinates of the PointRend sparse sampling voxel as the center. Multiple offset positions are generated in the local three-dimensional neighborhood space according to the preset resampling offset template. The preset resampling offset template consists of multiple three-dimensional normalized offset vectors. Each three-dimensional normalized offset vector contains a horizontal offset component, a vertical offset component, and a height offset component.

[0070] For each offset position, the corresponding 3D normalized offset vector is scaled and superimposed on the 3D physical coordinates of the sparse sampling voxel by scaling the horizontal grid resolution, the vertical grid resolution, and the height discrete layer interval of the bird's-eye view, respectively, to obtain the 3D physical coordinates of the resampling point. All resampling points generated by each PointRend sparse sampling voxel are summarized to obtain the PointRend resampling point set.

[0071] Using the three-dimensional coordinates in the PointRend resampling point set as a reverse index, the bird's-eye view temporal feature map set is extracted to extract the corresponding high-frequency feature vectors of the bird's-eye view. In Example 1, for each resampling point, its horizontal and vertical physical coordinates in the parking environment physical coordinate system are divided by the product of the downsampling step size of the bird's-eye view temporal feature map of the corresponding scale layer and the bird's-eye view grid resolution, respectively. The grid center offset of the vehicle origin in the feature map of the corresponding scale layer is then superimposed to shift the physical coordinates centered on the vehicle to the feature map tensor coordinate system, ensuring the non-negativity and accuracy of the reverse index coordinates, thus obtaining the reverse index coordinates of the resampling point in the bird's-eye view temporal feature map of the corresponding scale layer.

[0072] Feature values ​​of the integer grid position where the reverse index coordinates are located and the adjacent grid positions are selected. Bilinear interpolation weights are calculated based on the distance between the reverse index coordinates and the integer grid positions. The feature values ​​of the adjacent grid positions are weighted and summed according to the bilinear interpolation weights to obtain the high-frequency feature vector of the bird's-eye view.

[0073] The original multi-scale image features are extracted by using the three-dimensional coordinates in the PointRend resampling point set to inversely index the original multi-scale image features; In Example 1, the horizontal, vertical, and height physical coordinates of the resampled points in the parking environment physical coordinate system are combined with a constant 1 to form a homogeneous three-dimensional coordinate vector. The coordinate system is then transformed sequentially through the extrinsic parameter matrix of the corresponding camera, transforming the resampled points from the parking environment physical coordinate system to the camera coordinate system.

[0074] Considering that multi-view images of parking environments are usually captured by wide-angle or fisheye cameras, the three-dimensional coordinates are normalized to a normalized camera plane with a depth of 1 in the camera coordinate system. Then, the radial and tangential distortion models are combined to perform nonlinear distortion correction on the normalized coordinates to eliminate the physical distortion error caused by the lens. The corrected normalized coordinates are then projected and transformed using the intrinsic parameter matrix of the corresponding camera to obtain the homogeneous projection coordinates of the resampled points on the camera image plane.

[0075] Divide the horizontal and vertical components of the homogeneous projection coordinates by the depth component to obtain the two-dimensional pixel coordinates of the resampled point on the camera image plane. Divide the two-dimensional pixel coordinates by the image feature step size of the corresponding scale layer to obtain the original image inverse index coordinates of the resampled point on the corresponding scale layer of the corresponding camera.

[0076] Feature values ​​at the integer pixel position of the inverse index coordinate of the original image and its adjacent pixel positions are selected. The horizontal interpolation weight and vertical interpolation weight are calculated based on the offset between the inverse index coordinate of the original image and the integer pixel position. The feature values ​​of adjacent pixel positions are weighted and summed according to the horizontal interpolation weight and the vertical interpolation weight to obtain the high-frequency feature vector of the original image at the corresponding scale layer of the corresponding camera for the resampled points.

[0077] When the depth component of the resampled point in the camera coordinate system is less than or equal to zero, or the two-dimensional pixel coordinate falls outside the effective boundary range of the corresponding two-dimensional feature map, feature extraction is not performed on the corresponding scale layer of the camera, and a zero feature vector is output as an invalid placeholder.

[0078] The high-frequency feature vectors from the bird's-eye view and the high-frequency features of the original image are summarized to obtain the high-frequency feature information set of the corresponding boundary voxel set.

[0079] In Example 1, since the fields of view of the multi-view cameras in the parking environment overlap, the same resampling point may extract effective original image high-frequency feature vectors from multiple cameras. For any resampling point, the original image high-frequency feature vectors extracted from all effective cameras are aggregated along the channel dimension and fused into a single multi-view aggregated image feature vector with a fixed dimension. The bird's-eye view high-frequency feature vector of the resampling point and the multi-view aggregated image feature vector are concatenated along the channel dimension to obtain the high-frequency feature information set of the corresponding boundary voxel set.

[0080] The high-frequency feature information set is input into the PointRend dual-path multilayer perceptron rendering module. Spatial feature fusion and appearance feature fusion prediction are performed on each boundary voxel to generate fine-grained boundary occupancy probability. The fine-grained boundary occupancy probability is then used to update the FlashOcc coarse-grained 3D occupancy tensor to obtain the PointRend fine-grained 3D occupancy tensor. In this embodiment, the high-frequency features of all resampled points belonging to the same boundary voxel in the high-frequency feature information set are aggregated to obtain the bird's-eye view high-frequency aggregated feature vector and the original image high-frequency aggregated feature vector corresponding to each boundary voxel. In Example 1, the high-frequency feature information set is represented as a set of high-frequency features of resampled points corresponding to multiple boundary voxels. For the i-th effective boundary voxel in the t-th frame, the total number of all effective resampled points corresponding to the effective boundary voxel is obtained. The bird's-eye view high-frequency feature vectors of all effective resampled points corresponding to the effective boundary voxel on all scale layers are concatenated. Considering that the concatenation of multi-scale features will lead to a surge in channel dimensions, in order to adapt to the computing power limitations of the vehicle computing platform, a linear projection layer is used after concatenation to map the high-dimensional features to a preset fixed hidden layer dimension. At the same time, the resampled distance weights corresponding to the same effective boundary voxel are normalized. The bird's-eye view high-frequency feature vectors after dimensionality reduction are multiplied with the corresponding normalized resampled distance weights respectively. All product results are summed to eliminate the influence of the fluctuation in the number of sampling points on the feature amplitude, and the bird's-eye view high-frequency aggregated feature vectors corresponding to the effective boundary voxels are obtained.

[0081] Similarly, the high-frequency feature vectors of the original images of all effective resampling points corresponding to the effective boundary voxels are calculated on all cameras and all scale layers to obtain the high-frequency aggregated feature vectors of the original images corresponding to the effective boundary voxels.

[0082] The high-frequency aggregated feature vector of the bird's-eye view is input into the spatial feature fusion path in the PointRend dual-path multilayer perceptron rendering module, and the high-frequency aggregated feature vector of the original image is input into the appearance feature fusion path in the PointRend dual-path multilayer perceptron rendering module to obtain the spatial fusion feature vector and appearance fusion feature vector corresponding to each boundary voxel. The PointRend dual-path multilayer perceptron rendering module specifically includes: normalizing the horizontal, vertical, and height physical coordinates of the effective boundary voxels in the parking environment physical coordinate system, and then splicing them in a fixed order to form a spatial position encoding vector.

[0083] Obtain the FlashOcc coarse-grained 3D occupancy values ​​of effective boundary voxels across all occupancy categories, and concatenate them according to the occupancy category order to form a coarse-grained occupancy prior vector. Concatenate the high-frequency aggregated feature vector of the bird's-eye view, the spatial location encoding vector, and the coarse-grained occupancy prior vector according to the channel dimension order to form the input feature vector of the spatial feature fusion path.

[0084] The spatial feature fusion pathway consists of multiple fully connected layers stacked sequentially. For each layer, the channel value of the input feature vector is multiplied element-wise with the corresponding weight, and all product results are accumulated and a bias term is added. The superposition result is then subjected to a nonlinear transformation to obtain the corresponding layer's output feature vector. The output feature vector is then used as the input feature vector for the next layer. After layer-by-layer calculation, the output result of the last layer is used as the spatial fusion feature vector.

[0085] The high-frequency aggregated feature vector of the original image is used as the input feature vector of the appearance feature fusion path. The appearance feature fusion path is also composed of multiple fully connected layers stacked sequentially. For each layer, the corresponding layer output feature vector is obtained according to the same calculation method as the spatial feature fusion path, and the output feature vector is used as the input feature vector of the next layer. After layer-by-layer calculation, the output result of the last layer is used as the appearance fusion feature vector.

[0086] To ensure tensor alignment in dual-path fusion, the last fully connected layer of the spatial feature fusion path and the appearance feature fusion path is pre-defined to have the same output channel dimension, so that the obtained spatial fusion feature vector and appearance fusion feature vector are strictly aligned in channel size.

[0087] Perform dual-path fusion prediction on the spatial fusion feature vector and the appearance fusion feature vector to obtain the fine-grained boundary occupancy probability of each boundary voxel in each occupancy category; In Example 1, the spatial fusion feature vector and the appearance fusion feature vector are concatenated along the channel dimension. The concatenated feature vector is then multiplied with a preset gating weight and a bias term is added before being input into the Sigmoid mapping function to obtain the dual-path fusion gating coefficient. Based on the dual-path fusion gating coefficient, the spatial fusion feature vector and the appearance fusion feature vector are weighted and summed to obtain the dual-path fusion feature vector.

[0088] The dual-path fusion feature vector is multiplied by the classification weights corresponding to each occupation category and a bias term is added to obtain the fine-grained boundary occupation logarithm for each occupation category. Then, an exponential operation is performed and normalized to obtain the fine-grained boundary occupation probability of the effective boundary voxel in each occupation category.

[0089] The occupancy value of the corresponding boundary voxel position in the FlashOcc coarse-grained 3D occupancy tensor is updated based on the fine-grained boundary occupancy probability to obtain the PointRend fine-grained 3D occupancy tensor. In Example 1, the boundary confidence corresponding to the effective boundary voxel is obtained, and the PointRend update weight corresponding to the effective boundary voxel is obtained by subtracting the boundary confidence from 1. For the occupancy value of the effective boundary voxel in each occupancy category, the fine-grained boundary occupancy probability and the occupancy value at the corresponding position in the FlashOcc coarse-grained 3D occupancy tensor are weighted and summed through the PointRend update weight to obtain the updated occupancy value.

[0090] For non-boundary voxels not included in the boundary voxel set, keep the occupancy value in the FlashOcc coarse-grained 3D occupancy tensor unchanged, and fill the corresponding 3D voxel position with all updated occupancy values ​​to obtain the PointRend fine-grained 3D occupancy tensor.

[0091] Spatiotemporal consistency fusion is performed on the PointRend fine-grained 3D occupancy tensor. The 3D occupancy tensor of the previous frame is aligned to the current coordinate system after vehicle attitude compensation. It is then fused with the PointRend fine-grained 3D occupancy tensor of the current frame at the voxel level to obtain a spatiotemporally consistent 3D occupancy tensor. Morphological optimization processing is performed on the spatiotemporally consistent 3D occupancy tensor to generate the final 3D occupancy map of the parking environment.

[0092] In this embodiment, spatiotemporal consistency fusion is performed on the PointRend fine-grained 3D occupancy tensor. The 3D occupancy tensor of the previous frame is aligned to the current coordinate system after vehicle pose compensation, and voxel-level voting fusion is performed with the PointRend fine-grained 3D occupancy tensor of the current frame, including: Then, the PointRend fine-grained 3D occupancy tensor of the previous frame is aligned to the coordinate system of the current frame to obtain the pose-compensated aligned 3D occupancy tensor of the previous frame. In Example 1, if the current frame is the initial frame, the fine-grained 3D occupancy tensor of the current frame PointRend is directly used as the spatiotemporally consistent 3D occupancy tensor for subsequent processing. If the current frame is not the initial frame, the lateral and longitudinal displacements of the current frame relative to the previous frame are calculated based on the current position coordinates and the position coordinates of the previous frame. At the same time, the change in heading angle of the current frame relative to the previous frame is calculated based on the current heading angle and the heading angle of the previous frame. A rigid transformation relationship of the bird's-eye view plane is constructed based on the lateral displacement, longitudinal displacement, and change in heading angle, and an inverse transformation is performed on the rigid transformation relationship of the bird's-eye view plane.

[0093] For any bird's-eye view grid position in the current frame, the physical coordinates corresponding to the bird's-eye view grid position are mapped to the coordinate system of the previous frame through inverse transformation to obtain the corresponding reverse index physical coordinates. Then, the reverse index physical coordinates are converted into the reverse index coordinates of the bird's-eye view grid in the previous frame according to the horizontal and vertical grid resolutions of the bird's-eye view.

[0094] Bilinear interpolation is performed on the previous frame's PointRend fine-grained 3D occupancy tensor for each occupancy category: the horizontal and vertical interpolation weights are calculated based on the offset between the inverted index coordinates and their neighboring integer grid positions, and the neighboring occupancy values ​​are weighted and summed according to the horizontal and vertical interpolation weights to obtain the pose-compensated previous frame aligned 3D occupancy tensor; when the inverted index coordinates exceed the effective boundary range of the previous frame's PointRend fine-grained 3D occupancy tensor, the pose-compensated previous frame aligned 3D occupancy tensor at the corresponding position is set as the current frame's PointRend fine-grained 3D occupancy value.

[0095] The previous frame aligned 3D occupancy tensor after attitude compensation is fused with the current frame PointRend fine-grained 3D occupancy tensor at the voxel level to obtain a spatiotemporally consistent 3D occupancy tensor. In Example 1, the normalized motion amplitude is calculated based on the lateral displacement, longitudinal displacement, and heading angle change of the current frame relative to the previous frame. To eliminate the timing alignment error caused by the violent movement of the vehicle, the normalized motion amplitude is converted into the voting weight of the current frame through a monotonically increasing mapping function, so that the larger the vehicle motion amplitude, the higher the voting weight of the current frame. The voting weight of the previous frame is obtained by subtracting the voting weight of the current frame from 1.

[0096] The fine-grained 3D occupancy value of the current frame PointRend is multiplied with the voting weight of the current frame. The aligned 3D occupancy value of the previous frame after pose compensation is multiplied with the voting weight of the previous frame. The two product results are summed to obtain the fused occupancy value of the voxel position in the occupancy category. The same processing is performed on all voxel positions and all occupancy categories to obtain the spatiotemporally consistent 3D occupancy tensor.

[0097] Perform morphological optimization on the spatiotemporally consistent 3D occupancy tensor to obtain the morphologically optimized 3D occupancy tensor of the parking environment. In Example 1, for any voxel location, the dominant occupancy category is determined based on the voxel's occupancy value across all occupancy categories. Within the three-dimensional neighborhood centered on the corresponding voxel, the number of neighboring voxels with the same dominant occupancy category is counted to obtain the number of neighborhood supports of the same category.

[0098] When the number of neighborhood supports of the same type is greater than or equal to the shape preservation threshold, the current voxel's occupancy value in each occupancy category remains unchanged.

[0099] When the number of neighborhood supports of the same type is less than the shape preservation threshold, the occupancy value of each occupancy category within the three-dimensional neighborhood is accumulated to obtain the cumulative occupancy support value of each occupancy category. The occupancy category with the largest cumulative occupancy support value is selected as the new dominant occupancy category, and the occupancy value of the voxel in each occupancy category is replaced with the weighted average of the corresponding occupancy values ​​of all voxels within the three-dimensional neighborhood to achieve a smooth transition of spatial features and obtain the three-dimensional occupancy tensor of the parking environment after shape optimization.

[0100] The final 3D occupancy map of the parking environment is generated based on the morphologically optimized 3D occupancy tensor of the parking environment.

[0101] In Example 1, for each voxel location, the bird's-eye view horizontal grid index, bird's-eye view vertical grid index, and height discrete layer index are mapped to the bird's-eye view horizontal grid resolution, bird's-eye view vertical grid resolution, and height discrete layer corresponding physical height values, respectively, to obtain the corresponding parking environment physical coordinates; the occupancy category with the largest occupancy value is selected as the final occupancy category label based on the voxel's occupancy value across all occupancy categories.

[0102] The bird's-eye view grid coordinates of the voxels, the physical coordinates of the parking environment, and the final occupancy category labels are combined, and the same processing is performed on all voxel positions to obtain the final 3D occupancy map of the parking environment.

[0103] A computer vision-based parking environment mapping system is used to execute a computer vision-based parking environment mapping method, including: The data acquisition and preprocessing module is used to acquire and preprocess multi-view image sequences of the parking environment to obtain corrected multi-view image sequences of the parking environment. The temporal bird's-eye view feature encoding module is used to input the corrected multi-view image sequence of the parking environment into the temporal variable resolution bird's-eye view feature encoder to generate a set of temporal feature maps of bird's-eye view. The FlashOcc 3D occupancy generation module is used to call the improved FlashOcc channel-height folding mechanism on the bird's-eye view temporal feature map set to rearrange the channel dimension into the height dimension and generate the FlashOcc coarse-grained 3D occupancy tensor. The boundary voxel extraction module is used to perform uncertainty assessment on the FlashOcc coarse-grained 3D occupancy tensor to obtain the boundary voxel set; The PointRend point sampling feature extraction module is used to input the boundary voxel set into the PointRend adaptive point sampling module to extract the corresponding high-frequency feature information set. The PointRend boundary rendering module is used to input the high-frequency feature information set into the PointRend dual-path multilayer perceptron rendering module to obtain the PointRend fine-grained 3D occupancy tensor. The spatiotemporal fusion and map generation module is used to perform spatiotemporal consistency fusion on the PointRend fine-grained 3D occupancy tensor to obtain a spatiotemporally consistent 3D occupancy tensor, and perform morphological optimization processing on the spatiotemporally consistent 3D occupancy tensor to generate the final 3D occupancy map of the parking environment.

[0104] Example 2: The application scenario for this example is a closed underground parking area. The area includes vehicles parked on both sides, concrete pillars, corner bends, parking locks, limit switches, speed bumps, parking lines, suspended directional signs, and low-level fire hydrants. The lane width varies between 5.4m and 6.1m. Some areas have reflective surfaces, shadows, and sudden changes in local illumination. The test vehicle is equipped with eight surround-view cameras, covering the front, rear, left, right, and four oblique angles. The single-channel image resolution is 1920×1080, and the sampling frequency is 30 frames / second. The vehicle-side computing platform is a medium-power embedded platform, requiring the end-to-end inference cycle of the mapping link to be no more than 50ms; otherwise, it will significantly affect the smoothness and safety margin of parking control.

[0105] To verify the feasibility and effectiveness of the method of this invention, a sample set for training and testing was constructed before implementation. The sample set consisted of multi-view synchronous image sequences and corresponding 3D occupancy annotation results collected under multiple parking conditions, all using unified calibration parameters and unified voxel annotation specifications. The training set comprised 72,000 frames, the validation set 12,000 frames, and the test set 12,000 frames. Statistically, based on target instances, the training set included 46,320 pillar boundary samples, 12,840 parking lock and limiter samples, 61,200 parking space line corner point samples, 135,600 parked vehicle outline samples, 118,400 wall connection boundary samples, and 7,360 suspended object samples. All samples were uniformly projected onto a bird's-eye view grid with a voxel resolution of 0.10m horizontally, 0.10m vertically, and 0.15m high. The height discrete layer was 16 layers. The occupied categories included vacant areas, walls, columns, parked vehicles, parking lines, ground locks / limiters, and hanging objects, totaling 7 categories.

[0106] In a continuous parking test, the vehicle enters the parking area from the entrance. Approximately 6.8m ahead is a pillar, and about 2.1m to the right of the pillar is a parking lock. There is a corner point of the parking space line on the left edge of the parking space, and a suspended pipe approximately 2.15m high is located above and in front of the vehicle. The system first receives a sequence of eight corrected multi-view images of the parking environment. After time synchronization of the current consecutive frames, the residual time deviation is controlled within 2.1ms. After internal and external distortion correction, the average reprojection error of the multi-camera system is 0.43 pixels, and the maximum is 0.78 pixels. At this time, the vehicle's longitudinal speed is 1.72m / s, the longitudinal acceleration is -0.28m / s², and the yaw rate is 0.11rad / s. According to the resolution adjustment logic in claim 2, the system calculates the current frame resolution adjustment intensity to be 0.58, corresponding to a resolution adjustment coefficient of 0.84. Assuming a baseline input height of 576 pixels and a baseline input width of 960 pixels, the target input height after resampling in the current frame is 483 pixels and the target input width is 806 pixels. Compared to a fixed high-resolution input, the number of input pixels per frame decreases from 552,960 to 389,298, directly reducing the front-end feature extraction burden by approximately 29.6%.

[0107] After resampling, the system inputs eight images into a temporal variable resolution bird's-eye view feature encoder. The current frame outputs a 2D feature map of 202×121 at the first scale layer, 101×61 at the second scale layer, and 51×31 at the third scale layer. The system simultaneously reads features from previous historical frames for temporal 2D feature fusion. The current frame uses features from four previous frames for fusion, with weights of 0.41, 0.27, 0.19, and 0.13, respectively. The current frame's planar displacement relative to the previous keyframe is 0.74m, and its heading angle difference is 0.086rad, both exceeding the preset keyframe thresholds; therefore, the current frame is written to the keyframe buffer. After view transformation and keyframe alignment, a bird's-eye view temporal feature map set is obtained, covering a 20m×20m area around the vehicle, comprising 200×200 bird's-eye view grids.

[0108] When applying the improved FlashOcc channel-height folding mechanism to the bird's-eye view temporal feature map set, the system first calculates the parking direction sensitivity coefficient at each bird's-eye view grid position. Near the corner of the parking space line, the lateral local feature variation is 13.6, and the vertical local feature variation is 4.1, corresponding to a parking direction sensitivity coefficient of 0.768. In the pillar edge region, the lateral local feature variation is 9.8, and the vertical local feature variation is 8.9, corresponding to a parking direction sensitivity coefficient of 0.513, indicating that this region has obvious boundary features in both directions. The system sets the number of height discrete layers to 16 and the number of occupancy classes to 7, thus expanding the number of channels to 112. After lateral and vertical two-dimensional convolutional channel expansion and fusion according to the direction sensitivity coefficient, an expanded channel feature map adaptive to parking direction is obtained. Then, category-height coupling modulation is performed by combining the height priors for each occupation category. Specifically, the height center prior value for the ground lock category is set to 0.18m, and the height discrete prior value is set to 0.10m; the height center prior value for the parking line category is set to 0.02m, and the height discrete prior value is set to 0.04m; the height center prior value for the post category is set to 0.95m, and the height discrete prior value is set to 0.65m; and the height center prior value for the suspended object category is set to 2.10m, and the height discrete prior value is set to 0.22m. In the resulting coarse-grained 3D occupation results, at a voxel location approximately 2.1m to the right of the vehicle and 0.15m above the ground, the ground lock category occupancy value is 0.57, the free area occupancy value is 0.26, and the parking line occupancy value is 0.09. At a voxel location near the edge of a post, the post category occupancy value is 0.44, the free area occupancy value is 0.39, and the parked vehicle occupancy value is 0.08, indicating significant boundary ambiguity at this location.

[0109] The system performs uncertainty assessment on the FlashOcc coarse-grained 3D occupancy tensor. The current frame has a total of 200 × 200 × 16 = 640,000 voxels. First, parking sensitivity category weights are set based on key targets in the parking environment: pillars 1.00, walls 0.92, parked vehicles 0.83, parking lines 0.95, parking locks / limiters 1.00, suspended objects 0.71, and vacant areas 0.45. Based on the probability distribution and category distribution entropy of the parking sensitivity categories, the system calculates a parking sensitivity entropy of 0.74, a category probability separation of 0.11, and a category distribution uncertainty of 0.79 at the front right parking lock boundary; and a parking sensitivity entropy of 0.68, a category probability separation of 0.05, and a category distribution uncertainty of 0.82 at the pillar boundary. The system then calculates the 3D spatial gradient. At the ground lock boundary, the parking-sensitive spatial gradient is 0.31 in the horizontal direction along the bird's-eye view, 0.27 in the vertical direction, and 0.18 in the height direction, resulting in a fused parking-sensitive spatial gradient of 0.26. At the pillar boundary, the parking-sensitive spatial gradients in the three directions are 0.29, 0.32, and 0.21, respectively, resulting in a fused gradient of 0.28. The category distribution uncertainty is further fused with the parking-sensitive spatial gradient to obtain the boundary confidence. After full-space filtering, the current frame yields 31,846 boundary voxels. Since the maximum rendering point quota for the vehicle is set to 12,000, the system sorts the boundary voxels from low to high confidence, retaining the 12,000 most uncertain boundary voxels, while the remaining boundary voxels do not enter the PointRend reconstruction link.

[0110] For these 12,000 boundary voxels, the system enters the PointRend adaptive point sampling stage. Taking a valid boundary voxel of the right-side parking lock boundary as an example, its parking environment physical coordinates are (1.86, 4.32, 0.15), the calculated local distribution density is 0.81, and the spatial location priority value is 0.73. After weighting the two, the PointRend sampling priority value is 0.78. Taking another valid boundary voxel of the pillar boundary as an example, its physical coordinates are (3.42, 6.75, 0.60), the local distribution density is 0.67, and the spatial location priority value is 0.58, resulting in a PointRend sampling priority value of 0.63. The system sorts all valid boundary voxels according to their sampling priority values ​​and selects the first 4,096 as the PointRend sparse sampling voxel set. For the parking lock boundary voxels, the maximum number of resampling points is set to 8, so the number of resampling points calculated based on the voxel distribution density is 6; for the pillar boundary voxels, the number of resampling points is 5. A total of 21,534 resampling points were generated from all sparsely sampled voxels. For each resampling point, the system generated physical coordinates in a local three-dimensional neighborhood with the voxel center as the core according to a preset offset template. In Example 2, the physical coordinates of a certain resampling point on the boundary of the ground lock are (1.91, 4.27, 0.18) and another resampling point is (1.84, 4.36, 0.12).

[0111] The system utilizes the 3D coordinates of resampled points to back-index the bird's-eye view temporal feature map set and the original multi-scale image features. Taking the resampled point with physical coordinates (1.91, 4.27, 0.18) as an example, the bird's-eye view back-index coordinates at the first scale layer are (119.1, 142.7), and the bird's-eye view back-index coordinates at the second scale layer are (59.55, 71.35). After bilinear interpolation, the high-frequency feature vectors of the bird's-eye view at the two scale layers are obtained. After concatenation, the channel dimension is 160, and then it is compressed to 96 dimensions through linear projection. For the same resampling point, the system simultaneously performs reverse indexing of the high-frequency features of the original image on three visible cameras. The two-dimensional projection coordinates of one of the forward-looking cameras are (684.3, 392.6), the coordinates of another side-facing camera are (271.4, 446.8), and the projection point of the third camera falls outside the effective range of the feature map, outputting a zero feature vector as an invalid placeholder. After aggregating the features extracted from all effective cameras and all scale layers, a 128-dimensional high-frequency feature vector of the original image is obtained. The high-frequency feature vector of the bird's-eye view is combined with the high-frequency feature vector of the original image to form a set of high-frequency feature information for the corresponding resampling point.

[0112] After entering the PointRend dual-path multilayer perceptron rendering stage, the system aggregates the high-frequency features of all resampled points belonging to the same boundary voxel. For the aforementioned ground lock boundary voxel, the high-frequency features of its 6 resampled points at 3 scale layers are first stitched together, then reduced to a fixed hidden layer dimension through linear projection, and weighted summed according to the resampled distance weight to obtain a 96-dimensional high-frequency aggregated feature vector of the bird's-eye view. At the same time, the high-frequency features of the original image are aggregated on multiple cameras and multiple scale layers to obtain a 128-dimensional high-frequency aggregated feature vector of the original image. The spatial position encoding vector of the boundary voxel is ([0.31,0.72,0.18]), and the occupancy value of the ground lock category in its coarse-grained occupancy prior vector is 0.57, the occupancy value of the idle category is 0.26, and the occupancy value of the parking line category is 0.09. The system inputs the high-frequency aggregated feature vector of the bird's-eye view, the spatial position encoding vector, and the coarse-grained occupancy prior vector into the spatial feature fusion path, and inputs the high-frequency aggregated feature vector of the original image into the appearance feature fusion path. For the ground lock boundary voxel, the dual-path fusion gating coefficient is 0.43, indicating that the final judgment of this voxel depends more on appearance details. After classification, its fine-grained boundary occupancy probability in each occupancy category is updated as follows: 0.11 for free area, 0.83 for ground lock / limiter, 0.03 for parking line, and 0.03 for other categories in total. For the pillar boundary voxel, the dual-path fusion gating coefficient is 0.61, which is more biased towards spatial pathways. The updated fine-grained boundary occupancy probability is 0.79 for pillar, 0.14 for free area, 0.04 for parked vehicle, and 0.03 for other categories in total. The system calculates the PointRend update weight based on the boundary confidence. In Example 2, the boundary confidence of the ground lock boundary voxel is 0.28, so the update weight is 0.72; the boundary confidence of the pillar boundary voxel is 0.34, so the update weight is 0.66. After the update, voxels with ambiguous boundaries in the coarse-grained tensor are replaced with more defined fine-grained boundary occupancy values, ultimately generating the current frame's PointRend fine-grained 3D occupancy tensor.

[0113] After the mapping result of the current frame is generated, the system continues to perform spatiotemporal consistency fusion and morphological optimization. The lateral displacement between the previous frame and the current frame is 0.27m, the longitudinal displacement is 0.05m, and the heading angle change is 0.029rad. Based on this, the system aligns the PointRend fine-grained 3D occupancy tensor of the previous frame to the current coordinate system and performs voting fusion at the voxel level. The voting weight of the current frame is 0.58, and the voting weight of the alignment result of the previous frame is 0.42. Taking a voxel on the boundary of a foreground pillar as an example, the pillar occupancy value in the current frame is 0.79, the aligned pillar occupancy value in the previous frame is 0.73, and the fused value is 0.76; the voxel on the boundary of a groundlock on the front right side has a groundlock occupancy value of 0.83 in the current frame, the aligned value in the previous frame is 0.69, and the fused value is 0.77. After completing the temporal fusion, the system performs spatial consistency constraints and morphological optimization on all voxels. Before spatial consistency processing, there were 1426 isolated voxel clusters in the tensor; after processing, this number was reduced to 263. After 3D neighborhood morphology optimization, the number of burr voxels near the corner of the parking space line was reduced by 81.2%, the number of serrated layers at the top edge of the parking lock was reduced from 4 layers to 1 layer, and the average radius error of the column outline decreased from 0.086m to 0.037m. In the final 3D occupancy map of the parking environment, the front right parking lock was stably mapped as a low obstacle area of ​​approximately 0.51m × 0.17m × 0.16m, the column was mapped as a near-cylindrical high occupancy area, the corner of the parking space line was clearly closed, and the suspension pipe was stably preserved in the high-level voxels, without causing misjudgment of the ground passage space.

[0114] To demonstrate the effectiveness of the method of this invention, it was compared with the traditional 3D convolutional occupancy network method and the traditional low-resolution 2D BEV mapping method on the same test sample set. The traditional 3D convolutional occupancy network method uses dense 3D convolutional inference, while the traditional low-resolution 2D BEV mapping method only performs 2D bird's-eye view mapping without boundary point rendering and repair. The comparison results are shown in Table 1 below: Table 1. Comparison data between the present invention and traditional 3D convolutional occupancy network methods and traditional low-resolution 2D BEV mapping methods. Average inference frame rate / FPS 7.4 19.2 23.8 Average latency per frame (ms) 135.1 52.4 41.7 Peak memory usage / GB 5.42 1.73 1.91 Overall 3D IoU / % 69.8 63.5 74.6 Boundary Chamfer error / m 0.121 0.184 0.057 Average error of column profile / m 0.084 0.149 0.037 Ground lock recognition recall rate / % 71.6 56.8 92.4 Parking space line corner positioning error / m 0.108 0.161 0.043 False detection rate of suspended objects / % 8.9 13.7 4.8 Average number of trajectory corrections for continuous parking / times 3.1 4.4 1.2 Further analysis of key local target results from 12,000 test frames revealed that in the traditional 3D convolutional occupancy network method, 214 parking lock samples were partially absorbed into the ground background, and 57 pillar edges showed significant outward expansion. In the traditional low-resolution 2D BEV mapping method, 382 parking line corner point samples failed to form closed corner point representations, and 91 low obstacles retained only a single layer of blurred response. In contrast, the method of this invention achieved a 90.7% complete recovery rate for parking lock targets, a 93.1% closure rate for parking line corner points, and only 18 samples with pillar edge outward expansion errors greater than 0.10m. When reviewing the results of a single continuous parking process, the traditional low-resolution 2D BEV mapping method showed an outward drift of 0.19m for the obstacle boundary output at the front right parking lock position, resulting in an additional 0.31m leftward correction to the planned trajectory. The method of this invention showed a boundary drift of only 0.04m at the same location, with only a necessary 0.08m avoidance correction to the planned trajectory.

[0115] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. This invention discloses a parking environment mapping method based on computer vision, belonging to the field of parking environment mapping technology. It obtains a corrected multi-view image sequence of the parking environment; generates a set of temporal feature maps of the bird's-eye view in the bird's-eye view coordinate system using a keyframe caching mechanism; generates a FlashOcc coarse-grained 3D occupancy tensor; obtains a set of boundary voxels; extracts the corresponding high-frequency feature information set; generates fine-grained boundary occupancy probabilities, obtaining a PointRend fine-grained 3D occupancy tensor; obtains a spatiotemporally consistent 3D occupancy tensor, and generates the final 3D occupancy map of the parking environment. This invention effectively eliminates the jagged effect and boundary blurring problems, making the geometric contours of key structures in the parking environment smoother and more consistent with real physical dimensions, improving the usability and safety of the mapping results in the planning and control system.

2. The parking environment mapping method based on computer vision according to claim 1, characterized in that, The preprocessing includes time synchronization, intrinsic and extrinsic parameter correction, and distortion correction.

3. The parking environment mapping method based on computer vision according to claim 1, characterized in that, The step of inputting the corrected parking environment multi-view image sequence into a time-varying resolution bird's-eye view feature encoder includes: Based on the corrected multi-view image sequence of the parking environment, the resolution adjustment intensity corresponding to the t-th frame is calculated. The resolution adjustment intensity is then subjected to nonlinear normalization mapping with the preset minimum resolution adjustment coefficient and the preset maximum resolution adjustment coefficient to obtain the resolution adjustment coefficient corresponding to the t-th frame. Based on the resolution adjustment coefficient, the corrected multi-view images of the parking environment corresponding to each camera are resampled to obtain the number of pixels for target input height and the number of pixels for target input width. A spatial interpolation algorithm is used to resample the corrected multi-view images of the parking environment to obtain the resampled corrected multi-view images of the parking environment. The images are then input into a temporally variable resolution bird's-eye view feature encoder. Within the spatial range defined by the number of pixels in the target input height and the number of pixels in the target input width, the feature extraction network extracts the features of each spatial location to generate a set of multi-scale two-dimensional feature maps corresponding to the number of pixels in the target input height and the number of pixels in the target input width. Temporal two-dimensional feature fusion processing is performed on the multi-scale two-dimensional feature map set to obtain a temporal two-dimensional feature map set; Mapping the temporal two-dimensional feature map set to the bird's-eye view coordinate system yields the current frame's bird's-eye view feature map set; Compare the resolution adjustment intensity with the resolution adjustment intensity threshold, the Euclidean distance between the current frame and the previous keyframe in the planar position with the planar displacement threshold, and the difference in heading angle between the current frame and the previous keyframe with the heading angle change threshold. When any of these conditions are met, the current frame is determined to be a keyframe. When the current frame is determined to be a key frame, the bird's-eye view feature map set of the current frame and the current frame index are written together into the key frame cache to form a key frame cache set; Align the bird's-eye view feature map sets of each key frame in the key frame cache set to the bird's-eye view coordinate system of the current frame, and merge them with the bird's-eye view feature map set of the current frame to obtain the bird's-eye view temporal feature map set.

4. The parking environment mapping method based on computer vision according to claim 1, characterized in that, The step of calling the improved FlashOcc channel-height folding mechanism on the bird's-eye view temporal feature map set to rearrange the channel dimensions into height dimensions includes: Based on the differences in local feature changes of each bird's-eye view grid position in the horizontal and vertical extension directions in the bird's-eye view time series feature map, the parking direction sensitivity coefficient corresponding to each bird's-eye view grid position is calculated. The target bird's-eye view temporal feature map is expanded by two-dimensional convolutional channels in the horizontal and vertical directions. The results of the two-dimensional convolutional channel expansion are then weighted and fused according to the parking direction sensitivity coefficient to obtain an expanded channel feature map that is adaptive to the parking direction. Based on the prior physical distribution of each occupancy category in the height direction in the parking environment, the category-height coupling modulation processing is performed on the extended channel feature map that is adaptive to the parking direction to obtain the category-height coupling extended feature set; Perform adjacent height discrete layer coupling folding processing on the category-height coupled extended feature set, rearrange the channel dimension into the height dimension within the occupied category dimension, and generate a FlashOcc coarse-grained three-dimensional occupied log tensor with adjacent height association constraints; Based on the continuity differences of each occupancy category in the height direction in the parking environment, the category continuity correction process is performed on the FlashOcc coarse-grained three-dimensional occupancy log tensor with adjacent height association constraints to obtain the continuity-corrected FlashOcc coarse-grained three-dimensional occupancy log tensor. Normalization is performed on the occupancy class dimension of the continuous correction FlashOcc coarse-grained 3D occupancy logarithm tensor to generate the FlashOcc coarse-grained 3D occupancy tensor.

5. The parking environment mapping method based on computer vision according to claim 1, characterized in that, The uncertainty assessment of the FlashOcc coarse-grained three-dimensional occupancy tensor includes: Based on the class probability distribution of each voxel in the FlashOcc coarse-grained 3D occupancy tensor along the occupancy class dimension, a parking-sensitive class probability distribution for key targets in the parking environment is constructed, and the class distribution uncertainty corresponding to each voxel is calculated based on the parking-sensitive class probability distribution. Based on the local probability changes of the FlashOcc coarse-grained 3D occupancy tensor in the horizontal, vertical, and height directions of the bird's-eye view, the parking-sensitive spatial gradient corresponding to each voxel is calculated. The uncertainty of category distribution is fused with the parking-sensitive spatial gradient to obtain the boundary confidence of each voxel; Voxels in the FlashOcc coarse-grained 3D occupancy tensor are filtered based on boundary confidence to obtain boundary voxel indication results, and boundary voxel sets are extracted based on the boundary voxel indication results.

6. The parking environment mapping method based on computer vision according to claim 1, characterized in that, The step of inputting the boundary voxel set into the PointRend adaptive point sampling module and performing sparse sampling and resampling according to the voxel distribution density and spatial location includes: Calculate the PointRend sampling priority value for each boundary voxel based on its local distribution density and spatial location in the boundary voxel set; Sparse sampling is performed on the boundary voxel set according to the PointRend sampling priority value to obtain the PointRend sparse sampling voxel set; Resampling is performed based on the voxel distribution density and spatial location of each sparse sampled voxel in the PointRend sparse sampled voxel set to obtain the PointRend resampling point set; Using the three-dimensional coordinates in the PointRend resampling point set as a reverse index, the bird's-eye view temporal feature map set is extracted to extract the corresponding high-frequency feature vectors of the bird's-eye view. The original multi-scale image features are extracted by using the three-dimensional coordinates in the PointRend resampling point set to inversely index the original multi-scale image features; The high-frequency feature vectors from the bird's-eye view and the high-frequency features of the original image are summarized to obtain the high-frequency feature information set of the corresponding boundary voxel set.

7. The parking environment mapping method based on computer vision according to claim 1, characterized in that, The step of inputting the high-frequency feature information set into the PointRend dual-path multilayer perceptron rendering module, and performing spatial feature fusion and appearance feature fusion prediction on each boundary voxel, includes: The high-frequency features of all resampled points belonging to the same boundary voxel in the high-frequency feature information set are aggregated to obtain the bird's-eye view high-frequency aggregated feature vector and the original image high-frequency aggregated feature vector corresponding to each boundary voxel. The high-frequency aggregated feature vector of the bird's-eye view is input into the spatial feature fusion path in the PointRend dual-path multilayer perceptron rendering module, and the high-frequency aggregated feature vector of the original image is input into the appearance feature fusion path in the PointRend dual-path multilayer perceptron rendering module to obtain the spatial fusion feature vector and appearance fusion feature vector corresponding to each boundary voxel. Perform dual-path fusion prediction on the spatial fusion feature vector and the appearance fusion feature vector to obtain the fine-grained boundary occupancy probability of each boundary voxel in each occupancy category; The occupancy value of the corresponding boundary voxel position in the FlashOcc coarse-grained 3D occupancy tensor is updated based on the fine-grained boundary occupancy probability to obtain the PointRend fine-grained 3D occupancy tensor.

8. The parking environment mapping method based on computer vision according to claim 1, characterized in that, The spatiotemporal consistency fusion of the PointRend fine-grained 3D occupancy tensor involves aligning the previous frame's 3D occupancy tensor to the current coordinate system after vehicle pose compensation, and performing voxel-level voting fusion with the current frame's PointRend fine-grained 3D occupancy tensor, including: Align the PointRend fine-grained 3D occupancy tensor of the previous frame to the coordinate system of the current frame to obtain the pose-compensated aligned 3D occupancy tensor of the previous frame. The previous frame aligned 3D occupancy tensor after attitude compensation is fused with the current frame PointRend fine-grained 3D occupancy tensor at the voxel level to obtain a spatiotemporally consistent 3D occupancy tensor. Perform morphological optimization on the spatiotemporally consistent 3D occupancy tensor to obtain the morphologically optimized 3D occupancy tensor of the parking environment. The final 3D occupancy map of the parking environment is generated based on the morphologically optimized 3D occupancy tensor of the parking environment.

9. A computer vision-based parking environment mapping system, used to execute the computer vision-based parking environment mapping method according to any one of claims 1-8, characterized in that, include: The data acquisition and preprocessing module is used to acquire and preprocess multi-view image sequences of the parking environment to obtain corrected multi-view image sequences of the parking environment. The temporal bird's-eye view feature encoding module is used to input the corrected multi-view image sequence of the parking environment into the temporal variable resolution bird's-eye view feature encoder to generate a set of temporal feature maps of bird's-eye view. The FlashOcc 3D occupancy generation module is used to call the improved FlashOcc channel-height folding mechanism on the bird's-eye view temporal feature map set to rearrange the channel dimension into the height dimension and generate the FlashOcc coarse-grained 3D occupancy tensor. The boundary voxel extraction module is used to perform uncertainty assessment on the FlashOcc coarse-grained 3D occupancy tensor to obtain the boundary voxel set; The PointRend point sampling feature extraction module is used to input the boundary voxel set into the PointRend adaptive point sampling module to extract the corresponding high-frequency feature information set. The PointRend boundary rendering module is used to input the high-frequency feature information set into the PointRend dual-path multilayer perceptron rendering module to obtain the PointRend fine-grained 3D occupancy tensor. The spatiotemporal fusion and map generation module is used to perform spatiotemporal consistency fusion on the PointRend fine-grained 3D occupancy tensor to obtain a spatiotemporally consistent 3D occupancy tensor, and perform morphological optimization processing on the spatiotemporally consistent 3D occupancy tensor to generate the final 3D occupancy map of the parking environment.

Images