Vehicle passable area detection method, storage medium, electronic device, and vehicle

Abstract

The application relates to a vehicle passable area detection method, a storage medium, an electronic device and a vehicle. The method comprises the following steps: extracting initial image features related to a passable area, and generating three-dimensional position coding perception image features; performing cross-attention interaction between a preset number of passable boundary point Query feature vectors and the three-dimensional position coding perception image features, so as to optimize the passable boundary point Query feature vectors; and performing position regression on the optimized passable boundary point Query feature vectors, so as to obtain boundary point coordinates of the passable area. Based on the method, high-precision and low-computing-cost passable area detection can be realized in an intelligent driving scene, and the boundary positioning precision and the system operation efficiency are effectively improved.

Description

Technical Field

[0001] This application relates to the field of automotive technology, and in particular to a method for detecting vehicle passable areas, a storage medium, an electronic device, and a vehicle. Background Technology

[0002] In intelligent driving environmental perception tasks, passable area detection is the core of vehicle autonomous obstacle avoidance and safe driving. Its accuracy, generalization ability and inference efficiency directly determine the reliability of autonomous driving systems.

[0003] Traditional object detection is limited by fixed categories and cannot cover unknown obstacles and complex road conditions. Visible area detection based on visual images has become the mainstream approach. Existing solutions fall into two categories: one is based on 2D image semantic segmentation or boundary detection, which is then projected onto a bird's-eye view space through inverse perspective transformation. This relies on the ground plane assumption or additional depth prediction, resulting in large projection errors and complex multi-camera fusion. The other approach directly performs dense segmentation in the bird's-eye view space, requiring post-processing such as ray casting to extract boundaries. This approach suffers from high computational overhead, accuracy limitations due to segmentation image resolution, and complex post-processing, making it difficult to meet the low-latency, high-precision deployment requirements of automotive platforms. Summary of the Invention

[0004] This application provides a method for detecting vehicle passable areas, a storage medium, an electronic device, and a vehicle, which can solve at least one technical problem in the prior art.

[0005] Accordingly, this application provides a method for detecting a passable area of ​​a vehicle. The method includes: extracting initial image features related to the passable area and generating three-dimensional position-coded perception image features; performing cross-attention interaction between a preset number of passable boundary point Query feature vectors and the three-dimensional position-coded perception image features to optimize the passable boundary point Query feature vectors; and performing position regression on the optimized passable boundary point Query feature vectors to obtain the boundary point coordinates of the passable area.

[0006] In one embodiment of this application, generating three-dimensional position-coded perceptual image features includes: fusing the initial image features with three-dimensional position coding to obtain the three-dimensional position-coded perceptual image features.

[0007] In one embodiment of this application, the detection method further includes: inputting the initial image features into a depth estimation network, and performing joint supervised training on the depth estimation network using cross-entropy classification loss and scale-invariant log loss to obtain depth-enhanced image features.

[0008] In one embodiment of this application, the step of performing cross-attention interaction between a preset number of passable boundary point Query feature vectors and the three-dimensional position-encoded perception image features includes: performing polar coordinate position encoding on the passable boundary point Query feature vectors; and performing cross-attention interaction between the encoded passable boundary point Query feature vectors and the three-dimensional position-encoded perception image features.

[0009] In one embodiment of this application, the cross-attention interaction adopts multi-level iteration. The multi-level iterative interaction corrects the passable boundary point Query feature vector, so that the passable boundary point Query feature vector corresponds to the semantic features and three-dimensional spatial location features of the boundary point of the passable area.

[0010] In one embodiment of this application, the preset number of passable boundary point Query feature vectors are obtained by: performing ray preprocessing on the ground value of the passable region to extract the real boundary points; and determining the preset number of passable boundary point Query feature vectors based on the real boundary points.

[0011] In one embodiment of this application, the preset number of passable boundary point Query feature vectors are sparse query vectors, and their number is less than the number of dense grid query vectors.

[0012] Accordingly, embodiments of this application provide a computer-readable storage medium including a computer program, which, when run on a computer device, causes the computer device to perform the passable area detection method described in any of the above claims.

[0013] Accordingly, this application provides an electronic device, including: a memory storing a computer program thereon; and a processor for executing the computer program in the memory to implement the passable area detection method described above.

[0014] Accordingly, embodiments of this application provide a vehicle, including: the electronic device described above; or, a processor, the processor being used to execute the passable area detection method described above.

[0015] This application provides a method, storage medium, electronic device, and vehicle for detecting passable areas of a vehicle. By optimizing the cross-attention interaction between three-dimensional position encoding perception image features and sparse passable boundary point queries, and directly performing position regression on the optimized query features, the method can reduce the overhead of dense computation and post-processing while enabling the model to accurately focus on learning the boundaries of passable areas, effectively improving the boundary point localization accuracy, and achieving end-to-end, lightweight, and high-precision detection of passable areas of a vehicle. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating one implementation method of the passable area detection method of this application;

[0018] Figure 2 This is a comparison of the supervision effect of the depth estimation branch in the embodiments of this application;

[0019] Figure 3 This is a flowchart illustrating one embodiment of step S200 of this application;

[0020] Figure 4 This is a comparative diagram of the output results of sparse instance boundary points in this application and the output results of dense segmentation schemes in the prior art;

[0021] Figure 5 This is a schematic diagram of the sparse boundary point regression scheme of this application;

[0022] Figure 6 This is a schematic diagram of a dense segmentation scheme in the prior art;

[0023] Figure 7 This is a schematic diagram of the structure of the electronic device of this application. Detailed Implementation

[0024] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be noted that the following embodiments are merely illustrative of the present application and do not limit its scope. Similarly, the following embodiments are only some, not all, embodiments of the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0025] It should be understood that the terms "upper," "lower," "left," "right," "front," "back," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or relative positional relationship shown in the accompanying drawings. They are used solely for the convenience of describing this application and for simplification, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Unless otherwise specified, the above-mentioned orientational descriptions can be flexibly set in practical applications, provided that the relative positional relationships shown in the accompanying drawings are satisfied.

[0026] The terms "first" and "second" are configured for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0027] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," and "communication" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection. They can refer to a direct connection or an indirect connection through an intermediate medium, or a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0028] In embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, article, or apparatus that includes that element.

[0029] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0030] The following is a detailed analysis of the proposed solution with reference to the accompanying drawings:

[0031] Please see Figure 1 , Figure 1 This is a flowchart illustrating one implementation method of the vehicle passable area detection method of this application, as shown below. Figure 1 As shown, the detection method provided in this application includes the following steps:

[0032] S100: Extract initial image features related to the passable area and generate three-dimensional position-coded perceptual image features.

[0033] First, two-dimensional images of the road scene captured by an onboard multi-view camera are obtained. These images are then input into a backbone feature extraction network composed of ResNet (Residual Network). The backbone network extracts features from the input image from shallow to deep through multi-layer convolution, pooling, and residual connections. Shallow layers capture low-level visual information such as edges, textures, and colors, while deep layers further abstract high-level semantic information such as target semantics and scene structure. The final output is a two-dimensional image feature map with a fixed number of channels and spatial size (containing basic information such as road surface, obstacles, and background), which represents the initial image features related to the passable area.

[0034] Furthermore, the initial image features serve as shared basic features for the subsequent depth estimation branch and the 3D spatial location encoding branch, ensuring that the two branches are processed on the same semantic features, thus avoiding spatial positioning deviations caused by inconsistent features.

[0035] Furthermore, after obtaining the initial image features, three-dimensional position-coded perceptual image features can be generated based on the initial image features, as described in detail below:

[0036] First, based on the camera's intrinsic and extrinsic parameters and the preset BEV (Bird's EyeView) spatial range, a corresponding 3D position code (3DPE) is generated for each spatial location on the initial image feature map. This code represents the geometric positional relationship of the initial image features in 3D space (such as distance from the vehicle and left / right offset). Further, the generated 3D position code is added element-wise to the initial image features in either the channel dimension or the spatial dimension, enabling the original image features to acquire explicit 3D spatial position awareness. This results in 3D position-aware image features with semantic, depth, and spatial information. In other words, the final features not only contain image semantics but also clearly correspond to the 3D spatial position of each semantic region, essentially adding spatial coordinate labels to the image features. This feature serves as a key feature for subsequent cross-attention interactions, providing accurate spatial priors for boundary point spatial regression.

[0037] Furthermore, after obtaining the initial image features, the initial image features are input into the depth estimation network, and the depth estimation network is jointly supervised and trained by cross-entropy classification loss and scale-invariant log loss to obtain depth-enhanced image features.

[0038] The DepthNet network is specifically designed for depth prediction to improve image feature extraction. Through multiple convolutional layers and upsampling operations, DepthNet maps initial image features into a pixel-wise depth prediction map that matches the spatial dimensions of the input image, enabling the prediction of the three-dimensional spatial distance corresponding to each pixel in the image.

[0039] In the specific implementation of this application, during the model training phase, cross-entropy classification loss and SILog regression loss are used to explicitly supervise the depth estimation branch:

[0040] First, continuous depth values ​​are divided into several discrete depth intervals, and cross-entropy loss is used to classify and supervise the depth interval to which each pixel belongs. Simultaneously, SILog regression loss is used to directly supervise the true depth values ​​continuously. In this embodiment, SILog regression loss is scale-invariant, enabling optimization of depth error without relying on absolute scale. This makes the depth map output by the model more accurate at object edges and road surfaces, improving geometric consistency.

[0041] like Figure 2 As shown, Figure 2 This is a comparison of the supervision effect of the depth estimation branch in the embodiments of this application. The depth map obtained by joint supervision using cross-entropy classification loss and SILog regression loss is more continuous in depth distribution, more accurate in edge, and stronger in near and far depth consistency compared with the depth map using only classification loss. It is more in line with the real physical space structure, thus enabling the depth-enhanced image features to have more accurate three-dimensional spatial constraints.

[0042] Understandably, the depth prediction result is not directly used for subsequent feature fusion or interaction. Instead, it enhances the expressive power of the backbone network's image features through backpropagation, enabling the image features to implicitly contain spatial distance and three-dimensional geometric information, thus providing auxiliary constraints for subsequent three-dimensional spatial localization and boundary point regression.

[0043] In the above implementation, the dual-loss joint constraint ensures that the depth estimation results maintain high accuracy in both the near and far regions, the depth distribution is more consistent with the real physical space, and the depth convergence stability is improved.

[0044] S200 performs cross-attention interaction between a preset number of passable boundary point Query feature vectors and 3D position-encoded perception image features to optimize the passable boundary point Query feature vectors.

[0045] Please combine further Figure 3 , Figure 3 This is a flowchart illustrating an implementation method of step S200 of this application, as shown below. Figure 3 Step S200 includes the following sub-steps:

[0046] S210, polar coordinate position encoding is performed on the feature vector of the passable boundary point Query.

[0047] Furthermore, in this embodiment, the traditional dense BEV grid query method is abandoned, and a preset number of sparse boundary point Query feature vectors are used as the learning carriers for the boundaries of passable areas. This can reduce computational overhead and allow the model to focus on the boundary point locations rather than the entire area, laying the groundwork for subsequent accurate regression.

[0048] Understandably, existing dense BEV segmentation schemes require the initial output of dense BEV features. This is typically achieved through cross-attention interaction between dense BEVQuery and image features, or by constructing dense BEV features using a VT transform module, and then connecting these features to a segmentation task head to complete region segmentation. Such schemes output segmentation maps with low resolution, typically 0.4m / pixel in driving scenarios. Subsequent ray casting is required to extract the boundaries of passable areas, but the boundary accuracy is limited by the segmentation map resolution, and the post-processing is complex and computationally expensive.

[0049] Furthermore, the Query feature vectors of the passable boundary points in this application are sparse query vectors, and their number is less than that of dense grid queries. In a specific application scenario of this application, the number of Query feature vectors can be 360, which is far less than the 1024 of traditional dense BEV queries. It only focuses on boundary position learning and does not participate in the calculation of global dense regions, thereby significantly reducing the amount of computation.

[0050] Specifically, the preset number of passable boundary point Query feature vectors can be obtained as follows: Ray preprocessing is performed on the ground values ​​of the passable region to extract the true boundary points. Further, the preset number of passable boundary point Query feature vectors are determined based on these true boundary points.

[0051] Among them, the passable area ground truth is the OCC ground truth. In a specific implementation, the resolution can be 0.15m / pixel. It is obtained based on the input two-dimensional driving image, combined with the scene's real 3D point cloud data and LiDAR scanning results, and is obtained through manual annotation and algorithm calibration. It is used to provide a real boundary basis for the Query feature vector.

[0052] Further integration Figure 4 , Figure 4 This is a comparative diagram showing the output results of sparse instance boundary points in this application and the output results of existing dense segmentation schemes, as shown below. Figure 4 The left side shows the output of the dense segmentation scheme, which relies on low-resolution dense segmentation maps to extract boundaries, resulting in low boundary accuracy and coarse contours. Figure 4 The right side of the middle section shows the output results of the sparse instance boundary points of this invention. A continuous and smooth walkable region outline is obtained by directly regressing the sparse boundary points.

[0053] The sparse boundary point regression scheme employed in this application uses a preset number of sparse boundary point queries to directly interact with image features through cross-attention, and directly regresses the boundary point positions. The OCC ground truth used in this application is obtained through ray casting preprocessing, enabling the boundary point regression accuracy to reach ground truth resolution, significantly higher than the 0.4m / pixel accuracy limit of dense segmentation schemes. Simultaneously, this application drastically reduces the number of BEVQuery queries to approximately one-third of traditional dense schemes, effectively reducing the computational overhead of the Transformer decoder; the model directly outputs the boundary point positions without post-processing such as ray casting, improving the accuracy of passable region detection while simplifying the process and reducing latency.

[0054] Furthermore, before performing cross-attention interaction, the feature vector of the passable boundary point Query is encoded in polar coordinates: with the vehicle center as the origin, the preset boundary position corresponding to each passable boundary point Query is converted into polar coordinate parameters in the form of polar radius and polar angle; where the polar radius represents the radial distance from the boundary point to the vehicle center, and the polar angle represents the angular offset of the boundary point relative to the positive direction of vehicle travel. The polar radius and polar angle are converted into feature vectors through trigonometric function mapping or learnable encoding matrices, and then concatenated with or element-wise added to the feature vector of the passable boundary point Query in the channel dimension, thereby embedding the polar coordinate angle and radial distance information into the feature vector of the passable boundary point Query, enabling the model to perceive the ring distribution characteristics of the boundary points around the vehicle center in the early stages of training.

[0055] After completing the polar coordinate position encoding, the Query feature vector of the passable boundary point carrying the prior polar coordinate space is used as the query input and cross-attention interaction with the 3D position encoded perception image features. Specifically, the encoded passable boundary point Query feature vector is used as the Query part of the cross-attention, and the 3D position encoded perception image features are used as the Key and Value parts of the cross-attention. By calculating the attention weights, the information interaction and feature fusion between the passable boundary point Query feature vector and the 3D position encoded perception image features are realized.

[0056] S220 involves cross-attention interaction between the encoded passable boundary point Query feature vector and the 3D position-encoded sensing image features.

[0057] In this embodiment, the cross-attention interaction adopts a multi-level iterative approach. Through multi-level iterative interaction, the Query feature vector of the passable boundary point is gradually corrected so that the Query feature vector of the passable boundary point corresponds to the semantic features and three-dimensional spatial location features of the boundary point of the passable area.

[0058] Specifically, the sparse boundary point Query feature vector encoded with polar coordinates is input into the Transformer decoder structure. The 3D position-encoded perceptual image features are used as the Key and Value features for cross-attention, and multi-layered continuous cross-attention calculations are performed. During each layer of cross-attention calculation, the currently iterated traversable boundary point Query feature vector is used as the query term to perform global information retrieval of the 3D position-encoded perceptual image features. Through attention weight allocation, each traversable boundary point Query feature vector adaptively aggregates semantic information and 3D spatial location information related to the traversable region boundary from the image features.

[0059] Through iterative optimization of the multi-layer Transformer decoder, the passable boundary point Query feature vector is updated and corrected once in each layer iteration. It gradually extracts and fuses the effective feature information of the corresponding boundary points from the 3D position-encoded perception image features, so that the passable boundary point Query feature vector gradually approaches and matches the feature expression of the real boundary points of the passable area.

[0060] The aforementioned cross-attention interaction process directly completes information exchange and feature fusion between image features and sparse passable boundary point query feature vectors. It does not construct or generate dense BEV feature maps, nor does it perform global feature calculations under dense grids. Attention interaction is only performed between sparse query vectors and image features.

[0061] S300 performs position regression on the optimized passable boundary point Query feature vector to obtain the boundary point coordinates of the passable area.

[0062] Furthermore, a position regression operation is performed on the Query feature vector of the passable boundary points after multi-layer cross-attention iteration optimization. The coordinates of the boundary points of the passable area are obtained through regression calculation, thus completing the accurate positioning of the boundary of the passable area.

[0063] Specifically, the optimized passable boundary point Query feature vectors are input one by one into the preset boundary point regression head. The boundary point regression head is a dedicated output layer structure of the model, whose core is composed of fully connected layers. Alternatively, convolutional layers can be selected to replace fully connected layers according to the actual application scenario to realize the mapping of features to coordinates.

[0064] The specific working process of the regression head is as follows: First, it receives the optimized Query feature vector for each passable boundary point. This feature vector accurately captures the semantic features and three-dimensional spatial location features of the corresponding passable boundary point. Then, through the fully connected layer or convolutional layer inside the regression head, the Query feature vector is subjected to dimensional transformation and feature parsing, mapping the abstract feature vector to specific continuous numerical coordinates, and directly outputting the boundary point coordinate value corresponding to each passable boundary point Query feature vector. The coordinate values ​​can be output to the BEV space (bird's-eye view space) or the vehicle coordinate system according to actual needs. The vehicle coordinate system takes the center of the vehicle as the origin, the X-axis along the positive direction of vehicle travel, the Y-axis perpendicular to the direction of travel, and the Z-axis perpendicular to the ground, ensuring that the output coordinates can accurately represent the spatial position relationship of the boundary point relative to the vehicle, truly realizing the direct mapping from the passable boundary point Query feature vector to the boundary point spatial coordinates without any intermediate feature transformation steps.

[0065] The final output of the model is a set of sparse boundary point coordinates arranged in a preset order. This preset order matches the circular distribution characteristics of the passable area boundary, corresponding to the clockwise or counterclockwise arrangement of the outer contour of the passable area. By sequentially connecting the coordinates of adjacent boundary points in this ordered set, a complete and continuous outer contour of the passable area can be formed, clearly defining the passable range for vehicles.

[0066] It is important to clarify that in the embodiment of this application, during the output of boundary point coordinates, the coordinate values ​​are directly output from the optimized Query feature vector through the boundary point regression head. There is no need to generate a dense BEV segmentation map (i.e., no dense mesh feature modeling of the entire area surrounding the vehicle). Furthermore, no post-processing operations required in traditional methods are performed, including but not limited to complex operations such as ray casting, edge extraction, morphological dilation and erosion, and contour smoothing. This completely eliminates the time-consuming, cumbersome, and error-prone post-processing steps in traditional passable area detection methods. The output passable area boundary point coordinates can be directly integrated into the planning and control system of the autonomous driving system without additional coordinate transformation or format adaptation. This directly provides accurate passable area location information for downstream tasks such as obstacle avoidance decisions, path generation, and driving area constraints, ultimately completing the entire process of detecting the vehicle's passable area. Please refer to the figure for further details. Figure 5 and Figure 6 , Figure 5 This is a schematic diagram of the sparse boundary point regression scheme of this application. Figure 6 This is a schematic diagram of a dense segmentation scheme in the prior art. This application adopts a sparse boundary point regression scheme, which directly interacts with image features through a preset number of sparse boundary point queries, and the boundary point coordinates are directly output by the regression head, without the need for dense BEV feature construction and post-processing operations.

[0067] In the above implementation, by optimizing the cross-attention interaction between three-dimensional position encoding perceived image features and sparse passable boundary point Query, and directly performing position regression on the optimized Query features, the model can accurately focus on learning the boundaries of passable areas while reducing the overhead of dense computation and post-processing, effectively improving the boundary point positioning accuracy, and achieving end-to-end, lightweight, and high-precision vehicle passable area detection.

[0068] This application also provides an electronic device, such as... Figure 7 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0069] The electronic device may include components such as a processor 301 with one or more processing cores, a memory 302 with one or more storage media, a power supply 303, and an input unit 304. Those skilled in the art will understand that... Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0070] The processor 301 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines, and performs various functions and processes data by running or executing computer programs and / or modules stored in the memory 302, and by calling data stored in the memory 302. Optionally, the processor 301 may include one or more processing cores; optionally, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 301.

[0071] The memory 302 can be used to store computer programs and modules. The processor 301 executes various functional applications and vehicle control by running the computer programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one computer program required for a function (such as vehicle accessibility detection), etc.; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include memory electronics to provide the processor 301 with access to the memory 302.

[0072] The electronic device also includes a power supply 303 that supplies power to the various components. Optionally, the power supply 303 can be logically connected to the processor 301 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 303 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0073] The electronic device may also include an input unit 304, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0074] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the electronic device loads the executable files corresponding to the processes of one or more computer programs into the memory 302 according to the following instructions, and the processor 301 runs the computer programs stored in the memory 302 to realize various functions, such as:

[0075] Extract initial image features related to the passable area and generate three-dimensional position-coded sensing image features; perform cross-attention interaction between a preset number of passable boundary point Query feature vectors and the three-dimensional position-coded sensing image features to optimize the passable boundary point Query feature vectors; perform position regression on the optimized passable boundary point Query feature vectors to obtain the boundary point coordinates of the passable area.

[0076] Therefore, the electronic device provided in this application embodiment optimizes the cross-attention interaction between three-dimensional position encoding perception image features and sparse passable boundary point Query, and directly performs position regression on the optimized Query features. This can reduce the overhead of dense computation and post-processing, while enabling the model to accurately focus on learning the boundaries of passable areas, effectively improving the boundary point positioning accuracy, and achieving end-to-end, lightweight, and high-precision vehicle passable area detection.

[0077] For details on the specific implementation methods and corresponding beneficial effects of the above operations, please refer to the detailed description of the passable area detection method above, which will not be repeated here.

[0078] Therefore, embodiments of this application provide a computer-readable storage medium storing a computer program that can be loaded by a processor to execute the steps of any of the passable area detection methods provided in embodiments of this application. For example, the computer program can execute the following steps:

[0079] Extract initial image features related to the passable area and generate three-dimensional position-coded sensing image features; perform cross-attention interaction between a preset number of passable boundary point Query feature vectors and the three-dimensional position-coded sensing image features to optimize the passable boundary point Query feature vectors; perform position regression on the optimized passable boundary point Query feature vectors to obtain the boundary point coordinates of the passable area.

[0080] Therefore, the storage medium provided in this application embodiment optimizes the cross-attention interaction between three-dimensional position encoding perceived image features and sparse passable boundary point Query, and directly performs position regression on the optimized Query features. This can reduce the overhead of dense computation and post-processing, while enabling the model to accurately focus on learning the boundaries of passable areas, effectively improving the boundary point positioning accuracy, and achieving end-to-end, lightweight, and high-precision vehicle passable area detection.

[0081] For details on the specific implementation methods and corresponding beneficial effects of the above operations, please refer to the previous embodiments, which will not be repeated here.

[0082] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0083] Since the computer program stored in the storage medium can execute the steps in any of the vehicle passable area detection methods provided in the embodiments of this application, the beneficial effects that any of the vehicle passable area detection methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0084] This application also provides a vehicle that includes the aforementioned electronic device or processor, the processor being used in real time for the passable area detection method in any of the above embodiments.

[0085] The above implementation method optimizes the cross-attention interaction between three-dimensional position encoding perceived image features and sparse passable boundary point queries, and directly performs position regression on the optimized query features. This reduces the overhead of dense computation and post-processing, while enabling the model to accurately focus on learning the boundaries of passable areas, effectively improving the boundary point positioning accuracy, and achieving end-to-end, lightweight, and high-precision vehicle passable area detection.

[0086] The foregoing has provided a detailed description of a vehicle passable area detection method, storage medium, electronic device, and vehicle provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only configured to help understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0087] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for detecting vehicle passable areas, characterized in that, The method includes: Extract initial image features related to the passable area and generate 3D location-coded perceptual image features; A preset number of passable boundary point Query feature vectors are cross-attentionally interacted with the features of the three-dimensional position-encoded perception image to optimize the passable boundary point Query feature vectors. The optimized passable boundary point Query feature vector is used for position regression to obtain the boundary point coordinates of the passable area.

2. The method for detecting passable areas according to claim 1, characterized in that, The generated three-dimensional position-coded perceptual image features include: The initial image features are fused with three-dimensional position coding to obtain the three-dimensional position-coded perceptual image features.

3. The method for detecting passable areas according to claim 1 or 2, characterized in that, The detection method further includes: The initial image features are input into the depth estimation network, and the depth estimation network is jointly supervised and trained using cross-entropy classification loss and scale-invariant log loss to obtain depth-enhanced image features.

4. The method for detecting passable areas according to claim 1, characterized in that, The step of performing cross-attention interaction between the preset number of passable boundary point Query feature vectors and the 3D location-encoded perception image features includes: The Query feature vector of the passable boundary point is encoded in polar coordinates; The encoded passable boundary point Query feature vector is subjected to cross-attention interaction with the 3D location-encoded perceptual image features.

5. The method for detecting passable areas according to claim 4, characterized in that, The cross-attention interaction employs multi-level iteration. Through this multi-level iterative interaction, the passable boundary point Query feature vector is corrected, so that the passable boundary point Query feature vector corresponds to the semantic features and three-dimensional spatial location features of the boundary points of the passable region.

6. The method for detecting passable areas according to claim 1, characterized in that, The preset number of passable boundary point Query feature vectors are obtained through the following method: Ray preprocessing is performed on the true values ​​of the passable region to extract the true boundary points; Based on the actual boundary points, determine the preset number of passable boundary point Query feature vectors.

7. The method for detecting passable areas according to claim 1, characterized in that, The preset number of passable boundary point query feature vectors are sparse query vectors, and their number is less than the number of dense grid query vectors.

8. A computer-readable storage medium, characterized in that, Includes a computer program, which, when run on a computer device, causes the computer device to perform the passable area detection method according to any one of claims 1 to 7.

9. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the passable area detection method according to any one of claims 1-7.

10. A vehicle, characterized in that, The vehicles include: The electronic device as described in claim 9; Alternatively, a processor, the processor being configured to perform the passable area detection method according to any one of claims 1-7.

Images