A parking space detection method and related apparatus

Abstract

The application discloses a parking space detection method and related device, and relates to the technical field of intelligent driving, and comprises the following steps: acquiring a to-be-detected parking space image, generating a bird's-eye view feature map according to the to-be-detected parking space image, initializing semantic identities and initial position representations of a plurality of contour points by using a learnable query vector, sampling local features in the bird's-eye view feature map based on the initial position representations, interacting the local features with the query vector, generating corrected position representations of the plurality of contour points according to the query vector updated in the interaction process, returning the corrected position representations to the step of sampling the local features as the initial position representations, and until a preset iteration termination condition is met, taking the semantic identities and the last iterated corrected position representations of the plurality of contour points as a parking space detection result of the to-be-detected parking space. The application models a parking space edge as a plurality of contour points, and corrects position representations of the plurality of contour points through an iteration process, thereby realizing high-precision modeling of an arbitrary-morphology parking space.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to a parking space detection method and related device. Background Technology

[0002] With economic development and improved living standards, the number of private cars has increased dramatically. To facilitate quick and easy parking, automatic parking technology has emerged. Automatic parking systems use onboard sensors (such as cameras, lidar, and ultrasonic sensors) to perceive the surrounding environment and, combined with advanced control algorithms, enable vehicles to autonomously park and exit. This technology not only reduces the driver's workload and improves parking efficiency but also effectively utilizes limited parking space resources. However, the successful application of automatic parking technology relies heavily on accurate parking space detection technology. Only by accurately identifying available parking spaces can the automatic parking system make the correct parking decisions.

[0003] Current mainstream parking space detection technologies mainly include three categories: rectangular bounding box modeling, corner modeling, and parallel line modeling. Rectangular bounding box modeling treats the parking space as a rectangular target and predicts its bounding box coordinates using a model. Corner modeling models the parking space as a set of corner points (usually the four vertices of the parking space), directly predicts the corner point coordinates, and then connects the corner points to generate the parking space region. Parallel line modeling relies on manually designed line clustering rules and parallel line constraints to detect parallel line pairs in the image of the parking space to be detected, generating the parking space region.

[0004] However, the above-mentioned parking space detection methods can only detect parking spaces with polygonal shapes such as rectangles and parallelograms. Their ability to model parking spaces with arbitrary shapes such as arcs and irregularities is seriously insufficient, which leads to problems such as information loss, parking trajectory planning mismatch, limited scene coverage, and reduced parking safety when the subsequent parking system makes parking decisions. Summary of the Invention

[0005] In view of the above problems, this application provides a parking space detection method and related device to achieve accurate detection of parking spaces of any shape. The specific solution is as follows:

[0006] The first aspect of this application provides a parking space detection method, including:

[0007] Acquire an image of the parking space to be detected, and generate a bird's-eye view feature map based on the image of the parking space to be detected;

[0008] The semantic identities and initial position representations of multiple contour points are initialized using learnable query vectors, and local features are sampled in the bird's-eye view feature map based on the initial position representations. The multiple contour points are arranged sequentially according to the order determined by the semantic identities to form a parking space edge that is connected end to end, so as to model the geometric shape of the parking space to be detected.

[0009] The local features are interacted with the query vector, and the corrected position representation of the multiple contour points is generated based on the query vector updated during the interaction process.

[0010] The step of returning the corrected position representation as the initial position representation to sample local features continues until a preset iteration termination condition is met. Then, the semantic identity of the multiple contour points and the corrected position representation obtained in the last iteration are used as the parking space detection result of the parking space to be detected.

[0011] In one possible implementation, there are multiple learnable query vectors, each of which is used to detect one of the parking spaces to be detected;

[0012] Before interacting with the query vector using the local features, the method further includes:

[0013] Multiple query vectors are subjected to self-attention interaction to obtain multiple initial update query vectors;

[0014] The step of interacting the local features with the query vector includes:

[0015] The local features are interacted with the corresponding initial update query vector.

[0016] In one possible implementation, after interacting with the local features and the query vector, the method further includes:

[0017] The vehicle location confidence score of the parking space to be detected is generated based on the query vector updated during the interaction process. The vehicle location confidence score indicates whether the parking space to be detected is a reliable parking space.

[0018] In one possible implementation, generating a bird's-eye view feature map based on the image of the parking space to be detected includes:

[0019] The image of the parking space to be detected is input into the encoding module of the pre-trained parking space detection model to obtain the bird's-eye view feature map;

[0020] The steps of initializing the semantic identity and initial position representation of multiple contour points using learnable query vectors, sampling local features in the bird's-eye view feature map based on the initial position representations, interacting the local features with the query vectors, generating corrected position representations of the multiple contour points based on the updated query vectors during the interaction process, and returning the corrected position representations as the initial position representations to sample local features, until a preset iteration termination condition is met, include:

[0021] The bird's-eye view feature map is input into the decoding module of the parking space detection model. The decoding module initializes the semantic identity and initial position representation of multiple contour points using a learnable query vector. Based on the initial position representation, local features are sampled in the bird's-eye view feature map. The local features are interacted with the query vector. The revised position representation of the multiple contour points is generated according to the query vector updated during the interaction process. The revised position representation is used as the initial position representation and returned to the step of sampling local features. This process continues until a preset iteration termination condition is met. Finally, the semantic identity of the multiple contour points and the revised position representation obtained in the last iteration are output.

[0022] In one possible implementation, the training process of the parking space detection model includes:

[0023] Obtain parking space image samples;

[0024] According to the preset point marking rules, contour points are marked on the parking space lines of the parking space image sample to obtain the contour point set label corresponding to each parking space in the parking space image sample. The point marking rules include at least one of the following rules: point number and distribution rules for indicating the number and distribution of contour points marked on each parking space edge, point sorting rules for indicating the semantic identity of contour points and the point sorting order determined by the semantic identity of contour points, and point representation rules for indicating the position representation method of contour points.

[0025] The parking space detection model is trained using the parking space image samples and the set of contour points corresponding to each parking space in the parking space image samples.

[0026] In one possible implementation, the number of points and the distribution indicated by the distribution rule are uniform distributions;

[0027] And / or, the point sorting rule indicates that the point sorting order is as follows: all marked contour points are arranged in clockwise order of the parking space edges, with the parking entrance edge as the starting parking space edge for the arrangement;

[0028] And / or, the location representation indicated by the point representation rule is the coordinate representation in the three-dimensional spatial grid coordinate system where the bird's-eye view feature map is located.

[0029] In one possible implementation, generating a bird's-eye view feature map based on the image of the parking space to be detected includes:

[0030] Multi-scale two-dimensional visual features are extracted from the image of the parking space to be detected. The multi-scale two-dimensional visual features include the spatial positional relationship of the parking space to be detected and the edge detail information of the parking space.

[0031] The geometric projection relationship from the image pixel coordinate system to the three-dimensional spatial grid coordinate system is constructed using the calibration parameters of the target camera. Then, through feature sampling and fusion operations, the multi-scale two-dimensional visual features are filled into the corresponding grid cells to obtain the bird's-eye view feature map.

[0032] In one possible implementation, generating the corrected position representation of the plurality of contour points based on the query vector updated according to the interaction process includes:

[0033] The position offsets of the plurality of contour points are generated based on the updated query vector;

[0034] Based on the position offset of each contour point, the initial position representation of each contour point is corrected to obtain the corrected position representation of each contour point.

[0035] In one possible implementation, the decoding module is a deformable transformer architecture.

[0036] A second aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the parking space detection method of the first aspect or any implementation thereof.

[0037] A third aspect of this application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0038] The memory is used to store computer programs;

[0039] The processor is used to execute the computer program so that the electronic device can implement the parking space detection method of the first aspect or any implementation thereof.

[0040] A fourth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the parking space detection method described in the first aspect or any implementation thereof.

[0041] By employing the aforementioned technical solution, the parking space detection method provided in this application acquires an image of the parking space to be detected and generates a bird's-eye view feature map based on the image, thereby providing a unified spatial feature representation for subsequent parking space detection. After obtaining the bird's-eye view feature map, this application uses contour point modeling to represent the parking space edges. Specifically, a learnable query vector is used to initialize the semantic identity and initial position representation of multiple contour points. These contour points are arranged sequentially according to the order determined by their semantic identities to form a connected parking space edge, thereby modeling the geometric shape of the parking space to be detected. Compared to directly modeling parking space edges, the contour point-based modeling method can achieve unified modeling of parking space edges of arbitrary shapes, such as vertical, parallel, oblique, and even arc-shaped edges, significantly improving the scene generalization ability of the parking space detection method.

[0042] To address the issue of insufficient initial contour point localization accuracy, this application introduces an iterative correction mechanism based on bird's-eye view feature map sampling. Specifically, local features are sampled from the bird's-eye view feature map based on the initial position representation. These local features interact with the query vector, and the updated query vector generates corrected position representations for multiple contour points. These corrected position representations are then used as the initial position representation to return to the deformable sampling step, until a preset iteration termination condition is met. Thus, this application achieves a closed-loop iteration of "sampling-interaction-fine-tuning," enabling the contour point positions to gradually approach the actual parking space edges through multiple rounds of optimization, achieving high-precision fitting of the parking space edges.

[0043] Finally, the semantic identities of multiple contour points and the corrected position representation obtained from the final iteration are used as the parking space detection result. This application achieves accurate detection of parking space edges of arbitrary shapes by combining contour point modeling and iterative correction. The entire process does not rely on the preset geometric constraints of the parking space shape, thus more completely preserving the original geometric shape of the parking space, improving the robustness and positioning accuracy of the parking space detection results, and providing more reliable information input for subsequent parking decisions and trajectory planning, thereby ensuring the safety and applicability of the parking system in complex scenarios.

[0044] Meanwhile, the modeling method based on contour points in this application only affects some contour points when there is local occlusion at the edge of the parking space. Subsequent iterations can complete the complete modeling of the parking space based on the local features of the visible line points, which greatly reduces the detection error caused by occlusion and damage to the edge of the parking space. Attached Figure Description

[0045] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0046] Figure 1 A schematic diagram of a system architecture provided for this application;

[0047] Figure 2 A flowchart illustrating a parking space detection method provided in this application;

[0048] Figure 3 This is a schematic diagram of the possible parking space configurations that may be included in the image of the parking space to be detected.

[0049] Figure 4(a) is a schematic diagram of an irregular parking space;

[0050] Figure 4(b) is a schematic diagram of the parking space detection results obtained using the rectangular box modeling method;

[0051] Figure 4(c) is a schematic diagram of the parking space detection results obtained using the corner point modeling method;

[0052] Figure 4(d) is a schematic diagram of the parking space detection results obtained using the method of this application;

[0053] Figure 5 A schematic diagram of the detection process for the parking space detection method provided in this application;

[0054] Figure 6 This is a schematic diagram of the structure of a parking space detection device provided in this application;

[0055] Figure 7 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0056] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0057] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0058] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0059] This application provides a parking space detection method and related apparatus, which can be applied to scenarios requiring high-precision parking space area detection, such as parking scenarios. Of course, the parking scenario is merely an example and is not intended to limit this application.

[0060] Optionally, the parking space detection method provided in this application can be applied to, for example... Figure 1 The system architecture shown includes a terminal 100 and a server 200. The server 200 may include one or more servers (…). Figure 1 (This example uses a server as an illustration).

[0061] Either terminal 100 or server 200 can be used independently to execute the parking space detection method provided in the embodiments of this application. Alternatively, terminal 100 and server 200 can also be used collaboratively to execute the parking space detection method provided in the embodiments of this application.

[0062] For example, in one possible implementation, terminal 100 can acquire images captured by a camera and perform necessary preprocessing to form an image of the parking space to be detected, and input it into server 200. Server 200 then performs parking space detection according to the parking space detection method provided in this application to obtain the parking space detection result of the parking space to be detected.

[0063] The following description Figure 1 The product form of the mid-terminal 100;

[0064] The terminal 100 in this application embodiment can be a mobile phone, tablet computer, wearable device, vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0065] To enable those skilled in the art to better understand this application, the parking space detection method of this application embodiments will be described in detail below with reference to the accompanying drawings.

[0066] Reference Figure 2 , Figure 2 This is a flowchart illustrating a parking space detection method provided in an embodiment of this application, as follows: Figure 2 As shown, the parking space detection method may include:

[0067] Step S201: Obtain the image of the parking space to be detected, and generate a bird's-eye view feature map based on the image of the parking space to be detected.

[0068] Here, the image of the parking space to be detected refers to the image of the parking space that needs to be detected. The parking space in the image may be a common rectangle or parallelogram, such as... Figure 3 The letters A, B, C, E, F, G, and H in the diagram may also be irregular shapes, such as... Figure 3 D and I in the text.

[0069] To obtain the aforementioned image of the parking space to be detected, this embodiment can acquire at least one frame of the vehicle's surrounding environment image. Here, the vehicle's surrounding environment image refers to the image of the environment around the current vehicle (such as a vehicle that needs to park). Each frame of the vehicle's surrounding environment image is an image captured from different angles around the current vehicle. For example, by acquiring environmental images from four angles (front, rear, left, and right) using four fisheye cameras installed in front, behind, left, and right of the current vehicle, four frames of the vehicle's surrounding environment image are obtained.

[0070] Considering that the image acquisition process by the camera may be affected by factors such as equipment distortion and lighting differences, resulting in distortion of the vehicle surrounding environment image, this embodiment can preprocess each frame of the vehicle surrounding environment image to eliminate the image distortion and lighting differences caused by the camera equipment, thus obtaining a preprocessed image. Therefore, the preprocessed image corresponding to at least one frame of the vehicle surrounding environment image can be used as the parking space image to be detected.

[0071] Optionally, preprocessing may include one or more of the following: distortion correction, image size normalization, and pixel value normalization.

[0072] Considering that images captured by multiple cameras around a vehicle often show overlapping parking spaces, in order to avoid misidentifying overlapping parking spaces as two separate parking spaces and to improve the detection accuracy of overlapping parking spaces, alternatively, multiple frames of the vehicle surrounding environment image before preprocessing or multiple frames after preprocessing can be stitched together with the processed image to obtain a surround view image, and then the image of the parking space to be detected can be obtained based on the surround view image.

[0073] Of course, there are other methods to obtain images of the parking spaces to be detected, which will not be elaborated here.

[0074] In addition, considering that the bird's eye view (BEV) perspective can better eliminate image distortion and restore the spatial relationship of parking spaces, this embodiment can also generate a bird's eye feature map based on the image of the parking space to be detected, thereby providing a unified spatial feature representation for subsequent parking space detection.

[0075] Here, the bird's-eye view feature map contains the spatial positional relationship of the parking spaces to be detected and the detailed information of the parking space edges, so that subsequent parking space detection can make full use of the spatial positional relationship of the parking spaces and the detailed information of the parking space edges in the bird's-eye view feature map to more accurately model the parking space edges.

[0076] Step S202: Initialize the semantic identity and initial position representation of multiple contour points using learnable query vectors, and sample local features in the bird's-eye view feature map based on the initial position representation.

[0077] Considering the diverse shapes of parking spaces, especially irregular ones, which often include curved edges, unlike straight-line modeling, the curvature and direction of curved curves are not fixed. They may have continuous changes in direction and no fixed pattern, making it difficult to use uniform rules for stable representation and accurate fitting, which significantly increases the modeling difficulty.

[0078] In order to model curved and straight parking space edges of arbitrary shapes, this application provides a contour point modeling method, which models the parking space as an ordered parking space edge structure. Each edge (usually 4 edges) is represented by several contour points. In other words, all parking space edges of the parking space to be detected can be represented by multiple contour points.

[0079] Based on this, this embodiment can use a learnable query vector (query) to initialize and generate the semantic identities and initial position representations of multiple contour points. Here, the semantic identities are used not only to distinguish different contour points, but also to describe the relationship between the contour points and the parking space edges. That is, based on the semantic identities of multiple contour points, this embodiment can determine which contour points are used to fit the same parking space edge.

[0080] Optionally, the semantic identity of a contour point is related to its positional relationship with the parking entrance edge. Here, the parking entrance edge refers to the boundary line of the parking space on the side closest to the driving lane. It is a key boundary that a vehicle must pass through during the process of driving into or reversing into the parking space, used to define the parking space entrance location, constrain the vehicle's parking trajectory, and determine the parking status.

[0081] For example, we predefine that contour points with semantic identities 1-10 are on the edge of the parking entrance, contour points with semantic identities 11-20 are on the edge of the next parking space in a clockwise direction from the parking entrance, and so on. Then, if contour point 1 is the first contour point in a clockwise direction on the edge of the parking entrance, then the semantic identity of contour point 1 is label 1; if contour point 2 is the first contour point on the next edge in a clockwise direction from the parking entrance, then the semantic identity of contour point 2 is label 11.

[0082] In this embodiment, the semantic identities of multiple contour points can also determine their order. For example, starting from the parking entrance edge, the semantic identities of multiple contour points can be labeled 1, 2, ..., N in a clockwise direction. Thus, multiple contour points can be arranged sequentially according to the order determined by their semantic identities to form a parking space edge that is connected end to end, thereby modeling the geometric shape of the parking space to be detected.

[0083] For example, for the irregular parking space in Figure 4(a), the parking space detection result obtained by the rectangular box modeling method is shown in Figure 4(b), and the parking space detection result obtained by the corner point modeling method is shown in Figure 4(c). It can be seen that the modeling results of the two methods cannot completely fit the actual parking space edge, that is, there is a deviation in the parking space detection result. In the automatic parking scenario, when there is a deviation in the parking space detection result, it is very likely that the incorrect parking space detection result will lead to safety accidents such as vehicle collision with the wall.

[0084] Referring to Figure 4(d), the parking space detection result obtained by the method of this application is shown. By modeling the parking space edge with multiple contour points, a parking space area that perfectly fits the real parking space edge can be modeled, and the obtained parking space detection result is more accurate.

[0085] Understandably, the initial position representation does not reference any information from the real environment, and is likely to have positional deviations. To align with the actual parking space, the initial position representation needs to be fine-tuned. To ensure that the correction process is based on the semantic information of the real environment, this embodiment can sample local features from the bird's-eye view feature map based on the initial position representation.

[0086] For example, features at the initial location in the bird's-eye view feature map can be sampled as local features; or features at the initial location and the surrounding neighborhood can be sampled and fused as local features.

[0087] Of course, there are other sampling methods, which will not be specifically limited here.

[0088] Step S203: Interact with the local features and the query vector, and generate corrected position representations of multiple contour points based on the query vector updated during the interaction process.

[0089] Specifically, in this embodiment, local features can interact with the query vector, such as by performing cross-attention calculation to update the query vector. This allows the updated query vector to incorporate the semantics of local features, enhancing its ability to perceive the positional relationship between each contour point and the corresponding parking space edge. Then, the initial position representation of multiple contour points can be fine-tuned and corrected based on the updated query vector, thus obtaining the corrected position representation of multiple contour points.

[0090] Step S204: The step of returning the corrected position representation as the initial position representation to sample local features continues until the preset iteration termination condition is met. Then, the semantic identities of multiple contour points and the corrected position representation obtained in the last iteration are used as the parking space detection result of the parking space to be detected.

[0091] It is understandable that a single correction process may not be able to directly correct the initial position representation to a tolerable position deviation range. In order to obtain a more accurate position representation, this embodiment can use the corrected position representation as the initial position representation to return to the sampling of local features. That is, it can interact with the real environment again, update the query vector, and correct the new initial position representation again, and so on, until the iteration termination condition is reached.

[0092] Optionally, the iteration termination condition may include: convergence of the corrected position representation, or reaching a preset number of iterations. Of course, other iteration termination conditions are also possible and are not limited here.

[0093] Based on a multi-round iterative process of interaction with the real environment, the position representations of multiple contour points can be finally corrected to the edge of the parking space. It is understandable that correcting the position representation alone can only determine the positional relationship of each contour point, but cannot determine how the contour points are connected. Therefore, it is also necessary to input the semantic identity of each contour point as part of the parking space detection result into the downstream system so that the downstream system can connect the contour points according to their semantic identities to form the parking space area shown in Figure 4(d).

[0094] The parking space detection method provided in this application acquires an image of the parking space to be detected and generates a bird's-eye view feature map based on the image, thereby providing a unified spatial feature representation for subsequent parking space detection. After obtaining the bird's-eye view feature map, this application uses contour point modeling to represent the parking space edges. Specifically, a learnable query vector is used to initialize the semantic identity and initial position representation of multiple contour points. These contour points are arranged sequentially according to the order determined by their semantic identities to form a connected parking space edge, thereby modeling the geometric shape of the parking space to be detected. Compared with the method of directly modeling parking space edges, the contour point-based modeling method can achieve unified modeling of parking space edges of arbitrary shapes such as vertical, parallel, oblique, and even arc, significantly improving the scene generalization ability of the parking space detection method.

[0095] To address the issue of insufficient initial contour point localization accuracy, this application introduces an iterative correction mechanism based on bird's-eye view feature map sampling. Specifically, local features are sampled from the bird's-eye view feature map based on the initial position representation. These local features interact with the query vector, and the updated query vector generates corrected position representations for multiple contour points. These corrected position representations are then used as the initial position representation to return to the deformable sampling step, until a preset iteration termination condition is met. Thus, this application achieves a closed-loop iteration of "sampling-interaction-fine-tuning," enabling the contour point positions to gradually approach the actual parking space edges through multiple rounds of optimization, achieving high-precision fitting of the parking space edges.

[0096] Finally, the semantic identities of multiple contour points and the corrected position representation obtained from the final iteration are used as the parking space detection result. This application achieves accurate detection of parking space edges of arbitrary shapes by combining contour point modeling and iterative correction. The entire process does not rely on the preset geometric constraints of the parking space shape, thus more completely preserving the original geometric shape of the parking space, improving the robustness and positioning accuracy of the parking space detection results, and providing more reliable information input for subsequent parking decisions and trajectory planning, thereby ensuring the safety and applicability of the parking system in complex scenarios.

[0097] Meanwhile, the modeling method based on contour points in this application only affects some contour points when there is local occlusion at the edge of the parking space. Subsequent iterations can complete the complete modeling of the parking space based on the local features of the visible line points, which greatly reduces the detection error caused by occlusion and damage to the edge of the parking space.

[0098] Through experimental verification, the parking space detection method provided in this application requires only about 4-6 iterations for the parking space detection model to ensure that the corrected position representation (coordinates) of the contour points converges to a stable value, thereby improving the efficiency of parking space detection.

[0099] In one possible implementation, steps S201-S204 can be implemented using a pre-trained parking space detection model, which includes an encoding module and a decoding module. The encoding module implements step S201, inputting the image of the parking space to be detected into the encoding module. This module performs feature encoding on the image to obtain a bird's-eye view feature map. The decoding module implements steps S202-S204, inputting the bird's-eye view feature map into the decoding module of the parking space detection model. Within this module, the semantic identities and initial position representations of multiple contour points are initialized using a learnable query vector. Based on the initial position representations, local features are sampled in the bird's-eye view feature map. These local features interact with the query vector, and the updated query vector is used to generate corrected position representations for the multiple contour points. These corrected position representations are then used as initial position representations to return to the step of sampling local features. This process continues until a preset iteration termination condition is met, at which point the semantic identities of the multiple contour points and the final iterated corrected position representations are output.

[0100] Optionally, the process of "generating a bird's-eye view feature map from the image of the parking space to be detected" may include: extracting multi-scale two-dimensional visual features from the image of the parking space to be detected, the multi-scale two-dimensional visual features containing the spatial positional relationship and edge detail information of the parking space to be detected, constructing a geometric projection relationship from the image pixel coordinate system to the three-dimensional space grid coordinate system using the calibration parameters of the target camera, and filling the corresponding grid cells with the multi-scale two-dimensional visual features through feature sampling and fusion operations to obtain the bird's-eye view feature map.

[0101] For example, optionally, the encoding module in the parking space detection model may include a backbone network layer and a view transformation layer.

[0102] Optionally, the backbone network layer adopts the ResNet (Residual Network) architecture. The ResNet is used to downsample the parking space image to be detected by different factors to obtain multi-level features, which serve as multi-scale two-dimensional visual features.

[0103] Optionally, the backbone network layer can also adopt an architecture combining ResNet and FPN (Feature Pyramid Network). ResNet downsamples the parking space image to be detected by different factors to obtain multi-level features. Then, FPN performs top-down feature processing and fusion based on the multi-level features to generate multi-scale two-dimensional visual features, such as corresponding to four scales: 1 / 8, 1 / 16, 1 / 32, and 1 / 64.

[0104] In this embodiment, by extracting multi-scale two-dimensional visual features, both the detailed information of the parking space lines and the global information of the spatial relationship of the parking spaces can be preserved, thereby improving the accuracy of subsequent parking space detection.

[0105] Of course, other architectures can also be used for the backbone network layer, which will not be introduced here.

[0106] The aforementioned view transformation layer is used to transform multi-scale two-dimensional visual features to a three-dimensional spatial grid coordinate system, resulting in a bird's-eye view feature map. Here, the three-dimensional spatial grid coordinate system is a predefined coordinate system, and the grid resolution can be adaptively adjusted according to the detection accuracy requirements.

[0107] Optionally, the specific process of "constructing the geometric projection relationship from the image pixel coordinate system to the three-dimensional spatial grid coordinate system using the calibration parameters of the target camera, and filling the corresponding grid cells with multi-scale two-dimensional visual features through feature sampling and fusion operations to obtain a bird's-eye view feature map" can be as follows:

[0108] like Figure 5 As shown, this embodiment first constructs a three-dimensional spatial grid coordinate system and generates the three-dimensional coordinates of all grid centers.

[0109] Next, using the calibration parameters (including camera intrinsic and extrinsic parameters) of the target camera acquiring the images of the parking spaces to be detected, each BEV grid cell is projected onto the image pixel coordinates. Valid projection points falling within the effective range of the image are selected. Then, for each valid projection point, the most suitable feature map scale is selected from multi-scale two-dimensional visual features based on its depth or scale (e.g., a low-resolution feature map is selected for distant projection points, and a high-resolution feature map is selected for nearby projection points). Feature vectors are extracted on the selected feature maps using bilinear interpolation to obtain the feature vector corresponding to each BEV grid cell.

[0110] For the same BEV grid cell, it may receive feature vectors from multiple cameras. These feature vectors from multiple cameras can be fused, and the fused feature vector can be stored in the corresponding position of the BEV feature map to obtain the BEV bird's-eye view feature map.

[0111] For example, Figure 5 The illustration shows a parking space image to be detected containing four preprocessed frames. In this embodiment, each preprocessed frame can be processed to obtain a set of multi-scale two-dimensional visual features. Since there may be overlapping areas in two preprocessed images from different perspectives, the same BEV grid unit may require feature vector extraction from two sets of multi-scale two-dimensional visual features. Therefore, it is necessary to fuse the two extracted feature vectors as the BEV feature corresponding to the BEV grid unit.

[0112] In this embodiment, spatial grid mapping is used, which can effectively fuse camera features to obtain a clearer bird's-eye view feature map of the geometric structure, thereby improving the accuracy of parking space detection.

[0113] For example, see still Figure 5 Optionally, the process of "generating corrected position representations of multiple contour points based on the query vector updated in the interaction process" may include: generating position offsets of multiple contour points based on the query vector updated in the interaction process, and correcting the initial position representation of each contour point based on the position offset of each contour point to obtain the corrected position representation of each contour point.

[0114] Optionally, the above decoding module can be a deformable transformer architecture. In this case, the process of "generating multiple corrected position representations of contour points based on the query vector updated in the interaction process" can include: generating multiple position offsets for each contour point based on the query vector updated in the interaction process, and correcting the initial position representation of each contour point based on the multiple position offsets of each contour point to obtain multiple corrected position representations of each contour point.

[0115] Furthermore, for each contour point, after taking multiple corrected position representations as initial position representations, the process of "sampling local features in the bird's-eye view feature map based on the initial position representations" mentioned above can include: performing deformable sampling in the bird's-eye view feature map based on multiple initial position representations, and generating local features by fusing the sampled features.

[0116] In this embodiment, a deformable transformer architecture is adopted, which can generate multiple learnable position offsets based on the query content in each iteration. This allows for adaptive adjustment of the sampling point positions, concentrating the sampling points as much as possible in key areas such as the edges of parking spaces, thus improving the discriminative power of local features. At the same time, the method of sampling multiple points at once and fusing them is equivalent to allowing each query vector to focus on multiple key parts of the parking space simultaneously, so that the aggregated local features naturally contain the overall structural information of the parking space. In addition, the method of sampling multiple points at once ensures that even if some sampling points deviate due to inaccurate offset prediction, other sampling points can still provide effective information, improving the robustness of local features.

[0117] As introduced above, it is understandable that the query vector in this application can obtain a parking space detection result by continuously iterating and updating the positions of multiple contour points.

[0118] It is also understandable that if multiple parking spaces need to be detected, multiple learnable query vectors can be set up so that each query vector corresponds one-to-one with a parking space to be detected, thereby enabling each query vector to be used to detect one parking space.

[0119] Considering the spatial relationship between parking spaces, if each query vector is predicted independently, it may lead to problems such as the predicted parking spaces having geometric relationships that do not match the facts, or the same parking space being detected repeatedly. To avoid this problem, optionally, before step S203 "interacting local features with query vectors", multiple query vectors can be subjected to self-attention interaction to obtain multiple initial update query vectors. Specifically, "interacting local features with query vectors" can be an interaction between local features and the corresponding initial update query vectors.

[0120] It should also be noted that the process of self-attention interaction among multiple query vectors can be executed only before the first iteration or in every iteration; this application does not impose any specific limitations.

[0121] In another possible implementation, in order to determine whether the parking space detection result is reliable, optionally, after step S203 "interacting with the query vector with the local features", the parking space location reliability of the parking space to be detected can be generated based on the query vector updated in the interaction process. Here, the parking space location reliability characterizes whether the parking space to be detected is a reliable parking space.

[0122] In this embodiment, by dynamically adjusting the sampling position of BEV features during the iteration process, it can not only dynamically update the position representation of multiple contour points that characterize the outline of the parking space to be detected, ensuring that the contour points fall accurately on the edge of the parking space, but also determine whether the parking space area updated by each position adjustment is a reliable parking space, providing richer reference information for subsequent parking.

[0123] The training process of the above parking space detection model is described below.

[0124] Specifically, parking space image samples are acquired, and contour points are marked on the parking space lines of the parking space image samples according to preset point marking rules to obtain the contour point set label corresponding to each parking space in the parking space image samples. The parking space detection model to be trained is then trained using the parking space image samples and the contour point set label corresponding to each parking space in the parking space image samples.

[0125] Optionally, the point marking rules include at least one of the following rules: a point number and distribution rule for indicating the number and distribution of outline points marked on each parking space edge, a point sorting rule for indicating the semantic identity of outline points and the point sorting order determined by the semantic identity of outline points, and a point representation rule for indicating the position representation of outline points.

[0126] Optionally, the distribution indicated by the number of points and distribution rules is a uniform distribution. For example, one possible rule for the number of points and distribution rules could be "10 uniformly spaced contour points are configured on each parking space side, and the point spacing is adaptively adjusted according to the length of the parking space side to ensure adaptability to parking spaces of different sizes and shapes".

[0127] In the above rules for the number and distribution of points, the method of pre-setting the number of outline points on the edge of the parking space is adopted. It should be understood that, in addition to this, there are other ways to determine the number of outline points, such as determining the number of outline points according to the length of the parking space edge and the pre-set point interval.

[0128] This embodiment, by configuring the number of points and the rules for point distribution, enables the parking space detection model to learn in a constrained manner during training how to initialize and iteratively fine-tune the position representation of each contour point. This avoids the parking space detection model blindly fine-tuning the position representation of each contour point without constraints, which could lead to learning incorrect modeling methods and improves its accuracy in detecting parking spaces.

[0129] Optionally, the point sorting rule indicates the following point sorting order: all marked contour points are arranged in a clockwise order along the parking space edges, with the parking entrance edge as the starting edge for the arrangement. For example, a possible point sorting rule could be: "The contour points are arranged in a clockwise order along the parking space edges, where contour points 1-10 correspond to the parking entrance edge, and points 11-20, 21-30, and 31-40 correspond to the three subsequent edges clockwise from the parking entrance edge, forming a fixed and ordered vector sequence," as shown in Figure 4(d). Contour points on the parking entrance edge are represented by white dots, while contour points on the other parking space edges are represented by black dots for distinction.

[0130] This embodiment configures the point sorting order with the parking entrance edge as the starting constraint, which enables the parking space detection model to focus its learning ability on accurately modeling the spatial position of each contour point during training, without having to deal with the ambiguity of the order of multiple contour points adapting to semantic identities, thus improving the pertinence and stability of point position regression.

[0131] Optionally, the location representation indicated by the point representation rule is the coordinate representation in the three-dimensional spatial grid coordinate system where the bird's-eye view feature map is located. For example, one possible point representation rule is "each contour point contains (x,y) coordinate information, and the coordinates are defined based on the three-dimensional spatial grid coordinate system where the BEV is located".

[0132] This embodiment configures each contour point with bird's-eye view features. Figure 1 The consistent coordinate system allows for faster sampling of local features from the bird's-eye view feature map, improving the efficiency and reliability of parking space detection.

[0133] It should also be noted that, in addition to the rules mentioned above, the point marking rules can also include other rules, such as point type rules that indicate whether multiple contour points contain parking space corner points.

[0134] By configuring the point labeling rules described above to meet user needs, the training labels for the parking space detection model can be generated according to the point labeling rules, thus making the multiple vector points predicted by the parking space detection model more in line with user requirements. Especially when the point type rules indicate that multiple contour points must include parking space corner points (such as the four corner points of a parking space), this application can directly connect each contour point according to the corrected position representation and its semantic identity after the parking space detection model predicts the corrected position representation of multiple contour points to form a parking space region, without the need for any post-processing such as clustering or fitting, thereby improving the efficiency of parking space detection.

[0135] Furthermore, this embodiment only requires simply marking the contour points on the edge of the parking space to train the parking space detection model. Once trained, it can be directly used for parking space detection. The entire process does not require explicitly defining the geometric parameters of the parking space, making the detection process simpler and balancing detection accuracy and efficiency.

[0136] The above describes a parking space detection method provided by the embodiments of this application. The following describes the apparatus for performing the above parking space detection method.

[0137] Please see Figure 6 , Figure 6 This is a schematic diagram of a parking space detection device provided in an embodiment of this application. Figure 6 As shown, the parking space detection device may include:

[0138] The data input unit 501 is used to acquire an image of the parking space to be detected and generate a bird's-eye view feature map based on the image of the parking space to be detected.

[0139] The sampling unit 502 is used to initialize the semantic identity and initial position representation of multiple contour points using a learnable query vector, and to sample local features in the bird's-eye view feature map based on the initial position representation. The multiple contour points are arranged sequentially according to the order determined by the semantic identity to form a parking space edge connected end to end, so as to model the geometric shape of the parking space to be detected.

[0140] The position correction unit 503 is used to interact the local features with the query vector and generate the corrected position representation of the multiple contour points based on the query vector updated during the interaction process.

[0141] The detection output unit 504 is used to return the corrected position representation as the initial position representation to the step of sampling local features until a preset iteration termination condition is met, and then uses the semantic identity of the multiple contour points and the corrected position representation obtained in the last iteration as the parking space detection result of the parking space to be detected.

[0142] Each module in the aforementioned parking space detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0143] This application also provides an electronic device, which may include at least one processor and a memory connected to the processor, wherein:

[0144] Memory is used to store computer programs;

[0145] The processor is used to execute computer programs to enable the electronic device to implement any of the parking space detection methods provided in the embodiments of this application.

[0146] refer to Figure 7 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0147] like Figure 7 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. When the electronic device is powered on, the RAM 603 also stores various programs and data required for the operation of the electronic device. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0148] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, memory cards, hard drives, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0149] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the parking space detection methods provided in this application.

[0150] This application also provides a computer-readable storage medium carrying one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the parking space detection methods provided in this application.

[0151] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0152] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0153] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0154] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A parking space detection method, characterized in that, include: Acquire an image of the parking space to be detected, and generate a bird's-eye view feature map based on the image of the parking space to be detected; The semantic identity and initial position representation of multiple contour points are initialized using learnable query vectors, and local features are sampled in the bird's-eye view feature map based on the initial position representation. The semantic identity represents the parking space edge to which the contour point belongs and the arrangement order on the parking space edge. The multiple contour points are arranged sequentially according to the order determined by the semantic identity to form a parking space edge that is connected end to end, so as to model the geometric shape of the parking space to be detected. The local features are interacted with the query vector, and the corrected position representation of the multiple contour points is generated based on the query vector updated during the interaction process. The step of returning the corrected position representation as the initial position representation to sample local features continues until a preset iteration termination condition is met. Then, the semantic identity of the multiple contour points and the corrected position representation obtained in the last iteration are used as the parking space detection result of the parking space to be detected. There are multiple learnable query vectors, and each query vector is used to detect one of the parking spaces to be detected; Before interacting with the query vector using the local features, the method further includes: Multiple query vectors are subjected to self-attention interaction to obtain multiple initial update query vectors; The step of interacting the local features with the query vector includes: The local features are interacted with the corresponding initial update query vector.

2. The parking space detection method according to claim 1, characterized in that, After interacting with the local features and the query vector, the method further includes: The vehicle location confidence score of the parking space to be detected is generated based on the query vector updated during the interaction process. The vehicle location confidence score indicates whether the parking space to be detected is a reliable parking space.

3. The parking space detection method according to claim 1, characterized in that, The step of generating a bird's-eye view feature map based on the image of the parking space to be detected includes: The image of the parking space to be detected is input into the encoding module of the pre-trained parking space detection model to obtain the bird's-eye view feature map; The process involves initializing the semantic identities and initial position representations of multiple contour points using learnable query vectors, sampling local features in the bird's-eye view feature map based on the initial position representations, interacting the local features with the query vectors, generating corrected position representations of the multiple contour points based on the updated query vectors during the interaction process, and returning the corrected position representations as the initial position representations to sample local features. This process continues until a preset iteration termination condition is met, including: The bird's-eye view feature map is input into the decoding module of the parking space detection model. The decoding module initializes the semantic identity and initial position representation of multiple contour points using a learnable query vector. Based on the initial position representation, local features are sampled in the bird's-eye view feature map. The local features are interacted with the query vector. The revised position representation of the multiple contour points is generated according to the query vector updated during the interaction process. The revised position representation is used as the initial position representation and returned to the step of sampling local features. This process continues until a preset iteration termination condition is met. Finally, the semantic identity of the multiple contour points and the revised position representation obtained in the last iteration are output.

4. The parking space detection method according to claim 3, characterized in that, The training process of the parking space detection model includes: Obtain parking space image samples; According to the preset point marking rules, contour points are marked on the parking space lines of the parking space image sample to obtain the contour point set label corresponding to each parking space in the parking space image sample. The point marking rules include at least one of the following rules: point number and distribution rules for indicating the number and distribution of contour points marked on each parking space edge, point sorting rules for indicating the semantic identity of contour points and the point sorting order determined by the semantic identity of contour points, and point representation rules for indicating the position representation method of contour points. The parking space detection model is trained using the parking space image samples and the set of contour points corresponding to each parking space in the parking space image samples.

5. The parking space detection method according to claim 4, characterized in that, The distribution indicated by the number of points and the distribution rules is a uniform distribution; And / or, the point sorting rule indicates that the point sorting order is as follows: all marked contour points are arranged in clockwise order of the parking space edges, with the parking entrance edge as the starting parking space edge for the arrangement; And / or, the location representation indicated by the point representation rule is the coordinate representation in the three-dimensional spatial grid coordinate system where the bird's-eye view feature map is located.

6. The parking space detection method according to claim 1, characterized in that, The step of generating a bird's-eye view feature map based on the image of the parking space to be detected includes: Multi-scale two-dimensional visual features are extracted from the image of the parking space to be detected. The multi-scale two-dimensional visual features include the spatial positional relationship of the parking space to be detected and the edge detail information of the parking space. The geometric projection relationship from the image pixel coordinate system to the three-dimensional spatial grid coordinate system is constructed using the calibration parameters of the target camera. Then, through feature sampling and fusion operations, the multi-scale two-dimensional visual features are filled into the corresponding grid cells to obtain the bird's-eye view feature map.

7. The parking space detection method according to claim 1, characterized in that, The step of generating the corrected position representation of the multiple contour points based on the query vector updated during the interaction process includes: The position offsets of the plurality of contour points are generated based on the updated query vector; Based on the position offset of each contour point, the initial position representation of each contour point is corrected to obtain the corrected position representation of each contour point.

8. The parking space detection method according to claim 3, characterized in that, The decoding module is a deformable transformer architecture.

9. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the parking space detection method as described in any one of claims 1 to 8.

10. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the parking space detection method as described in any one of claims 1 to 8.

11. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the parking space detection method as described in any one of claims 1 to 8.

Images