IPM-based parking space detection method, system and electronic device

Abstract

This invention provides a parking space detection method, system, and electronic device based on Inverse Perspective Mapping (IPM). The method includes acquiring an initial surround view image of an autonomous vehicle and converting it into an initial IPM image using inverse perspective mapping (IPM). The initial IPM image is then optimized to generate an optimized IPM feature map. A pre-trained parking space detection model is used to process the optimized IPM feature map to determine the coordinate regions of the four corner points of the parking space, the coordinate region of the center point of the parking space, and the horizontal and vertical coordinate offsets of each pixel within the foreground region of the parking space relative to the four corner points. A neighborhood region of the center point of the parking space is determined from the foreground region. Furthermore, based on the horizontal and vertical coordinate offsets of each pixel within the neighborhood region of the center point of the parking space relative to the four corner points, the true coordinate positions of the four corner points are determined from their respective coordinate regions. This invention addresses the problem of low accuracy in existing parking space detection schemes.

Description

[0001] Technology Neighborhood This invention relates to the field of autonomous driving technology, and more specifically, to an IPM-based parking space detection method, system, and electronic device. Background Technology

[0002] One of the core goals of autonomous driving is to achieve a fully autonomous driving experience, and automatic parking, as a crucial part of the driving process, plays a pivotal role. With the rapid advancement of autonomous driving technology, parking perception and space detection have gradually become an indispensable key application in autonomous driving systems. Developing efficient and accurate parking perception and space detection algorithms can significantly improve the intelligence level of autonomous driving systems in complex environments and effectively enhance the vehicle's autonomous parking capabilities.

[0003] In this context, parking space detection technology in autonomous driving systems, especially image processing-based technology, is of irreplaceable importance. By analyzing image data within a certain range of the vehicle in real time, parking space detection technology can quickly and accurately identify the location and attributes of parking spaces (whether they are occupied), providing precise decision support for automatic parking systems.

[0004] However, existing parking space detection technologies still face the following technical challenges: 1. Insufficient accuracy of parking space corner points: Existing parking space detection methods typically predict the coordinates of the parking space center point, the coordinates of the four corner points of the parking space, and the offset of the parking space center point relative to the four corner points to construct the relative positional relationship between the corner points, thereby realizing the position detection of the entire parking space. However, the existing parking space detection methods lack stability in predicting the parking space center point, and since the positioning of the parking space corner points depends on the accuracy of the parking space center point, this instability in the detection of the parking space center point will directly affect the accuracy of subsequent corner point detection, resulting in insufficient accuracy of the overall parking space position detection.

[0005] 2. Significant misjudgment of parking space attributes: Parking space detection requires not only detecting parking space corners but also parking space attributes. Existing parking space detection methods usually classify parking space attributes based on the center point of the parking space. However, due to factors such as occlusion and mosaic, the classification of the center point of the parking space detection method is not robust, which inevitably leads to misjudgment of parking space attributes.

[0006] Based on the aforementioned technical issues, a high-precision parking space detection solution is urgently needed. Summary of the Invention

[0007] In view of the above problems, the purpose of this invention is to provide an IPM-based parking space detection method and system to solve the problem of low accuracy in existing parking space detection schemes.

[0008] The parking space detection method based on IPM provided by this invention is characterized by comprising: Acquire initial surround view images of autonomous vehicles, and convert the initial surround view images into initial IPM images based on inverse perspective mapping technology; The initial IPM image is optimized to generate an optimized IPM feature map; The pre-trained parking space detection model is used to process the IPM optimized feature map to determine the coordinate regions of the four corner points of the parking space, the coordinate region of the center point of the parking space, and the horizontal and vertical coordinate offsets of each pixel in the foreground area of ​​the parking space relative to the four corner points of the parking space. The center point of the parking space is determined from the region to which the coordinates of the center point of the parking space belong, and the neighborhood region of the center point of the parking space is determined from the foreground region of the parking space; and, based on the horizontal and vertical coordinate offsets of each pixel point in the center point of the parking space and its neighborhood region relative to the four corner points of the parking space, the true coordinate positions of the four corner points of the parking space are determined from the region to which the coordinates of the four corner points of the parking space belong.

[0009] Alternatively, the IPM optimized feature map can be processed by the parking space detection model to determine the occupancy label of the foreground area of ​​the parking space. Based on the occupancy label of the foreground area of ​​the parking space, the occupancy status of the corresponding parking space is determined.

[0010] Alternatively, the initial IPM image can be optimized by including color space enhancement; wherein the color space enhancement process includes: The initial IPM image is converted from the RGB color space to the HSV color space; Random perturbations are applied to the initial IPM image in the H, S, and V channels of the HSV color space, respectively. The perturbated initial IPM image is converted from the HSV color space to the RGB color space.

[0011] In addition, an optional approach is to further optimize the initial IPM image by including geometric transformation enhancement; wherein the process of geometric transformation enhancement includes: Using the center point of the initial IPM image as a reference, the initial IPM image is subjected to rotational affine transformation, horizontal flip affine transformation, vertical flip affine transformation, and translational affine transformation, respectively; wherein, During the rotational affine transformation, horizontal flip affine transformation, vertical flip affine transformation, and translation affine transformation of the initial IPM image, the continuity of the initial IPM image is maintained by a bilinear interpolation algorithm.

[0012] In addition, an alternative approach is to further optimize the initial IPM image by including: The initial IPM image is normalized to linearly scale the pixel values ​​of the initial IPM image to the range of [0, 1]. The initial IPM image after normalization is spatially downsampled by 8 times to obtain an IPM optimized feature map of a preset dimension.

[0013] Alternatively, the parking space detection model can be constructed based on an Hourglass network, and the model includes a downsampling encoding module, a bottom-level feature extraction module, an upsampling encoding module, a multi-scale feature fusion module, and a parking space regression module; wherein, The downsampling encoding module is used to perform three levels of spatial downsampling on the input IPM optimized feature map, generating downsampled features at the first, second, and third scales respectively, and further downsampling to the lowest level feature map; wherein, the downsampled features at each scale are cached after generation for subsequent cross-path feature fusion. The lowest-level feature extraction module performs high-level semantic modeling on the lowest-level feature map through non-linear convolution and compresses the channel dimension to generate the lowest-level features with strong context awareness. The upsampling decoding module starts from the lowest level feature and performs three levels of upsampling operations in sequence. It uses bilinear interpolation to gradually restore the spatial resolution, and obtains upsampled features at the third scale, second scale and first scale in sequence, and finally restores them to the original input size. The multi-scale feature fusion module is used to immediately add the current upsampled feature to the downsampled feature of the corresponding scale in the downsampled encoding module element by element after each upsampling level, so as to realize cross-path feature fusion. The parking space regression module is used to regress the coordinate regions of the four corner points of the parking space, the coordinate region of the center point of the parking space, the horizontal and vertical coordinate offsets of each pixel in the foreground region of the parking space relative to the four corner points of the parking space, and the semantic labels of the foreground region of the parking space.

[0014] Alternatively, the process of determining the neighborhood of the center point of the parking space from the foreground area of ​​the parking space includes: The selected coordinate position of the center point of the parking space is determined from the region to which the coordinates of the center point of the parking space belong; Based on the selected coordinates of the center point of the parking space, the neighborhood of the center point of the parking space is determined from the foreground area of ​​the parking space.

[0015] Alternatively, an optional approach is to determine the true coordinate positions of the four corner points of the parking space from the region to which the coordinates of the four corner points belong, based on the horizontal and vertical coordinate offsets of each pixel within the neighborhood of the center point of the parking space relative to the four corner points of the parking space. This process includes: By calculating the horizontal and vertical coordinate offsets of each pixel in the neighborhood of the parking space center point relative to the four corner points of the parking space, the regression point coordinates of the four corner points that match the parking space center point are calculated. Based on the regression point coordinates of the four corner points that match the center point of the parking space, the true coordinate positions of the four corner points of the parking space are determined from the area to which the coordinates of the four corner points of the parking space belong.

[0016] On the other hand, the present invention also provides an IPM-based parking space detection system, which includes: an image acquisition module, an image processing module, a model processing module, and a parking space post-processing module; wherein, The image acquisition module is used to acquire the initial surround view image of the autonomous vehicle and convert the initial surround view image into an initial IPM image based on inverse perspective mapping technology. The image processing module is used to optimize the initial IPM image to generate an optimized IPM feature map; The model processing module is used to process the IPM optimized feature map through the pre-trained parking space detection model to determine the coordinate regions of the four corner points of the parking space, the coordinate regions of the center point of the parking space, and the horizontal and vertical coordinate offsets of each pixel in the foreground area of ​​the parking space relative to the four corner points of the parking space. The parking space post-processing module is used to determine the parking space center point from the area to which the coordinates of the parking space center point belong, and to determine the neighborhood area of ​​the parking space center point from the foreground area of ​​the parking space; and, based on the horizontal and vertical coordinate offsets of each pixel point in the parking space center point and its neighborhood area relative to the four corner points of the parking space, to determine the true coordinate positions of the four corner points of the parking space from the area to which the coordinates of the four corner points of the parking space belong.

[0017] Alternatively, the parking space post-processing module may also include a parking space occupancy status detection module; wherein, The model processing module is also used to process the IPM optimized feature map to determine the occupancy label of the foreground area of ​​each parking space. The parking space occupancy detection module is used to determine the occupancy status of each parking space based on the occupancy label in the foreground area of ​​each parking space.

[0018] Compared with the prior art, the parking space detection method and system based on IPM provided by the present invention have the following advantages: To address the issue of insufficient accuracy in detecting parking space corner points in existing parking space detection solutions, the solution provided by this invention no longer simply predicts the offset of the parking space center point relative to the four corner points. Instead, it predicts the offset of the foreground area of ​​all parking spaces relative to the four corner points. Compared to solutions that predict the offset of a single parking space center point relative to the four corner points, this approach can more accurately analyze the spatial distribution of parking spaces, providing richer location information and making corner point positioning more precise. This allows for better reconstruction of the actual boundaries of parking spaces and reduces the problem of inaccurate corner point positioning caused by prediction errors in the offset of the center point relative to the four corner points.

[0019] To achieve the foregoing and related objectives, one or more aspects of the invention include the features which will be described in detail below and specifically pointed out in the claims. The following description and accompanying drawings illustrate certain exemplary aspects of the invention. However, these aspects indicate only a few of the various ways in which the principles of the invention can be used. Furthermore, the invention is intended to include all such aspects and their equivalents. Attached Figure Description

[0020] Other objects and results of the invention will become more apparent and readily understood with reference to the following description taken in conjunction with the accompanying drawings and the contents of the claims, and with a more complete understanding of the invention. In the drawings: Figure 1 A flowchart of an IPM-based parking space detection method provided according to an embodiment of the present invention; Figure 2 This is an internal logic diagram of the parking space detection model provided in an embodiment of the present invention; Figure 3 This is an internal logic diagram of an IPM-based parking space detection system provided according to an embodiment of the present invention. Figure 4 An internal logic diagram of an electronic device provided according to an embodiment of the present invention. Detailed Implementation

[0021] In the following description, numerous specific details are set forth for illustrative purposes and to provide a thorough understanding of one or more embodiments. However, it will be apparent that these embodiments may also be implemented without these specific details. In other instances, well-known structures and devices are shown in block diagram form for ease of description of one or more embodiments.

[0022] The embodiments of this invention can be applied to electronic devices such as computer systems / servers, which can operate with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with electronic devices such as computer systems / servers include, but are not limited to: personal computer systems, server computer systems, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, networked personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0023] Computer systems / servers and other electronic devices can be described in the general context of computer-executable instructions executed by the computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers and other electronic devices can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0024] It should be understood that the terminology used in this invention is merely for describing particular embodiments and is not intended to limit the invention. Furthermore, with respect to numerical ranges in this invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Every smaller range between any stated value or intermediate value within a stated range, and any other stated value or intermediate value within said range, is also included in this invention. The upper and lower limits of these smaller ranges may be independently included or excluded from the range.

[0025] Various modifications and variations can be made to the specific embodiments described in this specification without departing from the scope or spirit of the invention, as will be apparent to those skilled in the art. Other embodiments derived from this specification will also be readily apparent to those skilled in the art. This application specification and embodiments are merely exemplary.

[0026] In the following description, numerous specific details are set forth for illustrative purposes and to provide a thorough understanding of one or more embodiments. However, it will be apparent that these embodiments may also be implemented without these specific details.

[0027] The IPM-based parking space detection method provided by this invention will be described in detail below with reference to the accompanying drawings. Figure 1 The flowchart of the parking space detection method based on IPM provided according to an embodiment of the present invention is shown, by... Figure 1 It can be seen that the parking space detection method based on IPM provided by the present invention includes: S110: Acquires the initial surround view image of the autonomous vehicle and converts the initial surround view image into an initial IPM image based on inverse perspective mapping technology.

[0028] Specifically, the initial surround view image containing all environmental information around the vehicle is captured by multiple (e.g., four) pre-set surround view fisheye cameras (front, rear, left, and right) on the autonomous vehicle; then, based on the camera calibration parameters, the initial surround view image is transformed into an initial top view image, i.e., the initial IPM image, using inverse perspective mapping technology and projection transformation. Subsequently, the parking space is detected using the initial IPM image.

[0029] S120: Optimize the initial IPM image to generate an optimized IPM feature map.

[0030] Specifically, the optimization of the initial IPM image includes color space enhancement and geometric transformation enhancement. The color space enhancement process involves: converting the initial IPM image from the RGB color space to the HSV color space; applying random perturbations to the H, S, and V channels of the initial IPM image in the HSV color space; wherein the chromaticity perturbation range is set to ±10°, the saturation scaling factor is set to 0.8~1.2, and the luminance offset is set to ±20; and converting the perturbed initial IPM image from the HSV color space back to the RGB color space. This invention, by enhancing the color space of the initial IPM image, effectively improves the robustness of subsequent models to changes in parking space color under different lighting conditions (such as cloudy days, strong light, and dusk), avoiding parking space detection failure due to differences in ambient lighting.

[0031] More specifically, the geometric transformation enhancement process includes: using the center point of the initial IPM image as a reference, performing rotational affine transformation, horizontal flip affine transformation, vertical flip affine transformation, and translational affine transformation on the initial IPM image respectively; among them, for the rotational affine transformation, it is randomly rotated within ±15° to simulate the small angular deviation when the vehicle enters the parking space; for the horizontal flip affine transformation and the vertical flip affine transformation, it is flipped left and right or up and down with a 50% probability to increase the diversity of parking space orientation; for the translational affine transformation, it is randomly translated within ±10% of the image width and height to simulate the parking space position offset caused by calibration error or vehicle pose fluctuation during IPM projection.

[0032] It should be noted that during the rotational affine transformation, horizontal flip affine transformation, vertical flip affine transformation, and translational affine transformation of the initial IPM image, bilinear interpolation is required to maintain the continuity of the initial IPM image, and boundary padding (such as mirror padding) is used to avoid information loss. This invention, by enhancing the initial IPM image through geometric transformation, can significantly expand the geometric distribution coverage of the training samples for subsequent models, alleviate the overfitting problem caused by the variable poses of parking spaces in real-world scenarios, and thus improve the generalization ability of parking space detection and matching.

[0033] In a preferred embodiment of the present invention, the process of optimizing the initial IPM image may further include: The initial IPM image is normalized to linearly scale the pixel values ​​of the initial IPM image from a preset value (such as [0,255]) to the range of [0, 1]. This operation can accelerate network convergence and reduce feature shift caused by differences in illumination intensity. The initial IPM image after normalization is downsampled by 8 times to obtain the IPM optimized feature map of the preset dimension.

[0034] Specifically, a pre-defined cascaded downsampling module can be used to perform 8x spatial downsampling on the initial IPM image after normalization. This cascaded downsampling module sequentially stacks multiple composite layers consisting of convolutional layers, batch normalization (BN), nonlinear activation functions, and pooling layers. The cascaded downsampling module includes a first stage, a second stage, a third stage, and a final stage. In the first stage, a 3×3 convolutional kernel with a stride of 2 (32 output channels) is used, followed by a BN layer and a ReLU activation function to achieve 2x downsampling of the initial IPM image after normalization. In the second stage, a 3×3 convolutional kernel with a stride of 2 (64 output channels) is used, followed by a BN layer and a ReLU activation function to achieve... The initial IPM image after normalization is downsampled by a cumulative factor of 4. In the third stage, a convolutional layer with a 3×3 kernel and a stride of 2 (outputting 128 channels) is used, followed by a BN layer and a ReLU activation function to achieve a cumulative factor of 8 downsampling of the initial IPM image after normalization. In the final stage, a 1×1 convolutional layer is used to adjust the number of channels to 128 while maintaining a spatial resolution of 96×96, thus obtaining a feature tensor with dimensions of 96×96×128 (i.e., an IPM optimized feature map of the preset dimensions).

[0035] It should be noted that, for the cascaded downsampling module, convolutional layers are used to extract local spatial features, Batch Normalization (BN) layers can alleviate internal covariate shifts and improve training stability, the ReLU activation function introduces non-linear expressive power, and convolutions with strides replace traditional pooling layers, preserving more semantic information while downsampling. Through the above 8x downsampling process, sufficient high-level semantic features are still preserved while significantly reducing computational complexity, providing highly discriminative feature representations for subsequent parking space keypoint detection or parking space foreground region regression.

[0036] S130: The pre-trained parking space detection model processes the IPM optimized feature map to determine the coordinate regions of the four corner points of the parking space, the coordinate region of the center point of the parking space, and the horizontal and vertical coordinate offsets of each pixel in the foreground area of ​​the parking space relative to the four corner points of the parking space.

[0037] Specifically, the parking space detection model is built on the Hourglass network, a deep learning architecture commonly used for tasks such as keypoint detection. This network can effectively capture multi-level information from local to global by progressively downsampling and upsampling the input image, making it suitable for tasks that require rich contextual information.

[0038] More specifically, Figure 2 To illustrate the internal logic diagram of the parking space detection model provided in this embodiment of the invention, as shown below... Figure 2 As shown, the Hourglass network used in the parking space detection model adopts a fourth-order (n=4) recursive symmetric structure. The parking space detection model includes a downsampling encoding module, a bottom-level feature extraction module, an upsampling encoding module, a multi-scale feature fusion module, and a parking space regression module. Among them, the downsampling encoding module is used to perform step-by-step downsampling processing on the IPM optimized feature map to form a first-scale downsampling layer feature map, a second-scale downsampling layer feature map, a third-scale downsampling layer feature map, and a bottom-level feature map, respectively. And extract the corresponding first-scale downsampling features, second-scale downsampling features, and third-scale downsampling features from the first-scale downsampling layer feature map, the second-scale downsampling layer feature map, and the third-scale downsampling layer feature map, respectively.

[0039] For the downsampling encoding module, the input 96×96×128 feature map (i.e., the IPM optimized feature map) first passes through a 3×3 convolutional layer to preserve local details at the original resolution; simultaneously, max pooling halves the spatial resolution to 48×48, and then it passes through a channel-expanding convolution (output channel count = 128 + 128 = ...). 256) Entering the low-resolution branch, a first-scale downsampling hierarchical feature map is formed; this process is recursively performed four times, downsampling and expanding channels step by step, thus forming the following scale sequence: the first-scale downsampling hierarchical feature map is 48×48×256; the second-scale downsampling hierarchical feature map is 24×24×384; the third-scale downsampling hierarchical feature map is 12×12×512; the bottom-level feature map is 6×6×640; among them, each level of feature map extracts hierarchical semantic features through learnable convolution, gradually abstracting from the parking space boundary to the overall layout of the parking area, so as to pre-extract the corresponding first-scale downsampling features, second-scale downsampling features and third-scale downsampling features respectively.

[0040] For the lowest-level feature extraction module, at the lowest-resolution (6×6×640) feature map, recursive downsampling is no longer performed. Instead, two consecutive 3×3 convolutional layers are used to perform non-linear transformations and channel compression on the higher-level semantics to generate the lowest-level features. It should be noted that, because the lowest-level feature map is located at the level with the largest receptive field, it can effectively integrate global contextual information, providing strong semantic priors for subsequent upsampling.

[0041] For the upsampling encoding module, starting from the bottom-level output of 6×6×128 features, the network progressively restores the spatial resolution to the original size through bilinear interpolation. Bilinear interpolation achieves smooth upsampling by weighted averaging of four neighboring pixels. Specifically, the upsampling encoding module performs progressive upsampling on the bottom-level feature map to form third-scale upsampling layer feature maps, second-scale upsampling layer feature maps, first-scale upsampling layer feature maps, and the original-size feature map; and extracts the corresponding first-scale upsampling features, second-scale upsampling features, and third-scale upsampling features from the first-scale upsampling layer feature map, second-scale upsampling layer feature map, and third-scale upsampling layer feature map, respectively.

[0042] For the multi-scale feature fusion module, after each level of upsampling, the network adds and fuses the current feature with the high-resolution branch features corresponding to the downsampling path element-wise. This mechanism enables the network to inject deep semantic information while restoring spatial details, significantly improving robustness to complex scenes such as occlusion and blur. Specifically, the multi-scale feature fusion module is used to fuse the first-scale upsampled features and the first-scale downsampled features to form the first-scale fused feature, fuse the second-scale upsampled features and the second-scale downsampled features to form the second-scale fused feature, and fuse the third-scale upsampled features and the third-scale downsampled features to form the third-scale fused feature.

[0043] The parking space regression module is used to regress the coordinate regions of the four corner points of the parking space, the coordinate region of the center point of the parking space, and the horizontal and vertical coordinate offsets of each pixel in the foreground area of ​​the parking space relative to the four corner points of the parking space, based on the first-scale fusion features, the second-scale fusion features, the third-scale fusion features, and the lowest-level features.

[0044] It's important to note that in the Hourglass network, the downsampling and upsampling parts are symmetrical, ensuring the network can fully capture information at different levels. This structure helps the Hourglass network learn global and local features, thus improving its sensitivity to details. Furthermore, the Hourglass network can interact multiple times with features at different scales to capture a wide range of contextual information.

[0045] Furthermore, it should be noted that the parking space detection model based on the Hourglass network as the backbone provided by this invention needs to be trained, tested, and validated in advance using historical data to ensure that the parking space detection model reaches the preset accuracy. After the parking space detection model reaches the preset accuracy, the IPM optimized feature map is then processed by the parking space detection model to determine the coordinate regions of the four corner points of the parking space, the coordinate regions of the center point of the parking space, and the horizontal and vertical coordinate offsets of each pixel in the foreground area of ​​the parking space relative to the four corner points of the parking space.

[0046] In a specific embodiment of the present invention, the network output mainly includes four branches, which respectively output the regions belonging to the four corner points of the parking space, the region belonging to the center point of the parking space, the horizontal and vertical coordinate offsets of all parking space foreground regions relative to the four corner points of the parking space, and the semantic labels of the parking space foreground occupancy attribute. The regions belonging to the four corner points and the region belonging to the center point of the parking space output by the first two branches are Gaussian heatmaps with a standard deviation of 8. The actual positions of the corner points and the center point of the parking space are both within the corresponding heatmaps. As the distance increases, the heatmap values ​​gradually decrease. The actual position of the center point of the parking space is the mean of the horizontal and vertical coordinates of the actual positions of the four corner points of the parking space. The third branch is an offset branch, which is the horizontal and vertical coordinate deviation of the region enclosed by the four corner points of all parking spaces in the initial IPM image (i.e., the parking space foreground region) relative to the four corner points, and is normalized, resulting in a total of 8 channels. The fourth branch is the parking space occupancy attribute branch. If the parking space attribute is occupied, the semantic segmentation category is 1; if the parking space is available, the semantic segmentation category is 0.

[0047] S140: Determine the neighborhood of the center point of the parking space from the foreground area of ​​the parking space; and, based on the horizontal and vertical coordinate offsets of each pixel in the neighborhood of the center point of the parking space relative to the four corner points of the parking space, determine the true coordinate positions of the four corner points of the parking space from the area to which the coordinates of the four corner points of the parking space belong.

[0048] Specifically, the process of determining the neighborhood of the center point of the parking space from the foreground area of ​​the parking space includes: Determine the selected coordinate position of the parking space center point from the region to which the coordinates of the parking space center point belong; Based on the selected coordinates of the parking space center point, the neighborhood of the parking space center point is determined from the foreground area of ​​the parking space.

[0049] More specifically, the process of determining the true coordinate positions of the four corner points of a parking space from the region to which the coordinates of the four corner points belong, based on the horizontal and vertical coordinate offsets of each pixel within the neighborhood of the center point of the parking space relative to the four corner points, includes: By calculating the horizontal and vertical coordinate offsets of each pixel in the neighborhood of the parking space center point relative to the four corner points of the parking space, the coordinates of the four corner points that match the parking space center point are calculated. Based on the regression point coordinates of the four corner points that match the center point of the parking space, the actual coordinate positions of the four corner points of the parking space are determined within the area to which the coordinates of the four corner points of the parking space belong.

[0050] In the actual processing, firstly, the maximum value point of the feature map of the parking space center point is extracted from the region to which the center point of the parking space belongs, and used as the selected coordinate position of the parking space center point. Specifically, if the confidence threshold is greater than 0.4 and the center pixel is greater than the confidence of the surrounding eight neighbors, then the point is judged as a maximum value point and used as a candidate parking space center point. Then, a 3*3 window is selected from the foreground region of the parking space center point as the neighborhood region. Finally, the coordinates of the four corner points are regressed according to the offset matrix, and the average of the nine positions with a confidence greater than a preset threshold is taken as the regression point coordinates. The maximum value points of the feature map of the four corner points in the corresponding region are selected as candidate corner points, and the process is the same as that of selecting the maximum value of the center point. Corner points are searched within a circle with a radius of a set threshold (e.g., 40), and the corner point with the highest confidence among all the found corner points is taken as the true coordinate position of the parking space corner point. If a corner point is found, the true coordinate position of the corner point is output; otherwise, the regression point is output as the true coordinate position of the corner point.

[0051] It should be noted that in determining the actual positions of the four corner points of the parking space, it is necessary to calculate the confidence level of all foreground pixels within the regression area and determine the proportion of pixels whose confidence levels meet a certain threshold to the total number of pixels in the regression area. If the proportion is greater than the preset threshold, the parking space is retained; otherwise, it is deleted as an invalid parking space. Furthermore, if all four corner points of the parking space are detected, and the parking space shape does not conform to a parallelogram, it is assumed that only one corner point is misaligned. The process is repeated four times clockwise, fixing three corner points in turn, selecting the corner point that best fits the parallelogram from the candidate corner points, and then selecting the optimal shape for output. The condition for determining whether a parallelogram is satisfied is: if the sum of two sets of corresponding angles is between 175° and 185°, it is judged as a parallelogram.

[0052] Furthermore, the parking space detection method based on IPM provided by the present invention also includes: processing the IPM optimized feature map through a parking space detection model to determine the occupancy label of the parking space foreground area; and determining the occupancy status of the corresponding parking space based on the occupancy label of the parking space foreground area.

[0053] Specifically, to predict the occupancy status of parking spaces, the network output may also include a parking space semantic segmentation branch. This branch contains one channel with two categories: parking space foreground and parking space background. The occupancy label for the parking space foreground is 1, and the occupancy label for the parking space background is 0. When the occupancy label is 1, it means that the parking space is occupied; when the occupancy label is 0, it means that the parking space is not occupied.

[0054] As can be seen from the above specific embodiments, the parking space detection method based on IPM provided by the present invention has at least the following advantages: 1. Improved accuracy of parking space corner points in challenging scenarios: Predicts the offset of the foreground area of ​​all parking spaces relative to the four corner points of the parking space. Compared to predicting only the center point, it can provide richer positional information, making corner point positioning more accurate and better restoring the actual boundary of the parking space. Instead of relying solely on the prediction of the center point, it reduces the problem of inaccurate corner point positioning caused by the prediction error of the center point and improves the accuracy of corner point prediction. 2. Improve the accuracy of parking space attributes: By predicting the foreground and background of parking spaces to determine whether they are occupied, this method is more robust than existing keypoint classification and can improve the identification of parking space attributes. 3. Improved model training efficiency: By downsampling the data by eight times, training efficiency can be improved and the model convergence speed can be accelerated.

[0055] The parking space detection system 400 based on IPM provided by this invention can be installed in electronic devices. Depending on the functions implemented, such as... Figure 3 As shown, the IPM-based parking space detection system 400 may include: an image acquisition module 410, an image processing module 420, a model processing module 430, and a parking space post-processing module 440. The unit described in this invention can also be called a module, which mainly refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0056] In this embodiment, the functions of each module / unit are as follows: Image acquisition module 410 is used to acquire the initial surround view image of the autonomous vehicle and convert the initial surround view image into an initial IPM image based on inverse perspective mapping technology. Image processing module 420 is used to optimize the initial IPM image to generate an optimized IPM feature map; The model processing module 430 is used to process the IPM optimized feature map through the pre-trained parking space detection model to determine the coordinate regions of the four corner points of the parking space, the coordinate regions of the center point of the parking space, and the horizontal and vertical coordinate offsets of each pixel in the foreground area of ​​the parking space relative to the four corner points of the parking space. The parking space post-processing module 440 is used to determine the neighborhood area of ​​the parking space center point from the foreground area of ​​the parking space; and, based on the horizontal and vertical coordinate offsets of each pixel point in the neighborhood area of ​​the parking space center point relative to the four corner points of the parking space, it determines the real coordinate positions of the four corner points of the parking space from the area to which the coordinates of the four corner points belong.

[0057] Specifically, the parking space post-processing module also includes a parking space occupancy status detection module; wherein, the model processing module is also used to process the IPM optimized feature map to determine the occupancy label of the foreground area of ​​each parking space; the parking space occupancy status detection module is used to determine the occupancy status of each parking space based on the occupancy label of the foreground area of ​​each parking space.

[0058] It should be noted that the above-described embodiments of the IPM-based parking space detection system and the IPM-based parking space detection method can be used for reference, and will not be elaborated further here.

[0059] Figure 4 This is a schematic diagram of the electronic device used in the IPM-based parking space detection system of the present invention.

[0060] like Figure 4 As shown, the electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as an IPM-based parking space detection program 12.

[0061] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as the portable hard drive of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1. Furthermore, the memory 11 can include both internal and external storage units of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as program code, but also to temporarily store data that has been output or will be output.

[0062] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., an IPM-based parking space detection program) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0063] The bus can be a Peripheral Component Interconnect Standard (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0064] Figure 4 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 4 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0065] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0066] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0067] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or, optionally, a standard wired interface or a wireless interface. In some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an organic light-emitting diode touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0068] It should be understood that the embodiments described are for illustrative purposes only and do not limit the scope of protection of the present invention.

[0069] The parking space detection program 12 based on IPM stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When it is run in the processor 10, it can realize the parking space detection method based on IPM.

[0070] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory.

[0071] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0072] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0073] As per the above reference Figures 1 to 4 The IPM-based parking space detection method, system, and electronic device according to the present invention are described by way of example. However, those skilled in the art should understand that various modifications can be made to the IPM-based parking space detection method, system, and electronic device proposed in the present invention without departing from the scope of the invention. Therefore, the scope of protection of the present invention should be determined by the contents of the appended claims.

Claims

1. A parking space detection method based on IPM, characterized in that, include: Acquire initial surround view images of autonomous vehicles, and convert the initial surround view images into initial IPM images based on inverse perspective mapping technology; The initial IPM image is optimized to generate an optimized IPM feature map; The pre-trained parking space detection model is used to process the IPM optimized feature map to determine the coordinate regions of the four corner points of the parking space, the coordinate region of the center point of the parking space, and the horizontal and vertical coordinate offsets of each pixel in the foreground area of ​​the parking space relative to the four corner points of the parking space. The center point of the parking space is determined from the region to which the coordinates of the center point of the parking space belong, and the neighborhood region of the center point of the parking space is determined from the foreground region of the parking space; and, based on the horizontal and vertical coordinate offsets of each pixel point in the center point of the parking space and its neighborhood region relative to the four corner points of the parking space, the true coordinate positions of the four corner points of the parking space are determined from the region to which the coordinates of the four corner points of the parking space belong.

2. The parking space detection method based on IPM as described in claim 1, characterized in that, Also includes: The parking space detection model is used to process the IPM optimized feature map to determine the occupancy label of the foreground area of ​​the parking space. Based on the occupancy label of the foreground area of ​​the parking space, the occupancy status of the corresponding parking space is determined.

3. The parking space detection method based on IPM as described in claim 2, characterized in that, Optimization processing of the initial IPM image includes color space enhancement; wherein, the color space enhancement process includes: The initial IPM image is converted from the RGB color space to the HSV color space; Random perturbations are applied to the initial IPM image in the H, S, and V channels of the HSV color space, respectively. The perturbated initial IPM image is converted from the HSV color space to the RGB color space.

4. The parking space detection method based on IPM as described in claim 3, characterized in that, Optimization processing of the initial IPM image further includes geometric transformation enhancement; wherein, the process of geometric transformation enhancement includes: Using the center point of the initial IPM image as a reference, the initial IPM image is subjected to rotational affine transformation, horizontal flip affine transformation, vertical flip affine transformation, and translational affine transformation, respectively; wherein, During the rotational affine transformation, horizontal flip affine transformation, vertical flip affine transformation, and translation affine transformation of the initial IPM image, the continuity of the initial IPM image is maintained by a bilinear interpolation algorithm.

5. The parking space detection method based on IPM as described in claim 4, characterized in that, The process of optimizing the initial IPM image also includes: The initial IPM image is normalized to linearly scale the pixel values ​​of the initial IPM image to the [0,1] range; The initial IPM image after normalization is spatially downsampled by 8 times to obtain an IPM optimized feature map of a preset dimension.

6. The parking space detection method based on IPM as described in claim 5, characterized in that, The parking space detection model is built based on the Hourglass network; and the model includes a downsampling encoding module, a bottom-level feature extraction module, an upsampling encoding module, a multi-scale feature fusion module, and a parking space regression module; wherein... The downsampling encoding module is used to perform three levels of spatial downsampling on the input IPM optimized feature map, generating downsampled features at the first, second, and third scales respectively, and further downsampling to the lowest level feature map; wherein, the downsampled features at each scale are cached after generation for subsequent cross-path feature fusion. The lowest-level feature extraction module performs high-level semantic modeling on the lowest-level feature map through non-linear convolution and compresses the channel dimension to generate the lowest-level features with strong context awareness. The upsampling decoding module starts from the lowest level feature and performs three levels of upsampling operations in sequence. It uses bilinear interpolation to gradually restore the spatial resolution, and obtains upsampled features at the third scale, second scale and first scale in sequence, and finally restores them to the original input size. The multi-scale feature fusion module is used to immediately add the current upsampled feature to the downsampled feature of the corresponding scale in the downsampled encoding module element by element after each upsampling level, so as to realize cross-path feature fusion. The parking space regression module is used to regress the coordinate regions of the four corner points of the parking space, the coordinate region of the center point of the parking space, the horizontal and vertical coordinate offsets of each pixel in the foreground region of the parking space relative to the four corner points of the parking space, and the semantic labels of the foreground region of the parking space.

7. The parking space detection method based on IPM as described in claim 6, characterized in that, The process of determining the neighborhood region of the center point of the parking space from the foreground region of the parking space includes: The selected coordinate position of the center point of the parking space is determined from the region to which the coordinates of the center point of the parking space belong; Based on the selected coordinates of the center point of the parking space, the neighborhood of the center point of the parking space is determined from the foreground area of ​​the parking space.

8. The parking space detection method based on IPM as described in claim 7, characterized in that, The process of determining the true coordinate positions of the four corner points of the parking space from the region to which the coordinates of the four corner points belong, based on the horizontal and vertical coordinate offsets of each pixel in the neighborhood of the center point of the parking space relative to the four corner points of the parking space, includes: By calculating the horizontal and vertical coordinate offsets of each pixel in the neighborhood of the parking space center point relative to the four corner points of the parking space, the regression point coordinates of the four corner points that match the parking space center point are calculated. Based on the regression point coordinates of the four corner points that match the center point of the parking space, the true coordinate positions of the four corner points of the parking space are determined from the area to which the coordinates of the four corner points of the parking space belong.

9. A parking space detection system based on IPM, characterized in that, The parking space detection system includes: an image acquisition module, an image processing module, a model processing module, and a parking space post-processing module; wherein... The image acquisition module is used to acquire the initial surround view image of the autonomous vehicle and convert the initial surround view image into an initial IPM image based on inverse perspective mapping technology. The image processing module is used to optimize the initial IPM image to generate an optimized IPM feature map; The model processing module is used to process the IPM optimized feature map through the pre-trained parking space detection model to determine the coordinate regions of the four corner points of the parking space, the coordinate regions of the center point of the parking space, and the horizontal and vertical coordinate offsets of each pixel in the foreground area of ​​the parking space relative to the four corner points of the parking space. The parking space post-processing module is used to determine the parking space center point from the area to which the coordinates of the parking space center point belong, and to determine the neighborhood area of ​​the parking space center point from the foreground area of ​​the parking space; and, based on the horizontal and vertical coordinate offsets of each pixel point in the parking space center point and its neighborhood area relative to the four corner points of the parking space, to determine the true coordinate positions of the four corner points of the parking space from the area to which the coordinates of the four corner points of the parking space belong.

10. The parking space detection system as described in claim 9, characterized in that, The parking space post-processing module also includes a parking space occupancy status detection module; wherein... The model processing module is also used to process the IPM optimized feature map to determine the occupancy label of the foreground area of ​​each parking space. The parking space occupancy detection module is used to determine the occupancy status of each parking space based on the occupancy label in the foreground area of ​​each parking space.

Images