Parking space direction correction method and system based on parking space identification

Abstract

The invention discloses a parking space direction correction method and system based on parking space recognition, and belongs to the technical field of parking space recognition. The method comprises the following steps: performing feature extraction on a parking space environment image through a multi-task model to obtain a parking space line mask image and a plurality of groups of parking space line angular point recognition results of the parking space line mask image; and then, the parking space line mask image is segmented through parking space line angular point identification results to obtain a parking space line binary mask corresponding to each group of parking space line angular point identification results, so that skeleton center line extraction can be carried out on skeleton center lines of the parking space line binary masks, and the skeleton center lines of the parking space line binary masks can be extracted. The parking space correction angle value of the parking space line binary mask is determined according to the skeleton center line, the parking space line angular point recognition result is corrected according to the parking space correction angle value of the parking space line binary mask, and the accuracy of parking space recognition is improved.

Description

Technical Field

[0001] This invention belongs to the field of parking space recognition technology, and particularly relates to a parking space direction correction method and system based on parking space recognition. Background Technology

[0002] In modern intelligent driving systems, parking space recognition is the cornerstone of assisted driving functions, and its importance is increasingly prominent. As assisted driving functions evolve from highway navigation-assisted driving to a full-scenario, continuous experience, automatic parking has become a crucial indicator of intelligent driving capabilities. This function requires the vehicle to autonomously complete the entire process from any starting parking space to the target parking space, seamlessly connecting public road driving with the final parking maneuver. In this closed-loop experience, accurate and reliable parking space recognition is the fundamental prerequisite for successfully executing automatic parking operations; it directly determines whether the system can safely and efficiently find and utilize available space.

[0003] Current parking space recognition solutions mostly rely on a pure vision approach using four-way surround-view fisheye cameras. This method acquires images of parking spaces around the vehicle using the cameras and leverages the computing power of an in-vehicle embedded platform to identify the parking spaces. However, the current parking space markings are complex and varied, including irregularly shaped parking spaces with weakened directional features such as I-shapes, short T-shapes, and arcs. The single-task model on the in-vehicle embedded platform struggles to effectively capture clear directional cues, resulting in significant fitting errors in the parking space line directions and hindering accurate parking space recognition. Summary of the Invention

[0004] This invention aims to provide a parking space orientation correction method and system based on parking space recognition. By segmenting the parking space line mask image based on the parking space line corner point recognition results, a binary mask of the parking space line is obtained. The parking space orientation is corrected by determining the parking space correction angle value through the skeleton center line of the binary mask of the parking space line, thereby improving the accuracy of parking space recognition.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a parking space orientation correction method based on parking space recognition, comprising: The system collects images of the parking space environment of the vehicle and extracts features from the images based on a preset multi-task model to obtain a parking line mask image and several sets of parking line corner point recognition results from the parking line mask image. The parking line mask image is segmented based on the corner point recognition results of each group of parking lines to obtain the binary mask of the parking lines corresponding to each group of corner point recognition results. Extract the skeleton centerline of each parking space line binary mask, and determine the parking space correction angle value of each parking space line binary mask based on the skeleton centerline; The parking space line corner point identification result is corrected based on the parking space correction angle value to complete the parking space direction correction of the vehicle.

[0006] Understandably, compared to existing technologies, this invention extracts features from the parking space environment image using a multi-task model to obtain a parking line mask image and several sets of parking line corner point recognition results from the parking line mask image. Then, the parking line mask image is segmented using the parking line corner point recognition results to obtain a parking line binary mask corresponding to each set of parking line corner point recognition results. This allows for the extraction of the skeleton centerline of the parking line binary mask, determining the parking space correction angle value of the parking line binary mask using the skeleton centerline, and then correcting the parking line corner point recognition results using the parking space correction angle value of the parking line binary mask. This invention achieves the correction of parking line corner point recognition results through parking line mask image segmentation and the parking space correction angle value determined by the skeleton centerline, avoiding the direct use of the model's output parking space recognition results in traditional solutions and improving the accuracy of parking space recognition.

[0007] As a preferred embodiment, the process involves acquiring images of the parking space environment of the vehicle, extracting features from the images based on a preset multi-task model to obtain a parking line mask image, and several sets of parking line corner point recognition results from the parking line mask image, including: The pre-defined multi-task model includes: a shared feature extraction layer, a parking line segmentation branch, and a parking space recognition branch; Acquire images of the parking space environment of the vehicle, and perform perspective transformation on the images to obtain a bird's-eye view of the parking space environment. The bird's-eye view is input into the multi-task model so that the shared feature extraction layer, the parking line segmentation branch, and the parking space recognition branch can extract features from the bird's-eye view to obtain a parking line mask image and several sets of parking line corner point recognition results from the parking line mask image.

[0008] This preferred solution converts parking space environment images into bird's-eye views through perspective transformation, unifying distorted and inconsistent parking lines from different perspectives into a top-down standard coordinate system, providing an accurate data foundation for feature extraction in the multi-task model. Subsequently, by setting the multi-task model to include a shared feature extraction layer, a parking line segmentation branch, and a parking space recognition branch, accurate feature extraction from the bird's-eye view can be achieved through the synergy of these branches. Compared to a single-task model, this approach offers more discriminative and comprehensive representation, particularly with a stronger ability to capture key local features of irregularly shaped parking spaces with weakened orientation, thereby improving the accuracy of parking space recognition.

[0009] As a preferred embodiment, the bird's-eye view is input into the multi-task model, so that the shared feature extraction layer, the parking line segmentation branch, and the parking space recognition branch perform feature extraction on the bird's-eye view to obtain a parking line mask image and several sets of parking line corner point recognition results from the parking line mask image, including: The bird's-eye view is input into the shared feature extraction layer of the multi-task model, so that the shared feature extraction layer can extract features from the bird's-eye view to obtain the feature map of the bird's-eye view. The feature map is input into the parking line segmentation branch, so that the parking line segmentation branch performs convolution, activation and upsampling operations on the feature map to obtain a first feature map corresponding to the feature map; and a nonlinear transformation is performed on the first feature map based on a preset first activation function to obtain a parking line mask image; The feature map is input into the parking space recognition branch, so that the parking space recognition branch performs convolution, activation and upsampling operations on the feature map to obtain a second feature map corresponding to the feature map; and performs nonlinear transformation on the second feature map based on a preset first activation function and a preset second activation function to obtain several sets of parking space line corner point recognition results of the parking space line mask image.

[0010] This preferred solution extracts features from the bird's-eye view by sharing a feature extraction layer, enabling the parking line segmentation branch and the parking space recognition branch to have the same data input, thus achieving resource reuse and feature consistency. Subsequently, through the operations of the parking line segmentation branch and the parking space recognition branch, the problem that a single task model cannot effectively capture clear directional clues is avoided, resulting in a more discriminative and comprehensive representation. In particular, it has a stronger ability to capture key local features of irregularly shaped parking spaces with weakened orientation, thereby improving the accuracy of parking space recognition.

[0011] As a preferred embodiment, the step of segmenting the parking line mask image based on each group of parking line corner point recognition results to obtain the parking line binary mask corresponding to each group of parking line corner point recognition results includes: The results of each parking space line corner point recognition include: the coordinates of the parking space line corner point and the angle and direction of the parking space line; Starting from the coordinates of the parking space line corner point, extend a preset first pixel unit in the direction of the parking space line angle corresponding to the coordinates of the parking space line corner point to determine the center line of each group of parking space line corner point recognition results; Extend the center line to both sides by a preset second pixel unit to determine the rectangular area of ​​each group of parking space line corner point recognition results; The ROI region of each group of parking space line corner point recognition results is determined based on the rectangular region of each group of parking space line corner point recognition results; Based on the ROI region of each group of parking line corner point recognition results, the parking line mask image is segmented to obtain the parking line binary mask image corresponding to each group of parking line corner point recognition results. A morphological closing operation is performed on the binary mask image of the parking line corresponding to each group of parking line corner point recognition results to obtain the binary mask of the parking line corresponding to each group of parking line corner point recognition results.

[0012] This preferred solution extends the centerline by using the corner coordinates and angular direction of the parking line. Then, it expands to both sides of the centerline to form a rectangular area, which is further defined as the Region of Interest (ROI). The ROI is then segmented, and morphological closing operations are used to improve the completeness, connectivity, and accuracy of the binary mask of the parking line. This improves the accuracy of the subsequently extracted skeleton centerline, thereby improving the accuracy of parking space correction and parking space recognition.

[0013] As a preferred embodiment, the step of extracting the skeleton centerline of each parking space line binary mask and determining the parking space correction angle value of each parking space line binary mask based on the skeleton centerline includes: The skeleton center line of each parking space line binary mask is extracted based on a preset binary image skeletonization algorithm, and the two-dimensional point set of each parking space line binary mask is determined based on the skeleton center line. Angle fitting is performed on the two-dimensional point set of each parking space line binary mask to determine the parking space correction angle value of each parking space line binary mask.

[0014] This preferred solution extracts the center line of the skeleton of each parking space line binary mask using a binary image skeletonization algorithm, obtaining a two-dimensional point set of the parking space line binary mask. Angle fitting is then performed based on this two-dimensional point set to determine the parking space correction angle value. Fitting using the skeleton center line avoids the problem of the original mask potentially containing numerous irregular protrusions and noise, thus obtaining a more accurate parking space correction angle value, improving the accuracy of parking space correction, and enhancing the accuracy of parking space recognition.

[0015] As a preferred embodiment, the step of performing angle fitting on the two-dimensional point set of each parking space line binary mask to determine the parking space correction angle value of each parking space line binary mask includes: Based on the coordinates of each skeleton point in the two-dimensional point set, obtain the geometric center of each two-dimensional point set; Based on the geometric center of each of the two-dimensional point sets, each skeleton point in each of the two-dimensional point sets is centered to obtain the centered coordinates of each skeleton point in each of the two-dimensional point sets. Based on the centered coordinates of each skeleton point in each of the two-dimensional point sets, construct the covariance matrix of each of the two-dimensional point sets; The covariance matrix is ​​solved based on the principal component analysis algorithm to determine several eigenvalues ​​of each two-dimensional point set and the eigenvector of each eigenvalue; From a plurality of feature values ​​of each two-dimensional point set, select the first feature value with the largest feature value, and determine the feature angle value of each two-dimensional point set based on the feature vector corresponding to the first feature value; Based on the feature angle values ​​of each of the two-dimensional point sets, the parking space correction angle value of each parking space line binary mask is determined.

[0016] This preferred solution eliminates the influence of the absolute position of the two-dimensional point set in the image by centering the two-dimensional point set, thereby improving the accuracy of the eigenvalues ​​and eigenvectors obtained based on the centered coordinates. By selecting the first eigenvalue with the largest eigenvalue, the direction with the largest distribution variance in the two-dimensional point set can be displayed, thus obtaining the eigenvector corresponding to the first eigenvalue that displays the main axis direction of the parking space line. This improves the accuracy of the eigenvalue angle and the parking space correction angle, thereby improving the accuracy of parking space correction and parking space recognition.

[0017] As a preferred embodiment, the step of correcting the parking space line corner point recognition result based on the parking space correction angle value to complete the parking space orientation correction of the vehicle includes: Replace the corresponding parking space angle direction with the parking space correction angle value to correct the parking space line angle direction of the parking space line corner point recognition result; Based on the parking space angle direction after the corner point coordinates of the parking space line are corrected, the parking space direction correction result of the vehicle is determined, and the parking space direction correction of the vehicle is completed.

[0018] This preferred solution corrects the parking space recognition by replacing the corresponding parking space angle direction with the parking space correction angle value, thus avoiding the problem of inaccurate parking space recognition results output by the model in traditional technical solutions and improving the accuracy of parking space recognition.

[0019] Accordingly, this invention provides a parking space orientation correction system based on parking space recognition, including: a parking space recognition module, an image segmentation module, a parking space correction angle value acquisition module, and a parking space orientation correction module; The parking space recognition module is used to collect images of the parking space environment of the vehicle, and to extract features from the parking space environment image based on a preset multi-task model to obtain a parking space line mask image and several sets of parking space line corner point recognition results of the parking space line mask image. The image segmentation module is used to segment the parking line mask image based on each group of parking line corner point recognition results to obtain the parking line binary mask corresponding to each group of parking line corner point recognition results; The parking space correction angle value acquisition module is used to extract the skeleton center line of each parking space line binary mask, and determine the parking space correction angle value of each parking space line binary mask based on the skeleton center line. The parking space orientation correction module is used to correct the parking space line corner point recognition result based on the parking space correction angle value, thereby completing the parking space orientation correction of the vehicle.

[0020] As a preferred embodiment, the parking space recognition module includes: a parking space recognition unit; In the parking space recognition unit, the preset multi-task model includes: a shared feature extraction layer, a parking line segmentation branch, and a parking space recognition branch; The parking space recognition unit is used to collect images of the parking space environment of the vehicle and perform perspective transformation on the parking space environment images to obtain a bird's-eye view of the parking space environment images. The bird's-eye view is input into the multi-task model so that the shared feature extraction layer, the parking line segmentation branch, and the parking space recognition branch can extract features from the bird's-eye view to obtain a parking line mask image and several sets of parking line corner point recognition results from the parking line mask image.

[0021] As a preferred embodiment, the parking space recognition unit includes: a parking space recognition subunit; The parking space recognition subunit is used to input the bird's-eye view into the shared feature extraction layer of the multi-task model, so that the shared feature extraction layer can extract features from the bird's-eye view to obtain the feature map of the bird's-eye view. The feature map is input into the parking line segmentation branch, so that the parking line segmentation branch performs convolution, activation and upsampling operations on the feature map to obtain a first feature map corresponding to the feature map; and a nonlinear transformation is performed on the first feature map based on a preset first activation function to obtain a parking line mask image; The feature map is input into the parking space recognition branch, so that the parking space recognition branch performs convolution, activation and upsampling operations on the feature map to obtain a second feature map corresponding to the feature map; and performs nonlinear transformation on the second feature map based on a preset first activation function and a preset second activation function to obtain several sets of parking space line corner point recognition results of the parking space line mask image.

[0022] Understandably, compared to existing technologies, this system extracts features from the parking space environment image using a multi-task model to obtain a parking line mask image and several sets of parking line corner point recognition results from the parking line mask image. Then, it segments the parking line mask image using the parking line corner point recognition results to obtain a binary mask corresponding to each set of parking line corner point recognition results. This allows for the extraction of the skeleton centerline of the binary mask, determining the parking space correction angle value of the binary mask, and ultimately correcting the parking line corner point recognition results using this correction angle value. This invention achieves the correction of parking line corner point recognition results through parking line mask image segmentation and the parking space correction angle value determined by the skeleton centerline, avoiding the direct use of the model's output parking space recognition results in traditional solutions and improving the accuracy of parking space recognition. Attached Figure Description

[0023] Figure 1 A flowchart illustrating the steps of a parking space orientation correction method based on parking space recognition, provided in an embodiment of the present invention; Figure 2 A schematic diagram of a multi-task model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a parking space orientation correction system based on parking space recognition, provided as an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Example 1 Please refer to Figure 1 , Figure 1 The flowchart of a parking space orientation correction method based on parking space recognition provided in this embodiment of the invention includes steps S101 to S104.

[0026] Step S101: Collect parking space environment images of the vehicle, and extract features from the parking space environment images based on a preset multi-task model to obtain parking space line mask images and several sets of parking space line corner point recognition results of the parking space line mask images.

[0027] Step S102: Segment the parking line mask image based on the corner point recognition results of each group of parking lines to obtain the binary mask of the parking lines corresponding to each group of corner point recognition results.

[0028] Step S103: Extract the skeleton centerline of each parking space line binary mask, and determine the parking space correction angle value of each parking space line binary mask based on the skeleton centerline.

[0029] Step S104: Correct the parking space line corner point identification result based on the parking space correction angle value to complete the parking space direction correction of the vehicle.

[0030] This embodiment extracts features from the parking space environment image using a multi-task model to obtain a parking line mask image and several sets of parking line corner point recognition results from the parking line mask image. Then, the parking line mask image is segmented using the parking line corner point recognition results to obtain a binary mask corresponding to each set of parking line corner point recognition results. This allows for the extraction of the skeleton centerline of the binary mask, determining the parking space correction angle value of the binary mask, and ultimately correcting the parking line corner point recognition results using this correction angle value. This invention achieves the correction of parking line corner point recognition results through parking line mask image segmentation and the parking space correction angle value determined by the skeleton centerline, avoiding the direct use of the model's output parking space recognition results in traditional solutions and improving the accuracy of parking space recognition.

[0031] In this embodiment, the process of acquiring parking space environment images of vehicles and extracting features from these images based on a preset multi-task model to obtain parking line mask images and several sets of parking line corner point recognition results from the parking line mask images includes: The pre-defined multi-task model includes: a shared feature extraction layer, a parking line segmentation branch, and a parking space recognition branch; Acquire images of the parking space environment of the vehicle, and perform perspective transformation on the images to obtain a bird's-eye view of the parking space environment. The bird's-eye view is input into the multi-task model so that the shared feature extraction layer, the parking line segmentation branch, and the parking space recognition branch can extract features from the bird's-eye view to obtain a parking line mask image and several sets of parking line corner point recognition results from the parking line mask image.

[0032] In one optional embodiment, an image of the parking space environment is acquired by an in-vehicle surround-view fisheye camera system, and then the parking space environment image is converted into a bird's-eye view through perspective transformation. Since the perspective transformation technology has been maturely applied to the current parking space recognition scheme, this embodiment will not elaborate further.

[0033] In one alternative embodiment, please refer to Figure 2 , Figure 2 This is a schematic diagram of a multi-task model provided in an embodiment of the present invention; as shown below. Figure 2 As shown, the multi-task model includes: a shared feature extraction layer, a parking line segmentation branch, and a parking space recognition branch. The shared feature extraction layer is a lightweight backbone network, which can use networks such as MobileNetV3-SE or EfficientNet-Lite. The parking line segmentation branch includes three consecutive convolution, activation, and upsampling operations, ultimately outputting the result after a non-linear transformation using the Sigmoid function. The parking space recognition branch includes one convolution, activation, and upsampling operation, ultimately outputting the result after a non-linear transformation using the Sigmoid and Tanh functions. Furthermore, the training process of the pre-defined multi-task model includes: acquiring inverse perspective transformation images of the parking scene, labeling the inverse perspective transformation images of the parking scene, and using labels including: several sets of parking line corner point recognition results and semantic segmentation masks; each set of parking line corner point recognition results includes: parking line corner point coordinates and parking line angle direction; then setting the loss function of the parking line segmentation branch to BCEWithLogitsLoss (binary cross-entropy loss); and the loss function of the parking space recognition branch to SmoothL1Loss (smoothing L1 loss); simultaneously assigning a weight of 0.3 to the parking line segmentation branch and 0.7 to the parking space recognition branch. The weights are calculated as follows: before the number of training iterations is less than 10, the total weight of the multi-task model is 0.7 × the loss of the parking space recognition branch + 0.3 × the loss of the parking line segmentation branch. After the number of training iterations is less than 10, a sliding window of length 10 is used to calculate the historical moving average of each loss in the parking line segmentation branch and the parking space recognition branch, and the current loss is normalized by dividing it by its moving average. Then, the two normalized losses are weighted and summed with a weight of 0.5:0.5 to obtain the total loss. The training termination target is set as follows: the total loss of the validation set does not decrease for 15 consecutive epochs (training rounds) or reaches 300 epochs.

[0034] This embodiment converts parking space environment images into bird's-eye views through perspective transformation, unifying distorted and varying-scale parking lines from different viewpoints into a top-down standard coordinate system, providing an accurate data foundation for feature extraction in the multi-task model. Subsequently, by setting the multi-task model to include a shared feature extraction layer, a parking line segmentation branch, and a parking space recognition branch, accurate feature extraction from the bird's-eye view can be achieved through the synergy of these branches. Compared to a single-task model, it offers more discriminative and comprehensive representation, particularly with a stronger ability to capture key local features of irregularly shaped parking spaces with weakened orientation, thereby improving the accuracy of parking space recognition.

[0035] In this embodiment, the step of inputting the bird's-eye view into the multi-task model, so that the shared feature extraction layer, the parking line segmentation branch, and the parking space recognition branch can perform feature extraction on the bird's-eye view to obtain a parking line mask image, and several sets of parking line corner point recognition results of the parking line mask image, including: The bird's-eye view is input into the shared feature extraction layer of the multi-task model, so that the shared feature extraction layer can extract features from the bird's-eye view to obtain the feature map of the bird's-eye view. The feature map is input into the parking line segmentation branch, so that the parking line segmentation branch performs convolution, activation and upsampling operations on the feature map to obtain a first feature map corresponding to the feature map; and a nonlinear transformation is performed on the first feature map based on a preset first activation function to obtain a parking line mask image; The feature map is input into the parking space recognition branch, so that the parking space recognition branch performs convolution, activation and upsampling operations on the feature map to obtain a second feature map corresponding to the feature map; and performs nonlinear transformation on the second feature map based on a preset first activation function and a preset second activation function to obtain several sets of parking space line corner point recognition results of the parking space line mask image.

[0036] In one optional embodiment, a 640×640×3 bird's-eye view is input into the shared feature extraction layer of a multi-task model to obtain a multi-scale feature map pyramid, i.e., the feature map of the bird's-eye view. Then, the feature map is input into the parking line segmentation branch, and through three consecutive convolution, activation, and upsampling operations, a first feature map corresponding to the first feature map is obtained. Next, the first activation function is set to the Sigmoid function, and the Sigmoid function is used to perform a nonlinear transformation on the first feature map to obtain a parking line mask image. Simultaneously, the feature map is input into the parking recognition branch, and the parking recognition branch performs one convolution, activation, and upsampling operation on the feature map to obtain a second feature map corresponding to the first feature map. The second activation function is set to the Tanh function. A nonlinear transformation is performed on the second feature map using the Sigmoid function to obtain several parking line corner coordinates. A nonlinear transformation is performed on the second feature map using the Tanh function to obtain several parking line angle directions. The several parking line corner coordinates and several parking line angle directions form several sets of parking line corner recognition results.

[0037] This embodiment extracts features from the bird's-eye view by sharing a feature extraction layer, enabling the parking line segmentation branch and the parking space recognition branch to have the same data input, thus achieving resource reuse and feature consistency. Subsequently, through the operation of the parking line segmentation branch and the parking space recognition branch, the problem that a single task model cannot effectively capture clear directional clues is avoided, resulting in a more discriminative and comprehensive representation. In particular, it has a stronger ability to capture key local features of irregularly shaped parking spaces with weakened orientation, thereby improving the accuracy of parking space recognition.

[0038] In this embodiment, the step of segmenting the parking line mask image based on each group of parking line corner point recognition results to obtain the parking line binary mask corresponding to each group of parking line corner point recognition results includes: The results of each parking space line corner point recognition include: the coordinates of the parking space line corner point and the angle and direction of the parking space line; Starting from the coordinates of the parking space line corner point, extend a preset first pixel unit in the direction of the parking space line angle corresponding to the coordinates of the parking space line corner point to determine the center line of each group of parking space line corner point recognition results; Extend the center line to both sides by a preset second pixel unit to determine the rectangular area of ​​each group of parking space line corner point recognition results; The ROI region of each group of parking space line corner point recognition results is determined based on the rectangular region of each group of parking space line corner point recognition results; Based on the ROI region of each group of parking line corner point recognition results, the parking line mask image is segmented to obtain the parking line binary mask image corresponding to each group of parking line corner point recognition results. A morphological closing operation is performed on the binary mask image of the parking line corresponding to each group of parking line corner point recognition results to obtain the binary mask of the parking line corresponding to each group of parking line corner point recognition results.

[0039] In an optional embodiment, in each group of parking line corner point recognition results, a preset first pixel unit (set to a value of 30 to 50 pixels) is extended along the parking line angle direction corresponding to the parking line corner point coordinates as the starting point, and then a preset second pixel unit (set to a value of 10 pixels) is extended to both sides according to the center line to obtain a rectangular area of ​​each group of parking line corner point recognition results; then the pixel coordinates of the four corner points of the rectangular area of ​​each group of parking line corner point recognition results are extracted as the ROI (Region of Interest) of each group of parking line corner point recognition results; then the parking line mask image is segmented according to the ROI region to obtain a parking line binary mask image corresponding to each group of parking line corner point recognition results; then morphological closing operation is performed on the parking line binary mask image corresponding to each group of parking line corner point recognition results to obtain a parking line binary mask corresponding to each group of parking line corner point recognition results.

[0040] This embodiment extends the centerline by using the corner coordinates and angular direction of the parking line. Then, it expands to both sides of the centerline to form a rectangular area, which is further defined as the Region of Interest (ROI). The ROI is then segmented, and morphological closing operations are used to improve the completeness, connectivity, and accuracy of the binary mask of the parking line. This improves the accuracy of the subsequently extracted skeleton centerline, thereby improving the accuracy of parking space correction and parking space recognition.

[0041] In this embodiment, the step of extracting the skeleton centerline of each parking space line binary mask and determining the parking space correction angle value of each parking space line binary mask based on the skeleton centerline includes: The skeleton center line of each parking space line binary mask is extracted based on a preset binary image skeletonization algorithm, and the two-dimensional point set of each parking space line binary mask is determined based on the skeleton center line. Angle fitting is performed on the two-dimensional point set of each parking space line binary mask to determine the parking space correction angle value of each parking space line binary mask.

[0042] In an optional embodiment, the binary image skeletonization algorithm is set to the Zhang-Suen thinning algorithm. The Zhang-Suen thinning algorithm generates the skeleton center line of each parking space line binary mask, and the pixels on each skeleton center line constitute a two-dimensional point set.

[0043] It should be noted that the Zhang-Suen thinning algorithm is a classic, iterative binary image skeletonization algorithm. Its core objective is to simplify an object of a certain width into a single-pixel-wide skeleton centerline by peeling away edge pixels layer by layer, while maintaining the topological connectivity (not breaking the connection) and basic shape (not eroding key parts) of the target object.

[0044] This embodiment extracts the skeleton centerline of each parking space line binary mask using a binary image skeletonization algorithm, obtaining a two-dimensional point set of the parking space line binary mask. Angle fitting is then performed based on this two-dimensional point set to determine the parking space correction angle value. Fitting using the skeleton centerline avoids the problem of the original mask potentially containing numerous irregular protrusions and noise, thus obtaining a more accurate parking space correction angle value, improving the accuracy of parking space correction, and enhancing the accuracy of parking space recognition.

[0045] In this embodiment, the step of performing angle fitting on the two-dimensional point set of each parking space line binary mask to determine the parking space correction angle value of each parking space line binary mask includes: Based on the coordinates of each skeleton point in the two-dimensional point set, obtain the geometric center of each two-dimensional point set; Based on the geometric center of each of the two-dimensional point sets, each skeleton point in each of the two-dimensional point sets is centered to obtain the centered coordinates of each skeleton point in each of the two-dimensional point sets. Based on the centered coordinates of each skeleton point in each of the two-dimensional point sets, construct the covariance matrix of each of the two-dimensional point sets; The covariance matrix is ​​solved based on the principal component analysis algorithm to determine several eigenvalues ​​of each two-dimensional point set and the eigenvector of each eigenvalue; From a plurality of feature values ​​of each two-dimensional point set, select the first feature value with the largest feature value, and determine the feature angle value of each two-dimensional point set based on the feature vector corresponding to the first feature value; Based on the feature angle values ​​of each of the two-dimensional point sets, the parking space correction angle value of each parking space line binary mask is determined.

[0046] In one optional embodiment, the mean coordinates of each skeleton point in the two-dimensional point set are first calculated to obtain the geometric center of each two-dimensional point set. Then, the coordinates of the geometric center are subtracted from the coordinates of each skeleton point in the two-dimensional point set, thereby achieving centering of each skeleton point in the two-dimensional point set. This centering process eliminates the influence of the absolute position of the skeleton points in the image, allowing subsequent analysis to focus only on the distribution shape and direction of the points. After centering, the skeleton points will be distributed around the origin. Next, using the centered coordinates of each skeleton point in each two-dimensional point set, a covariance matrix for each two-dimensional point set is constructed. The covariance matrix is ​​set to a 2×2 specification, which describes the degree of variation of the skeleton points in the X and Y directions and the cooperative variation relationship between them. Then, Principal Component Analysis (PCA) is used to solve the covariance matrix to determine several eigenvalues ​​and eigenvectors of each two-dimensional point set. From the eigenvalues ​​of each two-dimensional point set, the largest eigenvalue is selected as the first eigenvalue, and the eigenvector corresponding to the first eigenvalue is used as the feature angle value of each two-dimensional point set. The eigenvector corresponding to the largest eigenvalue represents the distribution direction of the largest variance of the point set, which is the main direction of the skeleton extension, corresponding to the overall direction of the parking lines. Therefore, the eigenvector corresponding to the first eigenvalue is converted into an angle, i.e., the angle between the eigenvector and the X-axis is calculated, which is used as the feature angle value of the two-dimensional point set. The feature angle value of the two-dimensional point set is the parking correction angle value of the corresponding parking line binary mask.

[0047] It should be noted that principal component analysis is an unsupervised statistical method used for dimensionality reduction and feature extraction of datasets. Its core idea is to transform the original multivariate data onto a new set of orthogonal coordinate axes (called "principal components"), which are ordered in descending order of data variance.

[0048] This embodiment eliminates the influence of the absolute position of the two-dimensional point set in the image by centering the two-dimensional point set, thereby improving the accuracy of the feature values ​​and feature vectors obtained based on the centered coordinates. By selecting the first feature value with the largest feature value, the direction with the largest distribution variance in the two-dimensional point set can be displayed, thus obtaining the feature vector corresponding to the first feature value that displays the main axis direction of the parking space line, improving the accuracy of the feature angle value and the parking space correction angle value, and thus improving the accuracy of parking space correction and parking space recognition.

[0049] In this embodiment, the step of correcting the parking space line corner point recognition result based on the parking space correction angle value to complete the parking space orientation correction of the vehicle includes: Replace the corresponding parking space angle direction with the parking space correction angle value to correct the parking space line angle direction of the parking space line corner point recognition result; Based on the parking space angle direction after the corner point coordinates of the parking space line are corrected, the parking space direction correction result of the vehicle is determined, and the parking space direction correction of the vehicle is completed.

[0050] This embodiment corrects the parking space recognition by replacing the corresponding parking space angle direction with the parking space correction angle value, thus avoiding the problem of inaccurate parking space recognition results output by the model in traditional technical solutions and improving the accuracy of parking space recognition.

[0051] Example 2 Please refer to Figure 3 , Figure 3 A schematic diagram of a parking space orientation correction system based on parking space recognition provided in an embodiment of the present invention includes: a parking space recognition module 201, an image segmentation module 202, a parking space correction angle value acquisition module 203, and a parking space orientation correction module 204; The parking space recognition module 201 is used to collect images of the parking space environment of the vehicle, and to extract features from the parking space environment images based on a preset multi-task model to obtain a parking space line mask image and several sets of parking space line corner point recognition results of the parking space line mask image. The image segmentation module 202 is used to segment the parking line mask image based on each group of parking line corner point recognition results to obtain the parking line binary mask corresponding to each group of parking line corner point recognition results. The parking space correction angle value acquisition module 203 is used to extract the skeleton center line of each parking space line binary mask, and determine the parking space correction angle value of each parking space line binary mask based on the skeleton center line; The parking space orientation correction module 204 is used to correct the parking space line corner point identification result based on the parking space correction angle value, thereby completing the parking space orientation correction of the vehicle.

[0052] In this embodiment, the parking space recognition module 201 includes: a parking space recognition unit; In the parking space recognition unit, the preset multi-task model includes: a shared feature extraction layer, a parking line segmentation branch, and a parking space recognition branch; The parking space recognition unit is used to collect images of the parking space environment of the vehicle and perform perspective transformation on the parking space environment images to obtain a bird's-eye view of the parking space environment images. The bird's-eye view is input into the multi-task model so that the shared feature extraction layer, the parking line segmentation branch, and the parking space recognition branch can extract features from the bird's-eye view to obtain a parking line mask image and several sets of parking line corner point recognition results from the parking line mask image.

[0053] In this embodiment, the parking space recognition unit includes: a parking space recognition subunit; The parking space recognition subunit is used to input the bird's-eye view into the shared feature extraction layer of the multi-task model, so that the shared feature extraction layer can extract features from the bird's-eye view to obtain the feature map of the bird's-eye view. The feature map is input into the parking line segmentation branch, so that the parking line segmentation branch performs convolution, activation and upsampling operations on the feature map to obtain a first feature map corresponding to the feature map; and a nonlinear transformation is performed on the first feature map based on a preset first activation function to obtain a parking line mask image; The feature map is input into the parking space recognition branch, so that the parking space recognition branch performs convolution, activation and upsampling operations on the feature map to obtain a second feature map corresponding to the feature map; and performs nonlinear transformation on the second feature map based on a preset first activation function and a preset second activation function to obtain several sets of parking space line corner point recognition results of the parking space line mask image.

[0054] In this embodiment, the image segmentation module 202 includes: an image segmentation unit; In the image segmentation unit, the corner point recognition results for each group of parking space lines include: the coordinates of the parking space line corner point and the angular direction of the parking space line; The image segmentation unit is used to extend a preset first pixel unit from the parking space line corner point coordinates to the parking space line angle direction corresponding to the parking space line corner point coordinates to determine the center line of each group of parking space line corner point recognition results; Extend the center line to both sides by a preset second pixel unit to determine the rectangular area of ​​each group of parking space line corner point recognition results; The ROI region of each group of parking space line corner point recognition results is determined based on the rectangular region of each group of parking space line corner point recognition results; Based on the ROI region of each group of parking line corner point recognition results, the parking line mask image is segmented to obtain the parking line binary mask image corresponding to each group of parking line corner point recognition results. A morphological closing operation is performed on the binary mask image of the parking line corresponding to each group of parking line corner point recognition results to obtain the binary mask of the parking line corresponding to each group of parking line corner point recognition results.

[0055] In this embodiment, the parking space correction angle value acquisition module 203 includes: a parking space correction angle value acquisition unit; The parking space correction angle value acquisition unit is used to extract the skeleton center line of each parking space line binary mask based on a preset binary image skeletonization algorithm, and determine the two-dimensional point set of each parking space line binary mask based on the skeleton center line; Angle fitting is performed on the two-dimensional point set of each parking space line binary mask to determine the parking space correction angle value of each parking space line binary mask.

[0056] In this embodiment, the parking space correction angle value acquisition unit includes: a parking space correction angle value acquisition subunit; The parking space correction angle value acquisition subunit is used to obtain the geometric center of each two-dimensional point set based on the coordinates of each skeleton point in the two-dimensional point set; Based on the geometric center of each of the two-dimensional point sets, each skeleton point in each of the two-dimensional point sets is centered to obtain the centered coordinates of each skeleton point in each of the two-dimensional point sets. Based on the centered coordinates of each skeleton point in each of the two-dimensional point sets, construct the covariance matrix of each of the two-dimensional point sets; The covariance matrix is ​​solved based on the principal component analysis algorithm to determine several eigenvalues ​​of each two-dimensional point set and the eigenvector of each eigenvalue; From a plurality of feature values ​​of each two-dimensional point set, select the first feature value with the largest feature value, and determine the feature angle value of each two-dimensional point set based on the feature vector corresponding to the first feature value; Based on the feature angle values ​​of each of the two-dimensional point sets, the parking space correction angle value of each parking space line binary mask is determined.

[0057] In this embodiment, the parking space direction correction module 204 includes: a parking space direction correction unit; The parking space direction correction unit is used to replace the corresponding parking space angle direction with the parking space correction angle value in order to correct the parking space line angle direction of the parking space line corner point recognition result. Based on the parking space angle direction after the corner point coordinates of the parking space line are corrected, the parking space direction correction result of the vehicle is determined, and the parking space direction correction of the vehicle is completed.

[0058] Example 3 Based on the above embodiment of a parking space orientation correction method based on parking space recognition, embodiment three of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a parking space orientation correction method based on parking space recognition according to an embodiment of the present invention.

[0059] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0060] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0061] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0062] Based on the above-described method embodiments, this invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute a parking space orientation correction method based on parking space recognition as described in any of the above-described method embodiments.

[0063] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0064] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A parking space orientation correction method based on parking space recognition, characterized in that, include: The system collects images of the parking space environment of the vehicle and extracts features from the images based on a preset multi-task model to obtain a parking line mask image and several sets of parking line corner point recognition results from the parking line mask image. The parking line mask image is segmented based on the corner point recognition results of each group of parking lines to obtain the binary mask of the parking lines corresponding to each group of corner point recognition results. Extract the skeleton centerline of each parking space line binary mask, and determine the parking space correction angle value of each parking space line binary mask based on the skeleton centerline; The parking space line corner point identification result is corrected based on the parking space correction angle value to complete the parking space direction correction of the vehicle.

2. The parking space orientation correction method based on parking space recognition as described in claim 1, characterized in that, The process involves acquiring images of the parking space environment of the vehicle, extracting features from these images based on a preset multi-task model to obtain a parking line mask image, and several sets of parking line corner point recognition results from the parking line mask image, including: The pre-defined multi-task model includes: a shared feature extraction layer, a parking line segmentation branch, and a parking space recognition branch; Acquire images of the parking space environment of the vehicle, and perform perspective transformation on the images to obtain a bird's-eye view of the parking space environment. The bird's-eye view is input into the multi-task model so that the shared feature extraction layer, the parking line segmentation branch, and the parking space recognition branch can extract features from the bird's-eye view to obtain a parking line mask image and several sets of parking line corner point recognition results from the parking line mask image.

3. The parking space orientation correction method based on parking space recognition as described in claim 2, characterized in that, The process involves inputting the bird's-eye view into the multi-task model, enabling the shared feature extraction layer, parking line segmentation branch, and parking space recognition branch to extract features from the bird's-eye view, resulting in a parking line mask image and several sets of parking line corner point recognition results from the parking line mask image, including: The bird's-eye view is input into the shared feature extraction layer of the multi-task model, so that the shared feature extraction layer can extract features from the bird's-eye view to obtain the feature map of the bird's-eye view. The feature map is input into the parking line segmentation branch, so that the parking line segmentation branch performs convolution, activation and upsampling operations on the feature map to obtain a first feature map corresponding to the feature map; and a nonlinear transformation is performed on the first feature map based on a preset first activation function to obtain a parking line mask image; The feature map is input into the parking space recognition branch, so that the parking space recognition branch performs convolution, activation and upsampling operations on the feature map to obtain a second feature map corresponding to the feature map; and performs nonlinear transformation on the second feature map based on a preset first activation function and a preset second activation function to obtain several sets of parking space line corner point recognition results of the parking space line mask image.

4. The parking space orientation correction method based on parking space recognition as described in claim 1, characterized in that, The step of segmenting the parking line mask image based on the corner point recognition results of each group of parking line lines to obtain the binary mask of the parking line lines corresponding to each group of corner point recognition results includes: The results of each parking space line corner point recognition include: the coordinates of the parking space line corner point and the angle and direction of the parking space line; Starting from the coordinates of the parking space line corner point, extend a preset first pixel unit in the direction of the parking space line angle corresponding to the coordinates of the parking space line corner point to determine the center line of each group of parking space line corner point recognition results; Extend the center line to both sides by a preset second pixel unit to determine the rectangular area of ​​each group of parking space line corner point recognition results; The ROI region of each group of parking space line corner point recognition results is determined based on the rectangular region of each group of parking space line corner point recognition results; Based on the ROI region of each group of parking line corner point recognition results, the parking line mask image is segmented to obtain the parking line binary mask image corresponding to each group of parking line corner point recognition results. A morphological closing operation is performed on the binary mask image of the parking line corresponding to each group of parking line corner point recognition results to obtain the binary mask of the parking line corresponding to each group of parking line corner point recognition results.

5. The parking space orientation correction method based on parking space recognition as described in claim 1, characterized in that, The step of extracting the skeleton centerline of each parking space line binary mask and determining the parking space correction angle value of each parking space line binary mask based on the skeleton centerline includes: The skeleton center line of each parking space line binary mask is extracted based on a preset binary image skeletonization algorithm, and the two-dimensional point set of each parking space line binary mask is determined based on the skeleton center line. Angle fitting is performed on the two-dimensional point set of each parking space line binary mask to determine the parking space correction angle value of each parking space line binary mask.

6. The parking space orientation correction method based on parking space recognition as described in claim 5, characterized in that, The step of performing angle fitting on the two-dimensional point set of each parking space line binary mask to determine the parking space correction angle value of each parking space line binary mask includes: Based on the coordinates of each skeleton point in the two-dimensional point set, obtain the geometric center of each two-dimensional point set; Based on the geometric center of each of the two-dimensional point sets, each skeleton point in each of the two-dimensional point sets is centered to obtain the centered coordinates of each skeleton point in each of the two-dimensional point sets. Based on the centered coordinates of each skeleton point in each of the two-dimensional point sets, construct the covariance matrix of each of the two-dimensional point sets; The covariance matrix is ​​solved based on the principal component analysis algorithm to determine several eigenvalues ​​of each two-dimensional point set and the eigenvector of each eigenvalue; From a plurality of feature values ​​of each two-dimensional point set, select the first feature value with the largest feature value, and determine the feature angle value of each two-dimensional point set based on the feature vector corresponding to the first feature value; Based on the feature angle values ​​of each of the two-dimensional point sets, the parking space correction angle value of each parking space line binary mask is determined.

7. The parking space orientation correction method based on parking space recognition as described in claim 4, characterized in that, The step of correcting the parking space line corner point recognition result based on the parking space correction angle value to complete the parking space orientation correction of the vehicle includes: Replace the corresponding parking space angle direction with the parking space correction angle value to correct the parking space line angle direction of the parking space line corner point recognition result; Based on the parking space angle direction after the corner point coordinates of the parking space line are corrected, the parking space direction correction result of the vehicle is determined, and the parking space direction correction of the vehicle is completed.

8. A parking space orientation correction system based on parking space recognition, characterized in that, include: Parking space recognition module, image segmentation module, parking space correction angle value acquisition module, and parking space direction correction module; The parking space recognition module is used to collect images of the parking space environment of the vehicle, and to extract features from the parking space environment image based on a preset multi-task model to obtain a parking space line mask image and several sets of parking space line corner point recognition results of the parking space line mask image. The image segmentation module is used to segment the parking line mask image based on each group of parking line corner point recognition results to obtain the parking line binary mask corresponding to each group of parking line corner point recognition results; The parking space correction angle value acquisition module is used to extract the skeleton center line of each parking space line binary mask, and determine the parking space correction angle value of each parking space line binary mask based on the skeleton center line. The parking space orientation correction module is used to correct the parking space line corner point recognition result based on the parking space correction angle value, thereby completing the parking space orientation correction of the vehicle.

9. A parking space orientation correction system based on parking space recognition as described in claim 8, characterized in that, The parking space recognition module includes: a parking space recognition unit; In the parking space recognition unit, the preset multi-task model includes: a shared feature extraction layer, a parking line segmentation branch, and a parking space recognition branch; The parking space recognition unit is used to collect images of the parking space environment of the vehicle and perform perspective transformation on the parking space environment images to obtain a bird's-eye view of the parking space environment images. The bird's-eye view is input into the multi-task model so that the shared feature extraction layer, the parking line segmentation branch, and the parking space recognition branch can extract features from the bird's-eye view to obtain a parking line mask image and several sets of parking line corner point recognition results from the parking line mask image.

10. A parking space orientation correction system based on parking space recognition as described in claim 9, characterized in that, The parking space recognition unit includes: a parking space recognition subunit; The parking space recognition subunit is used to input the bird's-eye view into the shared feature extraction layer of the multi-task model, so that the shared feature extraction layer can extract features from the bird's-eye view to obtain the feature map of the bird's-eye view. The feature map is input into the parking line segmentation branch, so that the parking line segmentation branch performs convolution, activation and upsampling operations on the feature map to obtain a first feature map corresponding to the feature map; and a nonlinear transformation is performed on the first feature map based on a preset first activation function to obtain a parking line mask image; The feature map is input into the parking space recognition branch, so that the parking space recognition branch performs convolution, activation and upsampling operations on the feature map to obtain a second feature map corresponding to the feature map; and performs nonlinear transformation on the second feature map based on a preset first activation function and a preset second activation function to obtain several sets of parking space line corner point recognition results of the parking space line mask image.

Images