A parking space detection method, electronic equipment and vehicle
By combining deep learning networks and traditional methods, parking space representation information is obtained, matched, and fused, solving the environmental adaptability and accuracy problems of parking space detection methods and achieving high-precision parking space detection.
Patent Information
- Application Number
- CN202311227885.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-21
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2043-09-21
AI Technical Summary
Existing parking space detection methods have poor environmental adaptability and poor algorithm stability, resulting in low accuracy in the position and angle of parking space lines.
By combining deep learning networks and traditional parking space detection methods, the first and second images of the parking space are obtained. The deep learning network is used to obtain the parking space representation information, and the parking space lines are obtained by combining traditional methods. The results are then matched and fused to improve accuracy.
It improves the positional and angular accuracy of parking space detection results, simplifies the processing flow, reduces the amount of computation, and enhances the stability and environmental adaptability of the algorithm.
Smart Images

Figure CN118279849B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of vehicle technology, and more specifically to a parking space detection method, electronic equipment, and vehicle. Background Technology
[0002] With the development and advancement of technology, automatic parking systems are being applied to more and more vehicles. These systems can automatically detect parking spaces. However, among related technologies, parking space detection methods have poor environmental adaptability, are prone to failure, and have poor stability. Furthermore, due to limitations in annotation accuracy, the position and angle of the parking space markings inevitably deviate to some extent, resulting in low accuracy in the detected parking space line position and angle. Summary of the Invention
[0003] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This summary section is not intended to limit the key and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.
[0004] To address the existing problems, this application provides a parking space detection method, comprising: acquiring a first image and a second image of a parking space; inputting the first image into a trained parking space detection model based on a deep learning network to obtain parking space representation information; obtaining a first parking space centerline vector based on the parking space representation information; obtaining a first parking space line and a second parking space line based on the second image, wherein the first parking space line and the second parking space line are parallel; obtaining a second parking space centerline vector based on the first parking space line and the second parking space line; matching the first parking space centerline vector and the second parking space centerline vector, and determining whether the first parking space centerline vector and the second parking space centerline vector are successfully matched; when the first parking space centerline vector and the second parking space centerline vector are successfully matched, obtaining fused parking space representation information based on the parking space representation information, the first parking space line, and the second parking space line; and obtaining a parking space detection result based on the fused parking space representation information.
[0005] For example, the parking space representation information includes the coordinates of the first endpoint of the entry line, the coordinates of the second endpoint of the entry line, and the angle of the dividing line.
[0006] For example, the centerline vector of the first parking space is represented by the coordinates of the centerline vector of the first parking space and the angle of the centerline vector of the first parking space. The step of obtaining the centerline vector of the first parking space based on the parking space representation information includes: averaging the coordinates of the first endpoint of the entry line and the coordinates of the second endpoint of the entry line to obtain the coordinates of the centerline vector of the first parking space; the angle of the centerline vector of the first parking space is equal to the angle of the dividing line.
[0007] For example, obtaining the first parking space line and the second parking space line based on the second image includes: obtaining a grayscale image based on the second image; performing histogram equalization on the grayscale image to obtain an equalized grayscale image; performing noise reduction on the equalized grayscale image to obtain a denoised grayscale image; converting the denoised grayscale image into a binary image using an adaptive thresholding method; performing an opening operation on the binary image to obtain a denoised binary image; performing skeletonization on the denoised binary image to obtain a binary skeleton image; detecting potential parking space lines using the Hough line detection method; and filtering the potential parking space lines using prior knowledge to obtain the first parking space line and the second parking space line.
[0008] For example, obtaining a grayscale image based on the second image includes: converting the second image into an HSV image; and extracting the V channel image from the HSV image as the grayscale image.
[0009] For example, the first parking space line is represented by the coordinates of the first endpoint of the first parking space line and the coordinates of the second endpoint of the first parking space line; the second parking space line is represented by the coordinates of the first endpoint of the second parking space line and the coordinates of the second endpoint of the second parking space line.
[0010] For example, the second parking space centerline vector is represented by the coordinates of the second parking space centerline vector and the angle of the second parking space centerline vector. The step of obtaining the second parking space centerline vector based on the first parking space line and the second parking space line includes: averaging the coordinates of the first endpoint of the first parking space line and the first endpoint of the second parking space line to obtain the coordinates of the first endpoint of the second centerline; averaging the coordinates of the second endpoint of the first parking space line and the second endpoint of the second parking space line to obtain the coordinates of the second endpoint of the second centerline; averaging the coordinates of the first endpoint of the second centerline and the second endpoint of the second centerline to obtain the coordinates of the second parking space centerline vector; and obtaining the angle of the second parking space centerline vector based on the coordinates of the first endpoint of the second centerline and the second endpoint of the second centerline.
[0011] For example, the step of matching the centerline vector of the first parking space with the centerline vector of the second parking space and determining whether the centerline vector of the first parking space and the centerline vector of the second parking space are successfully matched includes: calculating the absolute value of the difference between the angle of the centerline vector of the first parking space and the angle of the centerline vector of the second parking space to obtain the centerline angle; calculating the distance from the coordinate of the centerline vector of the first parking space to the centerline vector of the second parking space to obtain the centerline distance; when the centerline angle is not greater than a preset angle and the centerline distance is not greater than a preset distance, it is determined that the centerline vector of the first parking space and the centerline vector of the second parking space are successfully matched.
[0012] For example, the preset included angle is 30 degrees.
[0013] For example, the fused parking space representation information includes the coordinates of a first intersection point, the coordinates of a second intersection point, and the parking space angle. The step of obtaining the fused parking space representation information based on the parking space representation information, the first parking space line, and the second parking space line includes: obtaining an entry line based on the coordinates of the first endpoint and the second endpoint of the entry line; obtaining the coordinates of the intersection point of the first parking space line and the entry line based on the first parking space line and the entry line, denoted as the first intersection point coordinates; obtaining the coordinates of the intersection point of the second parking space line and the entry line based on the second parking space line and the entry line, denoted as the second intersection point coordinates; and the parking space angle is equal to the angle of the centerline vector of the second parking space.
[0014] For example, when the centerline vector of the first parking space fails to match the centerline vector of the second parking space, the parking space detection result is obtained based on the parking space representation information.
[0015] For example, the training of the parking space detection model includes the following steps: constructing a dataset, the dataset including a training set; inputting sample images from the training set into the parking space detection model to be trained, and having the parking space detection model to be trained perform the following operations: extracting features from the sample images to obtain parking space feature images; obtaining parking space feature information based on the parking space feature images, and obtaining parking space representation information of the parking spaces in the sample images based on the parking space feature information; obtaining a loss function based on the parking space feature information, and updating the model parameters of the parking space detection model to be trained based on the loss function; iterating the operations until the parking space detection model to be trained converges, thereby obtaining the trained parking space detection model based on a deep learning network.
[0016] For example, the parking space feature image includes a horizontal axis coordinate feature image of the parking space corner point, a vertical axis coordinate feature image of the parking space corner point, a horizontal axis length feature image of the entry line, a vertical axis length feature image of the entry line, a sine value feature image of the dividing line, and a cosine value feature image of the dividing line.
[0017] For example, the parking space feature information includes the coordinates of the parking space corner point, the length of the entry line, and the angle of the dividing line. Obtaining the parking space feature information based on the parking space feature image includes: obtaining the parking space corner point coordinates based on the horizontal axis coordinate feature image and the vertical axis coordinate feature image of the parking space corner point; obtaining the entry line length based on the horizontal axis length feature image and the vertical axis length feature image of the entry line; and obtaining the dividing line angle based on the sine value feature image and the cosine value feature image of the dividing line.
[0018] For example, the parking space feature image further includes at least one of the following: a confidence feature image and an occupancy sign feature image.
[0019] For example, when the parking space feature information includes the confidence feature image, the parking space feature information also includes confidence information, and obtaining the parking space feature information based on the parking space feature image includes: obtaining the confidence information based on the confidence feature image; when the parking space feature information includes the occupancy sign feature image, the parking space feature information also includes occupancy sign information, and obtaining the parking space feature information based on the parking space feature image includes: obtaining the occupancy sign information based on the occupancy sign feature image.
[0020] In another aspect, this application provides an electronic device including a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the aforementioned parking space detection method.
[0021] In another aspect, this application provides a vehicle that includes the aforementioned electronic equipment.
[0022] The parking space detection method, electronic device, and vehicle of this application can process a first image based on a deep learning network parking space detection model to obtain parking space representation information, and then obtain the centerline vector of the first parking space. The parking space detection model based on the deep learning network has good generalization and environmental adaptability. Although it can obtain parking space detection results based on the parking space representation information, the positional accuracy and angular accuracy of the obtained parking space detection results are low. Therefore, it also uses a traditional parking space detection method to obtain the first parking space line and the second parking space line. The positional accuracy and angular accuracy of the first parking space line and the second parking space line are relatively high, and then the centerline vector of the second parking space is obtained. The centerline vector of the first parking space is matched with the centerline vector of the second parking space. When the matching is successful, the parking space representation information can be fused with the first parking space line and the second parking space line to obtain fused parking space representation information. The fused parking space representation information significantly improves the positional accuracy and angular accuracy based on the parking space representation information, and thus the accuracy of the parking space detection results obtained based on the fused parking space representation information is higher. Furthermore, obtaining the first and second parking space lines solely based on traditional parking space detection methods eliminates the need for complex post-processing matching and parsing, significantly reducing the computational load and simplifying the processing flow of traditional parking space detection methods. Attached Figure Description
[0023] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.
[0024] In the attached image:
[0025] Figure 1 A schematic flowchart of a parking space detection method according to a specific embodiment of this application is shown.
[0026] Figure 2 A schematic diagram of parking space representation information according to a specific embodiment of this application is shown.
[0027] Figure 3 A schematic diagram of the fusion of parking space representation information is shown in a specific embodiment of this application.
[0028] Figure 4 A schematic structural block diagram of an electronic device according to a specific embodiment of this application is shown. Detailed Implementation
[0029] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention described herein without inventive effort should fall within the protection scope of the present invention.
[0030] Below, in conjunction with Figures 1 to 3 This application describes a parking space detection method according to a specific embodiment. Figure 1 A schematic flowchart illustrating a parking space detection method according to a specific embodiment of this application is shown. Figure 2 This illustration shows a schematic diagram of parking space representation information according to a specific embodiment of this application. Figure 3 This diagram illustrates a specific embodiment of the present application of fused parking space representation information. Figure 1 As shown, the parking space detection method 100 may include the following steps:
[0031] In step S110, the first image and the second image of the parking space are acquired.
[0032] In step S120, the first image is input into the trained parking space detection model based on a deep learning network to obtain parking space representation information.
[0033] In step S130, the centerline vector of the first parking space is obtained based on the parking space representation information.
[0034] In step S140, a first parking space line and a second parking space line are obtained based on the second image, wherein the first parking space line and the second parking space line are parallel.
[0035] In step S150, the centerline vector of the second parking space is obtained based on the first parking space line and the second parking space line.
[0036] In step S160, the centerline vector of the first parking space is matched with the centerline vector of the second parking space, and it is determined whether the centerline vector of the first parking space and the centerline vector of the second parking space are successfully matched.
[0037] In step S170, when the centerline vector of the first parking space and the centerline vector of the second parking space are successfully matched, the fused parking space representation information is obtained based on the parking space representation information, the first parking space line and the second parking space line.
[0038] In step S180, the parking space detection result is obtained based on the fused parking space representation information.
[0039] In the embodiments of this application, a first image and a second image of the parking space can be acquired using a panoramic monitoring imaging system (AVM) mounted on the vehicle. The AVM can capture images using multiple (typically four) ultra-wide-angle fisheye lenses, and the captured images are then distorted and stitched together using a special algorithm to form a panoramic view of the vehicle's surroundings. For example, the first and second images can be images directly output by the AVM, meaning they can be identical. Alternatively, the first or second image can be an image obtained by further processing after being output by the AVM. For instance, the field of view of the image output by the AVM is typically 10m*10m or larger, but its edges are often blurry and distorted. Therefore, an image 6m wide and 9m high can be cropped from it as the first or second image. Next, the first image is input into a trained parking space detection model based on a deep learning network. This model can extract the necessary information from the first image and output parking space representation information. This parking space representation information can be used to represent a parking space, and the centerline vector of the first parking space can be obtained based on this representation information. Simultaneously, a first parking space line and a second parking space line can be obtained based on the second image. The first and second parking space lines are parallel, and the centerline vector of the second parking space can be obtained through these lines. For example, the second image can be processed using a traditional parking space detection method to obtain the first and second parking space lines. For example, the first and second parking space lines are dividing lines that separate different parking spaces. Finally, the centerline vectors of the first and second parking spaces are matched. When a successful match is determined, fused parking space representation information can be obtained based on the obtained parking space representation information, the first parking space line, and the second parking space line. The parking space detection result is then obtained based on this fused parking space representation information.
[0040] Therefore, the parking space method of this application embodiment can process the first image based on the parking space detection model of the deep learning network to obtain parking space representation information, and then obtain the first parking space centerline vector. The parking space detection model based on the deep learning network has good generalization and environmental adaptability. Although it can obtain parking space detection results based on the parking space representation information, the positional accuracy and angular accuracy of the obtained parking space detection results are low. Therefore, the first parking space line and the second parking space line are obtained based on the traditional parking space detection method. The positional accuracy and angular accuracy of the first parking space line and the second parking space line are relatively high. Then, the second parking space centerline vector is obtained, and the first parking space centerline vector and the second parking space centerline vector are matched. When the matching is successful, the parking space representation information can be fused with the first parking space line and the second parking space line to obtain fused parking space representation information. The fused parking space representation information significantly improves the positional accuracy and angular accuracy based on the parking space representation information, and thus the accuracy of the parking space detection results obtained based on the fused parking space representation information is higher. Furthermore, obtaining the first and second parking space lines solely based on traditional parking space detection methods eliminates the need for complex post-processing matching and parsing, significantly reducing the computational load and simplifying the processing flow of traditional parking space detection methods.
[0041] In the embodiments of this application, training a parking space detection model based on a deep learning network includes the following steps: constructing a dataset, the dataset including a training set; inputting sample images from the training set into the parking space detection model to be trained, and having the parking space detection model to be trained perform the following operations: extracting features from the sample images to obtain parking space feature images; obtaining parking space feature information based on the parking space feature images, and obtaining parking space representation information of the parking spaces in the sample images based on the parking space feature information; obtaining a loss function based on the parking space feature information, and updating the model parameters of the parking space detection model to be trained based on the loss function; iterating the operations until the parking space detection model to be trained converges, thus obtaining the trained parking space detection model based on the deep learning network. In the above process, through continuous iteration, the model parameters of the parking space detection model are optimized multiple times until the optimized model parameters meet the corresponding requirements, for example, the loss function value of the parking space detection model has stabilized at its minimum value. Furthermore, generally speaking, as the training set increases, the accuracy of the trained parking space detection model also increases. For example, the dataset may also include a validation set and a test set. The training set is the data samples used for model fitting, i.e., the sample set used for training, mainly used to train the model parameters of the parking space detection model. The validation set is a separate sample set reserved during model training, which can be used to adjust the model's hyperparameters and to perform a preliminary evaluation of the model's capabilities. The test set can be used to evaluate the model's performance. For example, the input image has a resolution of 512*512, and the input image can be an RGB image or a grayscale image, preferably an RGB image. For example, a parking space feature image is obtained by performing a convolution operation on the input image, and the resulting parking space feature image has a resolution of 16*16.
[0042] In the embodiments of this application, the parking space detection model based on deep learning networks adopts target recognition models such as YOLO v3, and there is no limitation on this.
[0043] In embodiments of this application, the parking space feature image includes a horizontal axis coordinate feature image of the parking space corner point, a vertical axis coordinate feature image of the parking space corner point, a horizontal axis length feature image of the entry line, a vertical axis length feature image of the entry line, a sine value feature image of the dividing line, and a cosine value feature image of the dividing line. Exemplarily, a feature image is an abstract representation of the input image, and different feature images can be used to represent different specific features in the input image.
[0044] In embodiments of this application, the parking space feature information includes the coordinates of the parking space corner point, the length of the entry line, and the angle of the dividing line. Obtaining the parking space feature information based on the parking space feature image includes: obtaining the parking space corner point coordinates based on the horizontal axis coordinate feature image and the vertical axis coordinate feature image; obtaining the entry line length based on the horizontal axis length feature image and the vertical axis length feature image; and obtaining the dividing line angle based on the sine value feature image and the cosine value feature image. For example, the horizontal axis coordinates of the parking space corner point can be obtained from the horizontal axis coordinate feature image, and the vertical axis coordinates of the parking space corner point can be obtained from the vertical axis coordinate feature image, thus obtaining the coordinates of the parking space corner point. Figure 2 As shown, the entry line AB includes a first endpoint A and a second endpoint B. The parking space corner point can be either the first endpoint A or the second endpoint B; the following description uses the first endpoint A as the parking space corner point. For example, the horizontal axis length of the entry line can be obtained from the horizontal axis length feature image, and the vertical axis length of the entry line can be obtained from the vertical axis length feature image. The entry line length is represented by the horizontal axis length and the vertical axis length, which are calculated based on the parking space corner point A. The horizontal axis length and the vertical axis length can be negative values, meaning they inherently contain directional information. For example, when the horizontal axis length is negative, it means that the entry line extends from the parking space corner point A in the negative direction of the horizontal axis. For example, the sine value of the segmentation line can be obtained from the sine value feature image, and the cosine value can be obtained from the cosine value feature image. Knowing the sine and cosine values of the segmentation line, the tangent value of the segmentation line can be obtained, and then the angle of the segmentation line can be obtained through the arctangent function, such as... Figure 2 As shown, AD' and BC' are both dividing lines. For dividing lines AD' and BC', only their angles are needed, not their lengths. For example, the coordinates obtained above can be coordinates in the image coordinate system or coordinates in the world coordinate system, but all coordinates should be in a unified coordinate system. For example, parking space corners are not necessarily actual parking space corners in the image, and entering lines and dividing lines are not actual solid lines, dashed lines, or other types of lines in the image, but rather abstract mathematical and geometric representations.
[0045] In embodiments of this application, the parking space feature image further includes at least one of the following: a confidence feature image and an occupancy sign image. For example, when the parking space feature image includes a confidence feature image, the parking space feature information also includes confidence information. Obtaining parking space feature information based on the parking space feature image includes: obtaining confidence information based on the confidence feature image. When the parking space feature information includes an occupancy sign feature image, the parking space feature information also includes occupancy sign information. Obtaining parking space feature information based on the parking space feature image includes: obtaining occupancy sign information based on the occupancy sign feature image. For example, the occupancy sign information can reflect whether the parking space is occupied, for example, whether there are obstacles in the parking space. For example, an availability sign feature image can be output to replace the occupancy sign feature image, and availability sign information can be obtained to replace the occupancy sign information. The availability sign information can reflect whether the parking space is available. Both the availability sign information and the occupancy sign information can reflect the usage status of the parking space.
[0046] In embodiments of this application, the parking space representation information includes the coordinates of the first endpoint of the entry line, the coordinates of the second endpoint of the entry line, and the angle of the dividing line. For example, as shown... Figure 2 As shown, AB is the entry line, and AD' and BC' are both dividing lines. A is the first endpoint of the entry line, B is the second endpoint, and A is the corner point of the parking space. As mentioned above, we can obtain the coordinates of the corner point A, the length of the entry line AB, and the angle between the dividing lines AD' and BC'. The length of the entry line AB is represented by its horizontal and vertical coordinates. For example, if the coordinates of the corner point A are (6, 6), and the length of the entry line AB is (-2, -3), then the coordinates of the second endpoint B of the entry line AB are (4, 3). Thus, we obtain the parking space information: the coordinates of the first endpoint A of the entry line, the coordinates of the second endpoint B of the entry line, and the angle between the dividing lines AD' and BC'. Furthermore, as... Figure 2 As shown, this parking space representation information can represent a parking space, meaning that parking space detection results can be obtained through this parking space representation information.
[0047] In embodiments of this application, the centerline vector of the first parking space is represented by the coordinates and angle of the centerline vector of the first parking space. Obtaining the centerline vector of the first parking space based on parking space representation information includes: averaging the coordinates of the first endpoint and the second endpoint of the entry line to obtain the coordinates of the centerline vector of the first parking space; the angle of the centerline vector of the first parking space is equal to the angle of the dividing line. For example, as... Figure 3As shown, AB is the entry line, A is the first endpoint of the entry line, B is the second endpoint of the entry line, and D is the midpoint of the entry line AB. AD' and BC' are both dividing lines. The coordinates of the centerline vector of the first parking space are the coordinates of a point through which the centerline vector of the first parking space passes. The centerline vector of the first parking space can be represented by these coordinates combined with the angle of the centerline vector of the first parking space. In this embodiment, the coordinates of D are the coordinates of the centerline vector of the first parking space, and the vector derived from D with respect to the angle of the centerline vector of the first parking space is the centerline vector of the first parking space. Figure 3 L0 in the example. Let the coordinates of A be (x_A, y_A), the coordinates of B be (x_B, y_B), the dividing line angle be theta, and the parking space representation information be represented as {x_A, y_A, x_B, y_B, theta}. Let the coordinates of D be (x_M, y_M), and the angle of the centerline vector of the first parking space be theta_M. Then the centerline vector L0 of the first parking space can be represented as {x_M, y_M, theta_M}, and we have:
[0048]
[0049] In embodiments of this application, obtaining a first parking space line and a second parking space line based on a first image includes: obtaining a grayscale image based on the second image; performing histogram equalization on the grayscale image to obtain an equalized grayscale image; performing noise reduction processing on the equalized grayscale image to obtain a denoised grayscale image; converting the denoised grayscale image into a binary image using an adaptive thresholding method; performing an opening operation on the binary image to obtain a denoised binary image; performing skeletonization processing on the denoised binary image to obtain a binary skeleton image; detecting potential parking space lines using the Hough line detection method; and filtering the potential parking space lines using prior knowledge to obtain the first parking space line and the second parking space line. For example, when the acquired second image is an RGB image, it is necessary to convert the RGB image into a grayscale image to reduce the processing difficulty of the algorithm. For example, obtaining a grayscale image based on the second image includes: converting the second image into an HSV image; and extracting the V channel image from the HSV image as the grayscale image. For example, to maximize the contrast of parking lines, the second image is first converted to an HSV image, and then the V channel is extracted as a grayscale image for subsequent parking line detection. For example, histogram equalization is performed on the grayscale image to obtain an equalized grayscale image, which enhances image contrast and amplifies the brightness difference between the parking lines and the background, facilitating subsequent parking line detection. For example, histogram equalization inevitably amplifies image noise; to reduce the impact of noise on subsequent detection, a 3x3 median filter can be used to denoise the equalized grayscale image, resulting in a denoised grayscale image. For example, converting the denoised grayscale image into a binary image using an adaptive thresholding method further reduces the difficulty of subsequent parking line detection. For example, an opening operation is performed on the binary image to obtain a denoised binary image; the opening operation refers to processing the binary image using an erosion-dilation method to remove isolated outlier noise from the environment as much as possible. For example, since parking lines have a certain width, to facilitate parking line detection, the denoised binary image is skeletonized to obtain a binary skeleton image. This ensures that when the Hough line detection method is used subsequently, only one straight line will be detected for each parking line. For example, prior knowledge is used to filter the potential parking lines. This prior knowledge may include prior knowledge such as whether the parking lines are paired parallel lines, or whether the spacing between the parallel lines meets the parking space size requirements.
[0050] In the embodiments of this application, the first parking line is represented by the coordinates of its first endpoint and second endpoint; the second parking line is represented by the coordinates of its first endpoint and second endpoint. Exemplarily, the first parking line should not be understood as a line segment, and the first and second endpoints are not actually endpoints of the first parking line, but rather two points through which the first parking line passes. The first parking line is simply represented by the coordinates of its first and second endpoints. Similarly, the second parking line should not be understood as a line segment, and the first and second endpoints are not actually endpoints of the second parking line, but rather two points through which the second parking line passes. The first parking line is simply represented by the coordinates of its first and second endpoints.
[0051] In embodiments of this application, the second parking space centerline vector is represented by the coordinates and angle of the second parking space centerline vector. Obtaining the second parking space centerline vector based on the first parking space line and the second parking space line includes: averaging the coordinates of the first endpoint of the first parking space line and the first endpoint of the second parking space line to obtain the coordinates of the first endpoint of the second centerline; averaging the coordinates of the second endpoints of the first parking space line and the second endpoint of the second parking space line to obtain the coordinates of the second endpoint of the second centerline; averaging the coordinates of the first endpoint of the second centerline and the second endpoint of the second centerline to obtain the coordinates of the second parking space centerline vector; and obtaining the angle of the second parking space centerline vector based on the coordinates of the first endpoint of the second centerline and the second endpoint of the second centerline. For example, as... Figure 3 As shown, L1 is the first parking line, L2 is the second parking line, and L3 is the center line vector of the second parking space. Let the coordinates of the first endpoint of the first parking line be (x1, y1), the coordinates of the second endpoint of the first parking line be (x1', y1'), the coordinates of the first endpoint of the second parking line be (x2, y2), and the coordinates of the second endpoint of the second parking line be (x2', y2'). Then, the first parking line L1 can be represented as {x1, y1, x1', y1'}, and the second parking line L2 can be represented as {x2, y2, x2', y2'}. The coordinates of the first endpoint of the second center line are set as (x1, y1, x2', y2'). m y m The coordinates of the second endpoint of the second median are (x m ', y m If '), then we have:
[0052]
[0053] At the same time, the centerline vector coordinates of the second parking space are set as (x mc y mc The angle of the centerline vector of the second parking space is theta. mc The centerline vector L3 of the second parking space can be represented as {x}mc y mc theta mc}, and there is:
[0054]
[0055] In embodiments of this application, matching the centerline vector of the first parking space with the centerline vector of the second parking space and determining whether the centerline vectors of the first and second parking spaces are successfully matched includes: calculating the absolute value of the difference between the angles of the first and second parking space centerline vectors to obtain the centerline angle; calculating the distance from the coordinates of the first and second parking space centerline vectors to obtain the centerline distance; and determining that the centerline vectors of the first and second parking spaces are successfully matched when the centerline angle is not greater than a preset angle and the centerline distance is not greater than a preset distance. Figure 3 As shown, let the angle between the medians be Δθ and the distance between the medians be Δd. Δd can be calculated using the formula for the distance from a point to a line:
[0056] Δθ=theta mc -theta_M
[0057] For example, the preset included angle is 30 degrees. For example, in the world coordinate system, the preset distance is 30cm.
[0058] In embodiments of this application, the fused parking space representation information includes the coordinates of a first intersection point, the coordinates of a second intersection point, and the parking space angle. Based on the parking space representation information, the first parking space line, and the second parking space line, the fused parking space representation information is obtained, including: obtaining an entry line based on the coordinates of the first endpoint and the second endpoint of the entry line; obtaining the coordinates of the intersection point of the first parking space line and the entry line based on the first parking space line and the entry line, denoted as the first intersection point coordinates; obtaining the coordinates of the intersection point of the second parking space line and the entry line based on the second parking space line and the entry line, denoted as the second intersection point coordinates; the parking space angle is equal to the angle of the centerline vector of the second parking space. For example, as... Figure 3 As shown, the intersection point of the first parking line L1 and the entry line AB is E. The coordinates of E can be calculated from the intersection of the entry line AB and the first parking line L1. The intersection point of the second parking line L2 and the entry line AB is F. The coordinates of F can be calculated from the intersection of the entry line AB and the second parking line L2. Let the coordinates of E be (x... E y E The coordinates of F are (x) F y F If the fused parking space representation information can be expressed as {x}, then... E y E x F y F theta mcIn other words, compared to the parking space representation information {x_A, y_A, x_B, y_B, theta}, the fused parking space representation information replaces the coordinates of A and B with the coordinates of E and F, and replaces the centerline vector angle of the first parking space with the centerline vector angle of the second parking space, which improves the position and angle accuracy of parking space detection and makes the parking space detection results more accurate.
[0059] In one example, when the centerline vector of the first parking space fails to match the centerline vector of the second parking space, the parking space detection result is obtained based on the parking space representation information. If the centerline angle is greater than a preset angle or the centerline distance is greater than a preset distance, it is determined that the centerline vectors of the first and second parking spaces have failed to match. In this case, the parking space detection result is obtained based on the parking space representation information {x_A, y_A, x_B, y_B, theta}. That is, only when a match fails is the parking space representation information output by the deep learning network-based parking space detection model trusted.
[0060] The above description exemplarily illustrates a parking space detection method according to a specific embodiment of this application. Based on the above description, the parking space detection method of this application can process a first image based on a parking space detection model using a deep learning network to obtain parking space representation information, and then obtain a first parking space centerline vector. The parking space detection model based on the deep learning network has good generalization and environmental adaptability. Although it can obtain parking space detection results based on the parking space representation information, the positional and angular accuracy of the obtained parking space detection results is low. Therefore, it also uses a traditional parking space detection method to obtain a first parking space line and a second parking space line. The positional and angular accuracy of the first and second parking space lines are relatively high, and then a second parking space centerline vector is obtained. The first parking space centerline vector and the second parking space centerline vector are matched. When the matching is successful, the parking space representation information can be fused with the first parking space line and the second parking space line to obtain fused parking space representation information. The fused parking space representation information significantly improves the positional and angular accuracy based on the parking space representation information, and thus the accuracy of the parking space detection results obtained based on the fused parking space representation information is higher. Furthermore, obtaining the first and second parking space lines solely based on traditional parking space detection methods eliminates the need for complex post-processing matching and parsing, significantly reducing the computational load and simplifying the processing flow. For example, the parking space detection method of this application integrates the advantages of traditional parking space detection methods and deep learning network-based methods, greatly improving the accuracy of detected positions and angles while ensuring algorithm stability and environmental adaptability. For example, the parking space detection method of this application does not fully rely on the parking space representation information output by the deep learning network-based parking space detection model; it only obtains the parking space detection result based on the parking space representation information when the centerline vectors of the first and second parking spaces fail to match. For example, matching the centerline vectors of the first and second parking spaces involves low computational cost and high algorithm robustness.
[0061] Below, refer to Figure 4 To describe the electronic device of this application. Figure 4 A schematic structural block diagram of an electronic device according to a specific embodiment of this application is shown. Figure 4 As shown, the electronic device 400 includes a processor 410 and a memory 420. The memory 420 stores a computer program. When the computer program is run by the processor 410, the processor 410 executes the parking space detection method described above.
[0062] For example, processor 410 may be a central processing unit (CPU) or other processing unit with data processing capabilities and / or instruction execution capabilities, used to issue instructions through a series of algorithmic processing flows. For example, memory 420 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.
[0063] This application also provides a vehicle that includes the electronic equipment described above. Exemplarily, the vehicle may also include other components, such as a signal transmission system for signal transmission, etc., which are not limited in this application.
[0064] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.
[0065] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0066] In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.
[0067] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.
[0068] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, the method of the invention should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.
[0069] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.
[0070] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.
[0071] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the article analysis device according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
[0072] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.
[0073] The above are merely specific embodiments or descriptions of the present invention, and the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A parking space detection method, characterized in that, include: Obtain a first image and a second image of the parking space, wherein the first image and the second image are identical images; The first image is input into a trained deep learning network-based parking space detection model to obtain parking space representation information, which includes the coordinates of the first endpoint of the entry line, the coordinates of the second endpoint of the entry line, and the angle of the dividing line. The centerline vector of the first parking space is obtained based on the parking space representation information; Based on the second image, a first parking space line and a second parking space line are obtained, and the first parking space line and the second parking space line are parallel. The centerline vector of the second parking space is obtained based on the first parking space line and the second parking space line; Match the centerline vector of the first parking space with the centerline vector of the second parking space, and determine whether the centerline vector of the first parking space and the centerline vector of the second parking space are successfully matched. When the centerline vector of the first parking space and the centerline vector of the second parking space are successfully matched, the fused parking space representation information is obtained based on the parking space representation information, the first parking space line and the second parking space line. The parking space detection result is obtained based on the fused parking space representation information; Wherein, the centerline vector of the first parking space is represented by the coordinates and angle of the centerline vector of the first parking space, and obtaining the centerline vector of the first parking space based on the parking space representation information includes: The average of the coordinates of the first endpoint of the entry line and the coordinates of the second endpoint of the entry line is used to obtain the centerline vector coordinates of the first parking space. The angle of the centerline vector of the first parking space is equal to the angle of the dividing line; The first parking space line is represented by the coordinates of the first endpoint and the second endpoint of the first parking space line; The second parking space line is represented by the coordinates of the first endpoint of the second parking space line and the coordinates of the second endpoint of the second parking space line; The second parking space centerline vector is represented by the coordinates of the second parking space centerline vector and the angle of the second parking space centerline vector. The step of obtaining the second parking space centerline vector based on the first parking space line and the second parking space line includes: The coordinates of the first endpoint of the first parking space line and the first endpoint of the second parking space line are averaged to obtain the coordinates of the first endpoint of the second center line. The coordinates of the second endpoint of the first parking space line and the second endpoint of the second parking space line are averaged to obtain the coordinates of the second endpoint of the second center line. The average of the coordinates of the first endpoint of the second centerline and the coordinates of the second endpoint of the second centerline is used to obtain the vector coordinates of the centerline of the second parking space. Based on the coordinates of the first endpoint of the second centerline and the second endpoint of the second centerline, the vector angle of the second parking space centerline is obtained.
2. The method according to claim 1, characterized in that, The step of obtaining the first parking space line and the second parking space line based on the second image includes: A grayscale image is obtained based on the second image; Perform histogram equalization on the grayscale image to obtain an equalized grayscale image; The equalized grayscale image is subjected to noise reduction processing to obtain a noise-reduced grayscale image; An adaptive thresholding method is used to convert the denoised grayscale image into a binary image; Perform an opening operation on the binary image to obtain a denoised binary image; The denoised binary image is skeletonized to obtain a binary skeleton image; The Hough line detection method is used to detect potential parking lines in the binary skeleton image. The potential parking space lines are filtered using prior knowledge to obtain the first parking space line and the second parking space line.
3. The method according to claim 2, characterized in that, The process of obtaining a grayscale image based on the second image includes: Convert the second image into an HSV image; The V channel image is extracted from the HSV image and used as the grayscale image.
4. The method according to claim 3, characterized in that, The step of matching the centerline vector of the first parking space with the centerline vector of the second parking space, and determining whether the centerline vector of the first parking space and the centerline vector of the second parking space are successfully matched, includes: Find the absolute value of the difference between the centerline vector angle of the first parking space and the centerline vector angle of the second parking space to obtain the included centerline angle. Find the distance from the centerline vector of the first parking space to the centerline vector of the second parking space to obtain the centerline distance; When the included angle of the centerline is not greater than a preset included angle and the distance between the centerlines is not greater than a preset distance, it is determined that the centerline vector of the first parking space and the centerline vector of the second parking space are successfully matched.
5. The method according to claim 4, characterized in that, The preset included angle is 30 degrees.
6. The method according to claim 1, characterized in that, The fused parking space representation information includes the coordinates of the first intersection point, the coordinates of the second intersection point, and the parking space angle. The process of obtaining the fused parking space representation information based on the parking space representation information, the first parking space line, and the second parking space line includes: The entry line is obtained based on the coordinates of the first endpoint and the second endpoint of the entry line; Based on the first parking space line and the entry line, the coordinates of the intersection point of the first parking space line and the entry line are obtained and denoted as the first intersection point coordinates; Based on the second parking space line and the entry line, the coordinates of the intersection point of the second parking space line and the entry line are obtained and denoted as the second intersection point coordinates; The parking space angle is equal to the centerline vector angle of the second parking space.
7. The method according to claim 1, characterized in that, When the centerline vector of the first parking space fails to match the centerline vector of the second parking space, the parking space detection result is obtained based on the parking space representation information.
8. The method according to claim 1, characterized in that, The training of the parking space detection model includes the following steps: Construct a dataset, which includes a training set; The sample images from the training set are input into the parking space detection model to be trained, and the parking space detection model to be trained performs the following operations: Feature extraction is performed on the sample image to obtain a parking space feature image; Parking space feature information is obtained based on the parking space feature image, and parking space representation information of the parking space in the sample image is obtained based on the parking space feature information. A loss function is obtained based on the parking space feature information, and the model parameters of the parking space detection model to be trained are updated based on the loss function. The operation is iterated until the parking space detection model to be trained converges, thus obtaining the trained parking space detection model based on the deep learning network.
9. The method according to claim 8, characterized in that, The parking space feature image includes the horizontal axis coordinate feature image of the parking space corner point, the vertical axis coordinate feature image of the parking space corner point, the horizontal axis length feature image of the entry line, the vertical axis length feature image of the entry line, the sine value feature image of the dividing line, and the cosine value feature image of the dividing line.
10. The method according to claim 9, characterized in that, The parking space feature information includes the coordinates of the parking space corner points, the length of the entry line, and the angle of the dividing line. Obtaining the parking space feature information based on the parking space feature image includes: The coordinates of the parking space corner are obtained based on the horizontal axis coordinate feature image and the vertical axis coordinate feature image of the parking space corner. The length of the entry line is obtained based on the horizontal axis length feature image and the vertical axis length feature image of the entry line; The angle of the dividing line is obtained based on the sine value feature image and the cosine value feature image of the dividing line.
11. The method according to claim 9, characterized in that, The parking space feature image also includes at least one of the following: Confidence feature image and occupancy flag feature image.
12. The method according to claim 11, characterized in that, When the parking space feature information includes the confidence feature image, the parking space feature information also includes confidence information. Obtaining the parking space feature information based on the parking space feature image includes: The confidence information is obtained based on the confidence feature image; When the parking space feature information includes the occupancy sign feature image, the parking space feature information also includes occupancy sign information. Obtaining the parking space feature information based on the parking space feature image includes: The occupancy marker information is obtained based on the occupancy marker feature image.
13. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the parking space detection method according to any one of claims 1-12.
14. A vehicle, characterized in that, The vehicle includes the electronic equipment as described in claim 13.
Citation Information
Patent Citations
Parking lot detection method and system, storage medium, and electronic device
CN107491738A
Parking space detection method and device
CN113705474A
Semantic map building and positioning method suitable for indoor parking lot
CN113903011A
Target identification method based on fusion of image information and laser radar point cloud information
CN116229408A
