A method, system, equipment, and computer-readable storage medium for detecting parking space features.

Abstract

This invention relates to a parking space feature detection method, detection system, detection equipment, and computer-readable storage medium. The parking space feature detection method comprises the following steps: S1, acquiring a surround-view image of a specified parking space and the corresponding existing parking space corner points; S2, inputting the surround-view image of the specified parking space into a parking space feature detection model to obtain a parking space feature map containing parking space features, which at least include parking space corner points, parking space lines, and limiters; S3, decoding the parking space feature map corresponding to each parking space feature to obtain the position coordinates of each parking space feature. This invention proposes a parking space feature detection method, detection system, detection equipment, and computer-readable storage medium, which can improve the accuracy of parking space detection.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a parking space feature detection method, detection system, detection equipment, and computer-readable storage medium. Background Technology

[0002] With rapid societal development, the number of private cars in major cities is increasing, making it difficult to find an available parking space. Studies show that most drivers cannot find free parking in densely trafficked areas. Furthermore, parking lot collisions account for a significant proportion of all car accidents.

[0003] In this context, a park assist system (PAS) has emerged, which mainly consists of three parts: target location identification, route planning, and parking guidance or route tracking. As an important component of the PAS system, the target location identification function is tasked with accurately detecting available parking spaces.

[0004] Identifying parking spaces by recognizing image features from around-view monitors (AVMs) is a current trend in parking space detection technology. Due to the powerful feature extraction capabilities of Deep Convolutional Neural Networks (DCNNs), methods based on DCNNs have gained increasing attention in recent years, significantly improving the accuracy of vacant parking space detection. However, besides detecting the location of the parking space, other features such as the location of parking limiters, parking locks, the presence and location of curbs around the space, and the positions of pillars and corners around the space should also be represented. This information is more beneficial for the planning module in path planning, increasing the safety of automatic parking and reducing its error rate. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, this invention proposes a parking space feature detection method, detection system, detection equipment, and computer-readable storage medium, which can improve the accuracy of parking space detection.

[0006] Specifically, this invention proposes a parking space feature detection method, comprising the following steps:

[0007] S1, obtain the surround view image of the specified parking space and the corresponding corner point of the original parking space;

[0008] S2, input the surround view image of the specified parking space into the parking space feature detection model to obtain a parking space feature map containing parking space features, wherein the parking space features include at least parking space corner points, parking space lines and limiters;

[0009] S3, Decode the parking space feature map corresponding to each parking space feature to obtain the position coordinates of each parking space feature.

[0010] According to an embodiment of the present invention, in step S3, the decoding operation includes the following steps:

[0011] S31, respectively set the parking space feature image of the parking space corner point as parking space corner point image A, the parking space feature image of the parking space line as parking space line image A, and the parking space feature image of the limiter as limiter image A. Based on the pixel value, perform image processing on the parking space corner point image A, parking space line image A and limiter image A to obtain parking space corner point image B, parking space line image B and limiter image B;

[0012] S32, perform morphological operations on the parking space corner image B, parking space line image B and limiter image B respectively to obtain parking space corner image C, parking space line image C and limiter image C;

[0013] S33, find the outline information of the parking space corner point in the parking space corner point image C, find the position information corresponding to the parking space corner point from the parking space corner point image A based on the outline information of the parking space corner point, and update the position coordinates of the original parking space corner point with the position information of the parking space corner point or the position coordinates of the parking space line.

[0014] Find the outline information of the parking line in the parking line image C, and obtain the position coordinates of the parking line based on the outline information of the parking line.

[0015] Find the contour information of the limiter in the limiter image C, and obtain the position coordinates of the limiter based on the contour information.

[0016] According to an embodiment of the present invention, in step S33, if the contour information of the parking space corner point cannot be obtained from the parking space corner point image C, the contour information of the parking space line in the parking space line image C is searched, and the position information corresponding to the parking space corner point is obtained from the parking space line image C based on the contour information of the parking space line, and the position coordinates of the original parking space corner point are updated with the position information of the parking space corner point.

[0017] According to an embodiment of the present invention, in step S33, the operation of updating the original position coordinates of the parking space corner point with the position information of the parking space corner point includes the following steps:

[0018] S331, calculate the Euclidean distance between the location information of the found parking space corner point and the location coordinates of the original parking space corner point;

[0019] S332, construct the cost matrix between the corner point of the parking space and the original corner point of the parking space based on the Euclidean distance;

[0020] S333, obtain a set of values ​​in the cost matrix that meet the optimal matching, and update the position coordinates of the original parking space corner point with the position information of the parking space corner point corresponding to the set of values.

[0021] According to an embodiment of the present invention, the training method of the parking space feature detection model includes the following steps:

[0022] T1, collects parking space surround view images in different scenarios;

[0023] T2, Select clear and complete parking space features from the parking space surround view image and annotate them to generate an annotated map;

[0024] T3. The annotation map is preprocessed based on the Gaussian function to generate a label map of the parking space features;

[0025] T4. Compare the labeled map and the tag map, and obtain the parameter set of the parking space feature detection model based on the loss function.

[0026] According to an embodiment of the present invention, in step T3, preprocessing the labeled map based on the Gaussian function includes:

[0027] T31, initialize the pixel values ​​of the parking space corner point and its surrounding area on the labeled map based on the Gaussian function, and generate the first label map containing the parking space corner point;

[0028] T32, based on the parking space corner positions obtained from the first label image, connect the parking space corner positions to form corresponding parking space lines, and use a Gaussian function to initialize the pixel values ​​of the parking space lines and their surrounding areas to generate a second label image;

[0029] T33, based on the Gaussian function, initializes the pixel values ​​of the limiter location and its surrounding area on the labeled image to generate the third label image.

[0030] According to an embodiment of the present invention, in step T4, MSE LOSS and / or CrossEntropy Loss are used as the training loss function of the label map, and the loss weights of the parking space corner point, limiter and parking line are set as a, b and c respectively, where a > b > c.

[0031] The present invention also provides a parking space feature detection system, applicable to the aforementioned parking space feature detection method, characterized in that the parking space feature detection system comprises:

[0032] The acquisition unit is configured to acquire a surround view image of a specified parking space and the corresponding corner point of the original parking space;

[0033] The detection unit is configured to input the surround view image of the specified parking space into the parking space feature detection model to obtain a parking space feature map containing parking space features, wherein the parking space features include at least parking space corner points, parking space lines and limiters;

[0034] The decoding unit is configured to perform a decoding operation on the parking space feature map corresponding to each parking space feature to obtain the position coordinates of each parking space feature.

[0035] According to one embodiment of the present invention, the decoding unit includes:

[0036] The image processing module is used to set the parking space feature image of the parking space corner point as parking space corner point image A, the parking space feature image of the parking space line as parking space line image A, and the parking space feature image of the limiter as limiter image A, and to perform image processing on the parking space corner point image A, parking space line image A and limiter image A based on pixel values ​​to obtain parking space corner point image B, parking space line image B and limiter image B;

[0037] The calculation module is used to perform morphological operations on the parking space corner image B, the parking space line image B, and the limiter image B respectively to obtain the parking space corner image C, the parking space line image C, and the limiter image C.

[0038] The search module is used to search for the outline information of the parking space corner point in the parking space corner point image C, and find the corresponding position information of the parking space corner point from the parking space corner point image A based on the outline information of the parking space corner point; search for the outline information of the parking space line in the parking space line image C, and obtain the corresponding position coordinates of the parking space line based on the outline information of the parking space line; and search for the outline information of the limiter in the limiter image C, and obtain the corresponding position coordinates of the limiter based on the outline information of the limiter.

[0039] The update module is used to update the original position coordinates of the parking space corner point by updating the position information of the parking space corner point or the position coordinates of the parking space line.

[0040] The present invention also provides a parking method, comprising the steps of:

[0041] R1, get a surround view image of the parking space;

[0042] R2, based on the surround view image, performs parking space detection, executes parking space tracking, and generates parking space trajectory;

[0043] R3, determine whether to perform a parking operation. If yes, proceed to step R4; otherwise, proceed to step R1.

[0044] R4: The parking space selected in the parking operation is taken as the designated parking space, and the aforementioned parking space feature detection method is executed to obtain the location coordinates of the parking space features.

[0045] R5 completes the parking operation based on the stated location coordinates.

[0046] The present invention also provides a parking space feature detection device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the aforementioned parking space feature detection methods.

[0047] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the parking space feature detection method described in any of the preceding claims.

[0048] This invention provides a parking space feature detection method, detection system, detection equipment, and computer-readable storage medium. The parking space feature detection model based on heatmaps obtains a parking space feature map containing parking space features, and then the parking space feature map is decoded to obtain the position coordinates of each parking space feature, thereby improving the accuracy of parking space detection.

[0049] It should be understood that the above general description and the following detailed description of the invention are exemplary and illustrative, and are intended to provide further explanation of the invention as described in the claims. Attached Figure Description

[0050] The accompanying drawings are included to provide further explanation of the invention. They are incorporated into and constitute a part of this application. The drawings illustrate embodiments of the invention and, together with this specification, serve to explain the principles of the invention.

[0051] In the attached image:

[0052] Figure 1 A flowchart of a parking space feature detection method according to an embodiment of the present invention is shown.

[0053] Figure 2 A schematic diagram of a parking space surround view image according to an embodiment of the present invention is shown.

[0054] Figure 3 A schematic diagram of pixel processing based on a Gaussian function according to an embodiment of the present invention is shown.

[0055] Figure 4 A schematic diagram of the first label image according to an embodiment of the present invention is shown.

[0056] Figure 5 A schematic diagram of a parking space line formed by connecting the corner points of a parking space according to an embodiment of the present invention is shown.

[0057] Figure 6 A schematic diagram of a second label image according to an embodiment of the present invention is shown.

[0058] Figure 7 A schematic diagram of the third label image according to an embodiment of the present invention is shown.

[0059] Figure 8 A schematic diagram of a parking space feature detection system according to an embodiment of the present invention is shown.

[0060] Figure 9 A flowchart of a parking method according to an embodiment of the present invention is shown. Detailed Implementation

[0061] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0062] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this application or its application or use. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0063] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0064] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0065] In the description of this application, it should be understood that the orientation or positional relationship indicated by directional terms such as "front, back, up, down, left, right", "horizontal, vertical, horizontal" and "top, bottom" is usually based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing this application and simplifying the description. Unless otherwise stated, these directional terms do not indicate or imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the scope of protection of this application; the directional terms "inner" and "outer" refer to the inner and outer contours relative to the outline of each component itself.

[0066] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, these terms have no special meaning and therefore should not be construed as limiting the scope of protection of this application. In addition, although the terminology used in this application is selected from commonly known and used terms, some terms mentioned in this application's specification may have been chosen by the applicant according to his or her judgment, and their detailed meanings are explained in the relevant sections of this description. Moreover, this application should be understood not only through the actual terms used, but also through the meaning implied by each term.

[0067] Figure 1 A flowchart of a parking space feature detection method according to an embodiment of the present invention is shown. As shown, a parking space feature detection method includes the following steps:

[0068] S1: Obtain the surround view image of the specified parking space and the corresponding corner point of the original parking space. Figure 2 This diagram illustrates a parking space surround view image according to an embodiment of the present invention. A vehicle can acquire surround view images of its surroundings using a surround view acquisition device, and these images can be stitched together to obtain a complete image. Figure 2 The 360° overhead panoramic view shown is used to detect parking spaces and obtain the original corner point of the designated parking space (reference). Figure 2 The red dots (i.e., the original parking space corner points p1, p2, p3, p4) are used to crop the 360° panoramic view based on the detected original parking space corner points (see reference). Figure 2 (with the blue border in the middle) to obtain a surround view image of the specified parking space.

[0069] S2, input the surround view image of the specified parking space into the heatmap-based parking space feature detection model to obtain a parking space feature map containing parking space features. The parking space features include at least the corner points of the parking space, the parking space lines, and the limiters. As an example and not a limitation, the parking space features may also include parking space locks, the curb around the parking space, the pillars or corners around the parking space, etc.

[0070] S3 decodes the parking space feature map corresponding to each parking space feature to obtain the location coordinates of each parking space feature.

[0071] The present invention provides a parking space feature detection method that utilizes heatmap and image segmentation. It obtains parking space feature maps of each parking space feature through a parking space feature detection model, and then decodes the parking space feature maps to obtain the position coordinates of each parking space feature. This improves the accuracy of parking space detection, facilitates subsequent parking operations, and reduces safety hazards in parking lots.

[0072] Preferably, in step S3, the decoding operation includes the following steps:

[0073] S31, set the parking space corner feature image as parking space corner image A, the parking space line feature image as parking space line image A, and the parking space feature image of the limiter as limiter image A respectively. Perform image processing on the parking space corner image A, parking space line image A and limiter image A based on the pixel values ​​to obtain parking space corner image B, parking space line image B and limiter image B;

[0074] S32, perform morphological operations on the parking space corner image B, parking space line image B and limiter image B respectively to obtain the parking space corner image C, parking space line image C and limiter image C. This step is used to remove noise and filter out some too small areas.

[0075] S33, find the contour information of the parking space corner point in the parking space corner point image C, find the corresponding position information of the parking space corner point from the parking space corner point image A based on the contour information of the parking space corner point, and update the position coordinates of the original parking space corner point with the position information of the parking space corner point; find the contour information of the parking space line in the parking space line image C, and obtain the corresponding position coordinates of the parking space line based on the contour information of the parking space line; find the contour information of the limit switch in the limit switch image C, and obtain the corresponding position coordinates of the limit switch based on the contour information of the limit switch.

[0076] It should be noted that a conventional parking space detection model can detect the coordinates and angles of two corner points (e.g., p1, p2) of an existing parking space, and calculate the coordinates of the other two corner points (p3, p4) based on the depth of a standard parking space. Therefore, there is a discrepancy between the coordinates of the four corner points of the original parking space and the actual corner points, requiring step S33 to update the coordinates of the original parking space corner points. Furthermore, the surround-view image is obtained through a stitching algorithm. Due to the field of view of the surround-view acquisition device and the technical limitations of stitching, the detected parking space generally cannot be fully displayed in the surround-view image. Naturally, the closer the surround-view image is to the detected parking space, the more accurate it is. Even if other parking space detection methods are used to detect the corner points and other features of the parking space in one go, the information obtained is still inaccurate. For these reasons, step S33 is needed to update the parking space feature information of the original parking space (the specified parking space). Updating the original parking space feature information can significantly improve the accuracy of parking space detection and the safety and reliability of the parking assistance system.

[0077] Preferably, in step S33, if the contour information of the parking space corner point cannot be obtained from the parking space corner point image C, the contour information of the parking space line in the parking space line image C is searched, and the position information corresponding to the parking space corner point is obtained from the parking space line image C based on the contour information of the parking space line, and the position coordinates of the original parking space corner point are updated with the position information of the parking space corner point.

[0078] Preferably, in step S31, the pixel values ​​of the parking space corner image A, the parking space line image A, and the limiter image A are binarized to obtain the parking space corner image B, the parking space line image B, and the limiter image B. When processing the parking space corner image A, a pixel threshold of 0.5 is set, and pixel values ​​greater than the threshold are assigned a value of 255, while those less than the threshold are assigned a value of 0, thus obtaining the parking space corner image B. When processing the parking space line image A, a pixel threshold of 0.7 is set, and pixel values ​​greater than the threshold are assigned a value of 255, while those less than the threshold are assigned a value of 0, thus obtaining the parking space line image B. When processing the limiter image A, a pixel threshold of 0.6 is set, and pixel values ​​greater than the threshold are assigned a value of 255, while those less than the threshold are assigned a value of 0, thus obtaining the limiter image B.

[0079] Preferably, in step S33, the operation of updating the original position coordinates of the parking space corner point with the position information of the parking space corner point includes the following steps:

[0080] S331, calculate the Euclidean distance between the location information of the found parking space corner point and the location coordinates of the original parking space corner point;

[0081] S332, construct the cost matrix between the corner point of the parking space and the original corner point of the parking space based on Euclidean distance;

[0082] S333: Obtain a set of values ​​in the cost matrix that meet the optimal matching, and update the position coordinates of the original parking space corner points with the position information of the parking space corner points corresponding to the set of values.

[0083] If the outline information of the parking space corner point cannot be obtained from the parking space corner point image C in step S33, the position coordinates of the obtained parking space line can be used to update the coordinates of the original parking space corner point. Specifically, the position coordinates of the parking space line include the position coordinates of the four vertices of the parking space line outline (which is rectangular). The position coordinates of the four vertices are used as the position information of the parking space corner point and substituted into steps S331 to S333 to update the position coordinates of the original parking space corner point.

[0084] For example, given the original parking space corner points p1, p2, p3, and p4, in step S33, the contour information of the parking space corner points in parking space corner point image C is searched. Based on the contour information, the corresponding position information of the parking space corner points is found from parking space corner point image A, obtaining the position information of five parking space corner points a1, a2, a3, a4, and a5. In step S331, the Hungarian algorithm is used to achieve optimal matching, calculating the Euclidean distance between each pair of the found parking space corner points a1, a2, a3, a4, and a5 and the position coordinates of the original parking space corner points p1, p2, p3, and p4. In step S332, the cost matrix shown in Table 1 is established.

[0085] Table 1

[0086] p1 p2 p3 p4 a1 5 50 80 110 a2 60 5 80 110 a3 60 70 20 50 a4 20 80 40 30 a5 110 90 8 60

[0087] In step S333, the optimal matching distance threshold is set to 10. Distances greater than this threshold indicate that the two parking space corner points do not match, while distances less than or equal to this threshold indicate that the two parking space corner points match. Using the Hungarian algorithm, it is known that parking space corner points a1 and p1 are paired, a2 and p2 are paired, and a5 and p3 are paired. However, parking space corner points a3 and a4 are not paired with p1, p2, p3, or p4. For successfully paired parking space corner points, the position information of parking space corner points a1, a2, and a5 is used to update the position coordinates of p1, p2, and p3. The position coordinates of the original parking space corner points that did not match remain unchanged.

[0088] Preferably, the training method for the parking space feature detection model includes the following steps:

[0089] T1 collects surround view images of parking spaces in different scenarios. Vehicles use surround view acquisition equipment to collect a large number of surround view images of parking lots (including parking spaces) in different scenarios.

[0090] T2: Select clear and complete parking space features from the surrounding parking space image and annotate them to generate an annotated map. For example, annotate the position information of parking space corner points p1, p2, p3, p4, and the position information of limit switches s11, s12, s21, s22. (Reference) Figure 2 In the panoramic view, the limiter is represented by two short line segments. The intersection of the two ends of the short line segments with the bottom surface is the position information of the limiter. s11 and s12 are a group used to represent the position information of the two ends of one short line segment, and s21 and s22 are another group used to represent the position information of the two ends of the other short line segment.

[0091] T3 preprocesses the labeled image using a Gaussian function to generate label images for parking space features. Each parking space surround view image that meets the selection criteria is preprocessed to generate a label image corresponding to the number of parking space features. For example, if there are three parking space features, including parking space corner points, parking space lines, and parking limiters, then three corresponding label images are generated. If, in addition to the aforementioned three features, ground locks and parking space curbs are also included, then five corresponding label images are generated.

[0092] T4 compares the labeled map and the tag map, and obtains the parameter set of the parking space feature detection model based on the loss function.

[0093] The effectiveness of a parking space feature detection model training method is influenced by factors such as dataset quality and feature extraction methods. Generally, detection on surround-view stitched images yields large images with insufficient detail in the resulting parking space features. This training method, however, annotates the surround-view images of specified parking spaces. By comparing the annotated and labeled images, the obtained parking space feature information becomes more accurate and detailed, thus improving the training effect of the parking space feature detection model.

[0094] Preferably, in step T3, the preprocessing of the labeled map based on the Gaussian function includes:

[0095] T31 initializes the pixel values ​​of the parking space corner points and their surrounding pixels on the labeled image using a Gaussian function, generating the first label image containing the parking space corner points. Based on the positions of parking space corner points p1, p2, p3, and p4, the Gaussian function is used to initialize the pixel values ​​of the parking space corner points and their surrounding pixels. Taking parking space corner point p1 as an example, all pixels around p1, p2, p3, and p4 are set to 0. Figure 3 The diagram illustrates pixel processing based on a Gaussian function according to an embodiment of the present invention. The vertex of the cone corresponds to the location of the parking space corner point p1, where the pixel value is 1. The surrounding pixel values ​​decrease sequentially according to the Gaussian function until they reach 0. Figure 4 This diagram illustrates the first label image according to an embodiment of the present invention. The positions of all parking space corner points and their surrounding pixel values ​​are initialized to generate a label image as shown below. Figure 4The first label image shown indicates the locations of parking space corner points p1, p2, p3, and p4, marked by white dots.

[0096] T32, based on the parking space corner positions obtained from the first label image, connects the parking space corner positions to form corresponding parking space lines, and uses a Gaussian function to initialize the pixel values ​​at the location of the parking space lines and their surroundings to generate the second label image. Figure 5 A schematic diagram of a parking space line formed by connecting the corner points of a parking space according to an embodiment of the present invention is shown. Figure 6 A schematic diagram of a second label image according to an embodiment of the present invention is shown. (See reference) Figure 5 Connecting adjacent parking space corner points forms parking lines. Points on the parking lines and their surrounding areas are sequentially initialized using a Gaussian function. Adjusting the standard deviation of the Gaussian function controls the size of the non-zero region, making it approximate the actual width of the parking line, thus generating... Figure 6 The second label image shown.

[0097] T33 initializes the pixel values ​​of the limiter location and its surrounding area on the annotation map using a Gaussian function, generating the third label map. Figure 7 This diagram illustrates a third label image according to an embodiment of the present invention. Based on the position information of limiters s11, s12, s21, and s22, the points on the lines connecting s11 and s12, and s21 and s22, and their surrounding areas, are sequentially initialized using a Gaussian function. The standard deviation of the Gaussian function is adjusted to control the line width, generating... Figure 7 The third label image shown.

[0098] Parking space corner points and limiters are actually used for key point detection, mainly through direct regression of key points and the use of heatmaps. If the regression method is used directly, it provides less supervision information, converges slowly, has larger errors, and performs slightly worse in actual training. The labels of the heatmap network are generally generated using Gaussian functions. When used for key point detection, it provides richer supervision information, converges quickly, and has higher localization accuracy of key points. Generally speaking, several layers of heatmaps are needed to detect several key points. Because the computing power of edge devices is limited, in order to accelerate model inference, two layers of heatmaps are used for detection here. One layer is used to detect parking space corner points, and the other layer is used to detect limiters. Combined with a unique decoding method, the coordinate information of the corresponding parking space features can be obtained. In addition, the parking line layer borrows the idea of ​​image segmentation. However, if image segmentation is used for labeling, the cost is high and the process is slow. Therefore, in step T32, the second label image of the parking line is obtained from the first label image of the parking space corner point. This is equivalent to obtaining the parking line label image through image segmentation, which can effectively reduce the cost. (If other parking space features are labeled, a similar labeling method can be used, such as labeling several key points along the wall and obtaining the corresponding label image using a method similar to step T32).

[0099] Preferably, in step T4, MSE Loss and / or CrossEntropy Loss are used as the training loss function for the label image, and the loss weights for parking space corners, limiters, and parking lines are set to a, b, and c, respectively, where a > b > c. The first label image represents the location information of the parking space corners. To accurately obtain the location coordinates of the parking space corners, MSE Loss is used as the training loss function to construct a two-dimensional Gaussian plane, and local maxima are used as focal positions during decoding. The second label image represents the outline of the parking lines. To obtain the accurate range of the parking lines, considering that there are many non-existent parking lines and the sample loss has an imbalance problem, focal loss (an upgraded version of CrossEntropy Loss) is used as the loss function to classify the parking lines. The third label image represents the location information of the limiters. The limiters are represented as a narrow short line segment. It is necessary to know the specific location of this short line segment, so MSE Loss is used as the training loss function to construct a two-dimensional Gaussian plane. Among the three parking space features, the degree of influence on the update of parking space detection information is as follows: parking space corner, limiter, and parking line. Therefore, the loss weights for parking space corner, limiter, and parking line can be set to 5, 3, and 2, respectively. After preparing a large number of parking space surround view images, the parking space feature detection model can be trained.

[0100] Preferably, the parking space feature detection model consists of three parts: an image feature encoder, an image feature fusion module, and an image decoder. The image feature encoder consists of multiple convolutional layers, activation layers, and a batch normalization module. The image feature encoder extracts parking space features from the surround view image of the parking space. The extracted parking space features include low-level semantic features and high-level semantic features. Among them, the low-level semantic features have higher resolution and contain more location and detail information. The corresponding parking space features are the location and detail information features of the surround view image. The high-level semantic features have stronger semantic information, but have very low resolution and poor ability to perceive details. The extracted parking space features are input into the image feature fusion module. The image feature fusion module fuses low-level and high-level semantic features to obtain fused features. First, pooling layers with different parameters are used to pool the input features to obtain feature maps of different sizes. Convolution and summation operations are performed on each feature map, and an upsampling layer is used to scale the output width and height to the width and height of the image feature encoder. The multiple upsampled feature layers are then merged with the output of the image feature encoder to form fused features. This fused feature not only aggregates features from different layers but also ensures the integrity and diversity of parking space features. The obtained fused features are then input to the image decoder. The image decoder decodes the fused features. This image decoder consists of multiple convolutional layers, dilated convolutional layers, transposed convolutional layers, and sigmoid layers. The convolutional layers are used for initial decoding of the fused features. The dilated convolutional layers are used to expand the receptive field and capture multi-scale contextual information. The transposed convolutional layers are used to change the width and height of the feature map. The sigmoid layer is the final processing step of the image decoder, which normalizes the output result and sets the result between 0 and 1 to accelerate the convergence speed. The network finally outputs three layers of parking space feature maps, corresponding to the parking space corner point layer, parking space line layer, and limiter layer, respectively.

[0101] Figure 8 A schematic diagram of a parking space feature detection system according to an embodiment of the present invention is shown. As shown in the figure, the present invention also provides a parking space feature detection system 800. This parking space feature detection system 800 is applicable to the aforementioned parking space feature detection method. The parking space feature detection system 800 mainly includes an acquisition unit 801, a detection unit 802, and a decoding unit 803.

[0102] The acquisition unit 801 is configured to acquire a surround view image of a specified parking space and the corresponding corner point of the original parking space.

[0103] The detection unit 802 is configured to input the surround view image of the specified parking space into the heatmap-based parking space feature detection model to obtain a parking space feature map containing parking space features, which include at least parking space corner points, parking space lines and limiters.

[0104] The decoding unit 803 is configured to perform a decoding operation on the parking space feature map corresponding to each parking space feature to obtain the position coordinates of each parking space feature.

[0105] Preferably, the decoding unit 803 includes:

[0106] The image processing module 804 is used to set the parking space corner image as parking space corner image A, the parking space line image as parking space line image A, and the parking space feature image of the limiter as limiter image A, respectively, and to perform image processing on the parking space corner image A, the parking space line image A and the limiter image A based on the pixel values ​​to obtain parking space corner image B, parking space line image B and limiter image B;

[0107] The calculation module 805 is used to perform morphological operations on the parking space corner image B, the parking space line image B, and the limiter image B respectively to obtain the parking space corner image C, the parking space line image C, and the limiter image C.

[0108] The lookup module 806 is used to look up the outline information of the parking space corner point in the parking space corner point image C, and find the corresponding position information of the parking space corner point from the parking space corner point image A based on the outline information of the parking space corner point; look up the outline information of the parking space line in the parking space line image C, and obtain the corresponding position coordinates of the parking space line based on the outline information of the parking space line; and look up the outline information of the limiter in the limiter image C, and obtain the corresponding position coordinates of the limiter based on the outline information of the limiter.

[0109] The update module 807 is used to update the original coordinates of the parking space corner points or the coordinates of the parking space lines.

[0110] Figure 9 A flowchart of a parking method according to an embodiment of the present invention is shown. As shown, the present invention also provides a parking method comprising the following steps:

[0111] R1, get a surround view image of the parking space;

[0112] R2 performs parking space detection based on the surround-view image, performs parking space tracking, and generates a parking space trajectory. The parking space detection here uses existing technology, which can typically obtain the position information of the four vertices (corner points) of the parking space.

[0113] Step R3 determines whether to perform a parking operation. If yes, proceed to step R4; otherwise, return to step R1. If the car owner has not confirmed the parking operation, return to step R1 to continue searching for a parking space.

[0114] R4: The parking space selected in the parking operation is taken as the designated parking space. The aforementioned parking space feature detection method is executed. The surround view image of the designated parking space is input into the heatmap-based parking space feature detection model to obtain a parking space feature map containing parking space features. The parking space feature map corresponding to each parking space feature is decoded to obtain the position coordinates of each parking space feature.

[0115] R5 completes parking operations based on continuously updated location coordinates.

[0116] The present invention also provides a parking space feature detection device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of any of the aforementioned parking space feature detection methods.

[0117] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the aforementioned parking space feature detection methods.

[0118] The specific implementation methods and technical effects of the parking space feature detection system, parking method, parking space feature detection equipment, and computer-readable storage medium can be found in the embodiments of the parking space feature detection method provided by the present invention, and will not be repeated here.

[0119] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0120] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternatives, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0121] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information to / from the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0122] In one or more exemplary embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. As used in this article, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0123] It will be apparent to those skilled in the art that various modifications and variations can be made to the exemplary embodiments described above without departing from the spirit and scope of the invention. Therefore, it is intended that this invention cover modifications and variations falling within the scope of the appended claims and their equivalents.

Claims

1. A method for detecting parking space features, characterized in that, Including the following steps: S1, obtain the surround view image of the specified parking space and the corresponding corner point of the original parking space; S2, input the surround view image of the specified parking space into the parking space feature detection model to obtain a parking space feature map containing parking space features, wherein the parking space features include at least parking space corner points, parking space lines and limiters; S3, Decode the parking space feature map corresponding to each parking space feature to obtain the position coordinates of each parking space feature; The training method for the parking space feature detection model includes the following steps: T1, collects parking space surround view images in different scenarios; T2, Select clear and complete parking space features from the parking space surround view image and annotate them to generate an annotated map; T3. Preprocess the labeled map based on the Gaussian function to generate a label map of the parking space features, including the following steps: T31, initialize the pixel values ​​of the parking space corner point and its surrounding area on the labeled map based on the Gaussian function, and generate the first label map containing the parking space corner point; T32, based on the parking space corner positions obtained from the first label image, connect the parking space corner positions to form corresponding parking space lines, and use a Gaussian function to initialize the pixel values ​​of the parking space lines and their surrounding areas to generate a second label image; T33, initialize the pixel values ​​of the limiter location and its surrounding area on the labeled image based on the Gaussian function to generate the third label image; T4. Compare the labeled map and the tag map, and obtain the parameter set of the parking space feature detection model based on the loss function.

2. The parking space feature detection method as described in claim 1, characterized in that, In step S3, the decoding operation includes the following steps: S31, respectively set the parking space feature image of the parking space corner point as parking space corner point image A, the parking space feature image of the parking space line as parking space line image A, and the parking space feature image of the limiter as limiter image A. Based on the pixel value, perform image processing on the parking space corner point image A, parking space line image A and limiter image A to obtain parking space corner point image B, parking space line image B and limiter image B; S32, perform morphological operations on the parking space corner image B, parking space line image B and limiter image B respectively to obtain parking space corner image C, parking space line image C and limiter image C; S33, find the outline information of the parking space corner point in the parking space corner point image C, find the position information corresponding to the parking space corner point from the parking space corner point image A based on the outline information of the parking space corner point, and update the original position coordinates of the parking space corner point with the position information of the parking space corner point. Find the outline information of the parking line in the parking line image C, and obtain the position coordinates of the parking line based on the outline information of the parking line. Find the contour information of the limiter in the limiter image C, and obtain the position coordinates of the limiter based on the contour information.

3. The parking space feature detection method as described in claim 2, characterized in that, In step S33, if the outline information of the parking space corner point cannot be obtained from the parking space corner point image C, then the outline information of the parking space line in the parking space line image C is searched, and the position information corresponding to the parking space corner point is obtained from the parking space line image C based on the outline information of the parking space line. The position coordinates of the original parking space corner point are updated using the position information of the parking space corner point.

4. The parking space feature detection method as described in claim 2 or 3, characterized in that, In step S33, the operation of updating the original position coordinates of the parking space corner point with the position information of the parking space corner point includes the following steps: S331, calculate the Euclidean distance between the location information of the found parking space corner point and the location coordinates of the original parking space corner point; S332, construct the cost matrix between the corner point of the parking space and the original corner point of the parking space based on the Euclidean distance; S333, obtain a set of values ​​in the cost matrix that meet the optimal matching, and update the position coordinates of the original parking space corner point with the position information of the parking space corner point corresponding to the set of values.

5. The parking space feature detection method as described in claim 1, characterized in that, In step T4, MSE LOSS and / or CrossEntropy Loss are used as the training loss function for the label map, and the loss weights for the parking space corner, limiter and parking line are set to a, b and c respectively, where a > b > c.

6. A parking space feature detection system, applicable to the parking space feature detection method of claim 1, characterized in that, The parking space feature detection system includes: The acquisition unit is configured to acquire a surround view image of a specified parking space and the corresponding corner point of the original parking space; The detection unit is configured to input the surround view image of the specified parking space into the parking space feature detection model to obtain a parking space feature map containing parking space features, wherein the parking space features include at least parking space corner points, parking space lines and limiters; The decoding unit is configured to perform a decoding operation on the parking space feature map corresponding to each parking space feature to obtain the position coordinates of each parking space feature.

7. The parking space feature detection system as described in claim 6, characterized in that, The decoding unit includes: The image processing module is used to set the parking space feature image of the parking space corner point as parking space corner point image A, the parking space feature image of the parking space line as parking space line image A, and the parking space feature image of the limiter as limiter image A, and to perform image processing on the parking space corner point image A, parking space line image A and limiter image A based on pixel values ​​to obtain parking space corner point image B, parking space line image B and limiter image B; The calculation module is used to perform morphological operations on the parking space corner image B, the parking space line image B, and the limiter image B respectively to obtain the parking space corner image C, the parking space line image C, and the limiter image C. The search module is used to search for the outline information of the parking space corner point in the parking space corner point image C, and find the corresponding position information of the parking space corner point from the parking space corner point image A based on the outline information of the parking space corner point; search for the outline information of the parking space line in the parking space line image C, and obtain the corresponding position coordinates of the parking space line based on the outline information of the parking space line; and search for the outline information of the limiter in the limiter image C, and obtain the corresponding position coordinates of the limiter based on the outline information of the limiter. The update module is used to update the original position coordinates of the parking space corner point by updating the position information of the parking space corner point or the position coordinates of the parking space line.

8. A parking method, characterized in that, Including the following steps: R1, get a surround view image of the parking space; R2, based on the surround view image, performs parking space detection, executes parking space tracking, and generates parking space trajectory; R3, determine whether to perform a parking operation. If yes, proceed to step R4; otherwise, proceed to step R1. R4, taking the parking space selected in the parking operation as the designated parking space, executes the parking space feature detection method according to any one of claims 1 to 4 to obtain the position coordinates of the parking space feature; R5 completes the parking operation based on the stated location coordinates.

9. A parking space feature detection device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the parking space feature detection method as described in any one of claims 1-5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the parking space feature detection method as described in any one of claims 1-5.

Images