Parking space detection method and device based on parking space angular points

Abstract

The invention discloses a parking space detection method and device based on parking space angular points. The method comprises the following steps: acquiring environment images of a plurality of different directions around a vehicle, and splicing a bird's-eye view; extracting features from the aerial view by using a backbone network, generating a feature map, and extracting a first angular point feature of the parking space; performing feature fusion on the feature map in a feature pyramid module, and introducing an attention mechanism in a fusion stage to perform weighted summation on each feature point to obtain a fused feature map; the detection head obtains parking space detection basic information according to the parking space first angular point features in the fusion feature map; and carrying out information integration on the parking space detection basic information by utilizing a post-processing module to generate final parking space detection result information. According to the invention, an accurate and reliable parking space detection result can be generated, false alarms and missing alarms can be reduced, and the availability of parking space detection can be improved. According to the method, more attention is paid to the accuracy of the parking stall corner, the accurate parking stall corner position is given to the downstream, tedious template matching is not needed, and the method can be suitable for various types of parking stalls.

Description

Technical Field

[0001] The present application belongs to the technical field of parking space recognition, and specifically relates to a parking space detection method and device based on parking space corner points. Background Art

[0002] As an important part of autonomous driving technology, the Automatic Parking Assistant (APA) system enables the vehicle to automatically find, identify and park in parking spaces. Parking space detection, as a key technology in automatic parking, integrates advanced sensors and complex algorithms, enabling the vehicle to actively identify various types of parking spaces, including vertical parking spaces, horizontal parking spaces, and inclined parking spaces, and accurately park in the selected parking space.

[0003] From the algorithmic perspective, the existing solutions are mainly based on traditional vision and deep learning. The core idea of ​​the traditional vision solution is to use the color and gradient information of the HSI color space to segment the parking space lines, and then use the traditional method of edge detection to extract the parking space line contour to obtain the parking space corner point position. This solution has low computational overhead, but poor accuracy, and cannot detect when encountering occlusion and shadows. The second is based on a deep learning solution. The algorithm first detects all the features containing the parking space corner points in the bird's-eye view, and then determines a complete parking space through template matching. The disadvantage is that the two-stage learning method of classification first and then detection is inefficient, and the template matching method is complex in terms of permutations and combinations, and cannot cover types such as oblique parking spaces.

[0004] Therefore, it is necessary to propose a single-stage parking space detection method that can identify various types of parking spaces. Summary of the invention

[0005] The technical purpose of this application is to provide a single-stage parking space detection method and device that can identify various types of parking spaces in order to address the problem that the existing technology cannot cover various types of parking spaces and the learning method is inefficient.

[0006] In order to achieve the above technical objectives, this application adopts the following technical solutions.

[0007] In a first aspect, an embodiment of the present application provides a parking space detection method based on parking space corner points, comprising:

[0008] Acquire multiple environmental images at different orientations around the vehicle, and stitch all of the environmental images into a bird's-eye view of a preset size;

[0009] A backbone network is used to extract features from the bird's-eye view to generate a feature map, and the feature map is divided into a plurality of positioning reference areas; for each positioning reference area, it is determined whether there is a first corner point of a parking space, and if there is the first corner point of the parking space, the feature of the first corner point of the parking space is located;

[0010] In the feature pyramid module, the feature maps of different scales are subjected to feature fusion, an attention mechanism is introduced in the fusion stage to determine the weight of each feature, and each feature is weighted summed to obtain a fused feature map;

[0011] The fused feature map is input to the detection head, and the detection head determines the position of the first corner point of the parking space according to the first corner point feature of the parking space; the entry line offset in the x-axis and y-axis directions is predicted according to the position of the first corner point of the parking space, and the basic information of parking space detection is output based on the position of the first corner point of the parking space and the entry line offset; the entry line represents the line segment of the vehicle entering the parking space, and the two end points of the entry line are the first corner point feature of the parking space and the second corner point feature of the parking space respectively;

[0012] The post-processing module is used to integrate the parking space detection basic information to generate final parking space detection result information.

[0013] In a second aspect, an embodiment of the present application provides a parking space detection device based on parking space corner points, the device comprising:

[0014] An image processing module, used to obtain multiple environmental images of different directions around the vehicle, and stitch all the environmental images into a bird's-eye view of a preset size;

[0015] The backbone network is used to extract features from the bird's-eye view to generate a feature map, and divide the feature map into a plurality of positioning reference areas; for each positioning reference area, determine whether there is a first corner point of a parking space, and if there is the first corner point of the parking space, locate the feature of the first corner point of the parking space;

[0016] A feature pyramid module is used to fuse the feature maps of different scales, introduce an attention mechanism in the fusion stage to determine the weight of each feature, perform weighted summation on each feature, and obtain a fused feature map;

[0017] A detection head is used to input a fused feature map, determine the position of the first corner point of the parking space according to the first corner point feature of the parking space; predict the offset of the entry line in the x-axis and y-axis directions according to the position of the first corner point of the parking space, and output basic parking space detection information based on the position of the first corner point of the parking space and the entry line offset; the entry line marks the line segment of the vehicle entering the parking space, and the two end points of the entry line are the first corner point feature of the parking space and the second corner point feature of the parking space respectively;

[0018] The post-processing module is used to integrate the basic parking space detection information to generate final parking space detection result information.

[0019] Compared with the prior art, the parking space detection method and device based on parking space corner points provided by the present application have the following beneficial technical effects: by acquiring environmental images of multiple different orientations around the vehicle and stitching them into a bird's-eye view, a wider perspective can be provided, thereby capturing the environmental information around the vehicle more comprehensively; performing feature fusion of different scales helps to accurately identify parking space corner points at different distances and angles, and introducing an attention mechanism to determine the weight of each feature can further improve the accuracy and robustness of detection. The present application helps to generate accurate and reliable parking space detection results, which can reduce false positives and false negatives and improve the availability of parking space detection. The present application scheme pays more attention to the accuracy of parking space corner points rather than the parking space contours, and provides the downstream with accurate parking space corner point locations. It does not require cumbersome template matching and can be applied to various types of parking spaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are for explanation purposes only and are not intended to limit the scope of the present application in any way. In addition, the shapes and proportional dimensions of the components in the drawings are only for illustration purposes and are used to help understand the present application. They do not specifically limit the shapes and proportional dimensions of the components of the present application. Under the guidance of the present application, those skilled in the art can select various possible shapes and proportional dimensions to implement the present application according to specific circumstances. In the drawings:

[0021] Figure 1 A schematic diagram of the process of a parking space detection method based on parking space corner points provided in an embodiment;

[0022] Figure 2 A schematic diagram for describing a parking space;

[0023] Figure 3 This is a schematic diagram of using four fisheye cameras to capture environmental images in four directions in an embodiment;

[0024] Figure 4 Schematic diagram of the backbone network, feature pyramid module and detection head in the embodiment;

[0025] Figure 5 A schematic diagram of a predicted rectangular frame and a real rectangular frame parking space in calculating a loss function in an embodiment;

[0026] Figure 6 Schematic diagram of the optimal parking space corner point search process in the embodiment;

[0027] Figure 7 A schematic diagram of the structure of a parking space detection device based on parking space corner points provided in an embodiment. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0029] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features.

[0030] This embodiment provides a parking space detection method based on parking space corner points. Figure 1 As shown, the following steps are included:

[0031] Step 1: Acquire multiple environmental images at different orientations around the vehicle, and stitch all environmental images into a bird's-eye view of a preset size;

[0032] Step 2: Use the backbone network to extract features from the bird's-eye view to generate a feature map, and divide the feature map into multiple positioning reference areas; for each positioning reference area, determine whether there is a first corner point of the parking space, and if there is a first corner point of the parking space, locate the first corner point feature of the parking space;

[0033] Step 3: Fuse the feature maps of different scales in the feature pyramid. In the fusion stage, introduce the attention mechanism to determine the weight of each feature, perform weighted summation on each feature, and obtain the fused feature map.

[0034] Step 4: Input the fused feature map to the detection head, which determines the position of the first corner point of the parking space according to the feature of the first corner point of the parking space; predict the entry line offset in the x-axis and y-axis directions according to the position of the first corner point of the parking space, and output the basic information of parking space detection based on the position of the first corner point of the parking space and the entry line offset; the entry line marks the line segment where the vehicle enters the parking space, and the two end points of the entry line are the first corner point feature of the parking space and the second corner point feature of the parking space respectively;

[0035] Step 5: Use the post-processing module to integrate the basic information of parking space detection and generate the final parking space detection result information.

[0036] In order to better understand the present application, the following is a parking space description definition.

[0037] Abstract the parking space into a quadrilateral, usually a rectangle and a parallelogram, such as Figure 2 As shown, the parking spaces are defined as follows:

[0038] (1) Define AB as the entry line of the parking space. The entry line marks the direction in which the vehicle enters the parking space. The direction and length of the entry line of the parking space must be determined;

[0039] (2) The first corner point (point A) of the parking space, the second corner point B of the parking space, the third corner point C of the parking space, and the fourth corner point D of the parking space are four points counterclockwise to form a parking space. AD and BC are both parking space dividing lines. The dividing lines can divide the parking space into different segments. Figure 2 In each parking space shown, there are two dividing lines, AD and BC, which include the line segment between the first corner point of the parking space and the fourth corner point of the parking space and the line segment between the second corner point of the parking space and the third corner point of the parking space. A, B, C, and D constitute the parking space counterclockwise;

[0040] (3) Take AB as an example: the entry line offset in the x-axis and y-axis directions: delta_x=x_A-x_B, delta_y=y_A-y_B; in some embodiments, delta_x and delta_y need to be normalized, that is, divided by the parking space width W and the parking space height H. After knowing the position (i.e., coordinates) of point A, the length and direction of the AB vector are determined.

[0041] (4) Predict the cosine and sinine values ​​of the dividing line AD. By inversely calculating arctan, we can obtain the angle between the dividing line and the positive direction of the x-axis, which is also called the direction angle of the dividing line.

[0042] (5) Points A, B, C, and D are all corner points of the parking space, which facilitates downstream positioning and path planning.

[0043] If the parking space is at the cutoff point of the image, then points C and D are defined at the cutoff point. Figure 2 It covers the definition of all parking corner points on the left and right sides of the bird's-eye view.

[0044] In order to ensure the accuracy and effect of parking space corner point detection, some embodiments use single-view images of fisheye cameras in different directions. In the embodiments, the installation method of the fisheye camera is as follows: Figure 3 As shown in the figure, four fisheye cameras can be installed on the front bumper, trunk, and rearview mirror of the vehicle to obtain images from the front, rear, left, and right perspectives. The field of view of the fisheye camera is greater than 180 degrees, which can completely cover the surrounding area of ​​the vehicle body. The original images of the four fisheye cameras are dedistorted, the four fisheye cameras are jointly calibrated, and fused into a bird's-eye view, which is used as the input of the parking space detection method.

[0045] The size of the spliced ​​bird's-eye view can be customized according to the number of parking spaces, such as the size of the bird's-eye view can be set to 256x256 or 512x512. In some embodiments, the bird's-eye view can be enhanced by left translation, rotation, etc. to expand data diversity. In an embodiment, the size of the output feature map is related to the number of parking spaces in the bird's-eye view.

[0046] Figure 4 This is a schematic diagram of the backbone network, feature pyramid module and detection head in the embodiment, which can be referred to Figure 4 ,In step 2, the bird's-eye view image is input into the backbone network (Backbone), and the backbone network, as a feature extraction network, extracts features from the input bird's-eye view image and generates a feature map.

[0047] After the feature map is generated, the backbone network can set anchors or grids on the feature map to divide multiple positioning reference areas (anchor areas or grid areas), and determine whether there is a first corner point of the parking space (point A) in each positioning reference area. Specifically, for each positioning reference area, the first corner point of the parking space can be detected using a trained deep learning model; if the first corner point of the parking space is detected in a certain area, it is determined that the first corner point of the parking space exists in the positioning reference area.

[0048] If there is a first corner point of the parking space, the first corner point feature of the parking space is located. Each positioning reference area needs to correspond to the point A feature. By determining the interval distance between the point A feature and the relative grid, the parking space position can be roughly located.

[0049] Please continue to refer to Figure 4 In step 3, the feature pyramid module includes the concatenation of the feature pyramid (Feature Pyramid Networks, FPN) and the last layer of the last layer of resolution output, and the feature pyramid module constitutes the Neck part. For example, in the embodiment, the 16*16 and 32*32 feature maps are fused once to better combine the high and low layer features, which is verified to be effective by experiments.

[0050] In this application, an attention mechanism is introduced in the fusion stage to determine the weight of each feature, and each feature is weighted and summed to obtain a fusion feature map, which specifically includes: taking the first corner point feature of the parking space as the query Query, and other related parking space features as the key, calculating the similarity score between the first corner point feature of the parking space and other related parking space features and normalizing them using the Softmax function as the feature weight; weighted summing the feature information values ​​of all features to obtain a new fusion feature;

[0051] The expression is as follows:

[0052] ;

[0053] Among them, Source is a series of<Key, Value> Key-value pairs, Key i Indicates the i-th other feature in Source, Value i Representation and Key i Related feature information value, i is the other feature number, Lx is the total number of other features, weight coefficient Similarity(Query, Key i ) indicates the calculation of Query and Key i The function of the similarity between .

[0054] In a specific embodiment, the function for calculating the similarity may be a dot product, a cosine similarity, or any other function that can measure the similarity between two vectors, and this is not limited.

[0055] By introducing the attention mechanism, the first corner point of the parking space (point A) and the parking space features of the entire image are weighted and fused. For example, a 32*32 feature map is obtained, which is expanded into a sequence of 32*32*channel number (channel num). The weight of each element is calculated based on the softmax similarity after the inner product calculation of the query and the key. Point A and the relevant features of the parking space in the entire image are autocorrelated, so that point A takes into account both the corner point features and the features of the entire parking space, avoiding the misdetection of parking spaces with features similar to point A.

[0056] On the feature map, one feature map pixel can be regressed to one parking space. For example, if the resolution of the backbone network input is 512*512, the resolution of the feature map after fusion output by the detection head in step 4 is 32*32, the channels of the backbone network input are 3 (RGB) or 1 (Gray grayscale image), and the channels output by the detection head are determined according to the needs. In one embodiment, the detection head outputs 13 output channels.

[0057] In an embodiment, the size and distribution of the positioning reference area on the feature map may be considered, and the position of the first corner point of the parking space may be identified according to the offset of the first corner point feature of the parking space in the positioning reference area (grid area).

[0058] As an example, the entry line offset in the x-axis and y-axis directions is obtained according to the prediction of the first corner point position of the parking space, and the parking space detection basic information is output based on the first corner point position of the parking space and the entry line offset, including:

[0059] Based on the position of the first corner point of the parking space, the regression model is used to predict the entry line offset (entry line delta_x, delta_y), the angle between the dividing line and the positive direction of the x-axis, the position of the second corner point of the parking space (point B), the position of the third corner point of the parking space (point C), and the position of the fourth corner point of the parking space (point D); based on the information obtained, the basic information of the parking space detection is finally output, such as Figure 4 As shown, the basic information of parking space detection includes parking position confidence, the first corner point position of the parking space (A_x, A_y), the second corner point position of the parking space B_x, B_y, the entry line offset, the dividing line offset (dividing line delta_x, delta_y), the sine value and cosine value of the angle (dividing line cos, sin) and whether the parking space is occupied (isOccupied).

[0060] In some embodiments, during the training process, the detection head calculates the loss using a loss function according to the first corner point position of the parking space, the second corner point position of the parking space, the third corner point position of the parking space, and the fourth corner point position of the parking space; and continuously iterates until the iteration stop condition is met.

[0061] In an embodiment, the loss function uses l2-loss plus polygon IOU-loss, and GIou, SIou or segmentation loss supervised learning can be tried. In an embodiment, a polygon loss function calculation method is adopted.

[0062] like Figure 5 As shown, the black box represents the real rectangular parking space (true parking space box), the midpoint of the real points A, B, C and D is Ogth, the green box represents the predicted parking space box, the midpoint of the predicted points A, B, C and D is Optr, and the midpoint and point A (same for B, C, D) are used as diagonal lines to make rectangular boxes (the red dotted line part is the intersection of the true parking space box and the predicted parking space box). Calculate the SIou values ​​of the four partial rectangular boxes. SIou can include the cost calculation of angle, distance, area iou and shape, and take the average of the four groups of boxes as the polygon loss. Such loss supervision makes parking space positioning more accurate.

[0063] As an example, the loss function is expressed as follows:

[0064] ;

[0065] Indicates the SIou value of the bbox-th rectangular box;

[0066] The method for determining the rectangular frame is: based on the first corner point position of the parking space, the second corner point position of the parking space, the third corner point position of the parking space and the fourth corner point position of the parking space, determine the center point; make four rectangular frames with the center point and the first corner point position of the parking space, the second corner point position of the parking space, the third corner point position of the parking space and the fourth corner point position of the parking space as the other end of the diagonal; the predicted rectangular frame and the real rectangular frame are determined in the same way.

[0067] The calculation formula is: ;

[0068] in, ; ; ;

[0069] B ptr is the area of ​​the predicted rectangular box, B gth is the area of ​​the real rectangular box composed of the real first corner point position, the second corner point position of the parking space, the third corner point position of the parking space and the fourth corner point position of the parking space, γ is the adjustment factor, ρ is the distance measure between the predicted rectangular box and the real rectangular box, θ usually represents the rotation angle of the rectangular box, t represents the position label, x and y represent points on the coordinate axis, w represents the width of the rectangular box, and h represents the height of the rectangular box.

[0070] Since the main idea of ​​the method of this application is to first confirm the position of point A of the parking space (the offset of the small grid in the feature map) to determine whether the parking space exists, that is, to compress the parking space information to point A, the detection of point A is accurate and stable. Since point B is obtained based on the direction distance regression of A x / A_y and B x / B_y (expressed by AB offset), it will jitter relative to point A.

[0071] Therefore, to address this problem, in some embodiments, an optimal corner point search method is used to perform a local feature search for point B, and the position of point B is determined secondary to resolve the instability caused by simple distance regression.

[0072] As an example, Figure 1 and Figure 6 As shown, the parking space detection method based on parking space corner points also includes: if there is a first corner point of the parking space in the positioning reference area, then positioning the second corner point (point B) feature of the parking space and the confidence of the second corner point (point B) of the parking space in the positioning reference element; after predicting the second corner point position RAW_B of the parking space (using AB offset), searching for other parking space second corner point features within the radius range with a set fixed value as the radius at the predicted second corner point position RAW_B of the parking space, if the maximum confidence value of the second corner points of other parking spaces is greater than a preset confidence threshold, updating the second angle position of the parking space according to the second corner point feature of the parking space corresponding to the maximum confidence value; otherwise, if the maximum confidence value of the second corner point of the parking space is less than or equal to the threshold, the predicted second corner point position of the parking space is still used.

[0073] In the embodiment, the position of the second corner point of the parking space can be identified based on the offset of the second corner point feature of the parking space in the positioning reference area (grid area), thereby avoiding the jitter relative to point A that may be caused by using the regression method.

[0074] In some embodiments, in step 5, the post-processing module performs sigmoid processing on the detection head output channels Ax and Ay to obtain normalized offset coordinates; deltaX and deltaY entering line BA are first normalized to obtain deltaX / W and deltaY / H, and then obtained by tanh, and the separation line channel output is also processed by tanh.

[0075] In some embodiments, Figure 1 As shown, the method also includes: in a post-processing module, sorting the parking spaces from high to low according to their confidence; starting from the parking space with the highest confidence, calculating the intersection-and-union ratio with other parking spaces in turn; if the intersection-and-union ratio of two parking spaces exceeds a preset intersection-and-union ratio threshold (such as 0.3 or 0.5), calculating the distance between the first corner points of the two parking spaces; if the distance is less than a preset distance threshold (set according to the parking space size and image resolution), the two parking spaces are considered to be adjacent parking spaces.

[0076] The parking space detection method based on parking space corner points provided in this application focuses more on the accuracy of parking space corner points rather than the parking space contours, and provides accurate parking space corner point positions to downstream. It does not require cumbersome template matching and can be applied to various types of parking spaces.

[0077] Based on the same inventive concept as the parking space detection method based on parking space corner points provided in the above embodiment, the embodiment of the present application also provides a parking space detection device based on parking space corner points, such as Figure 7 As shown, it includes image processing module, backbone network, feature pyramid module, detection head and post-processing module.

[0078] The image processing module is used to obtain environmental images of multiple different directions around the vehicle and stitch all environmental images into a bird's-eye view of a preset size.

[0079] The backbone network is used to extract features from the bird's-eye view and generate a feature map, which is divided into multiple positioning reference areas. For each positioning reference area, it is determined whether there is a first corner point of a parking space. If there is a first corner point of a parking space, the feature of the first corner point of the parking space is located.

[0080] The feature pyramid module is used to fuse feature maps of different scales. In the fusion stage, the attention mechanism is introduced to determine the weight of each feature point, and the weighted sum of each feature point is performed to obtain a fused feature map.

[0081] The detection head is used to input the fused feature map and determine the position of the first corner point of the parking space according to the feature of the first corner point of the parking space; the entry line offset in the x-axis and y-axis directions is predicted according to the position of the first corner point of the parking space, and the basic information of the parking space detection is output based on the position of the first corner point of the parking space and the entry line offset; the entry line marks the line segment where the vehicle enters the parking space, and the two end points of the entry line are the first corner point feature of the parking space and the second corner point feature of the parking space respectively.

[0082] The post-processing module is used to integrate the basic information of parking space detection and generate the final parking space detection result information.

[0083] In the embodiment, the Backbone (backbone network) and the detection head can be customized, and a stronger classification network can be selected, such as a CNN network.

[0084] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0085] This application solution focuses more on the accuracy of parking space corners rather than the parking space contours, giving the downstream accurate parking space corner positions. It does not require cumbersome template matching and can be applied to various types of parking spaces.

[0086] The parking space detection method and device based on parking space corner points provided by the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the concept of the present application and should not be understood as limiting the scope of protection of the present application.

Claims

1. A parking space detection method based on parking space corner points, characterized in that: include: Acquire multiple environmental images at different orientations around the vehicle, and stitch all of the environmental images into a bird's-eye view of a preset size; Extracting features from the bird's-eye view using a backbone network to generate a feature map, and dividing the feature map into a plurality of positioning reference areas; For each positioning reference area, determine whether there is a first corner point of the parking space, and if there is the first corner point of the parking space, locate the first corner point feature of the parking space; In the feature pyramid module, the feature maps of different scales are fused, and an attention mechanism is introduced in the fusion stage to determine the weight of each feature, and each feature is weighted summed to obtain a fused feature map; The fused feature map is input to the detection head, and the detection head determines the position of the first corner point of the parking space according to the first corner point feature of the parking space; the entry line offset in the x-axis and y-axis directions is predicted according to the position of the first corner point of the parking space, and the basic information of parking space detection is output based on the position of the first corner point of the parking space and the entry line offset; the entry line represents the line segment of the vehicle entering the parking space, and the two end points of the entry line are the first corner point feature of the parking space and the second corner point feature of the parking space respectively; The post-processing module is used to integrate the parking space detection basic information to generate final parking space detection result information.

2. The parking space detection method based on parking space corner points according to claim 1, characterized in that: In the fusion stage, the attention mechanism is introduced to determine the weight of each feature, and each feature is weighted summed, including: The first corner feature of the parking space is used as the query, and other related parking space features are used as keys. The similarity scores between the first corner feature of the parking space and other related parking space features are calculated and normalized using the Softmax function as the feature weights. The feature information values ​​of all features are weighted and summed to obtain new fusion features. The expression is as follows: ； Among them, Source is a series of<Key, Value> Key-value pairs, Key i Indicates the i-th other feature in Source, Value i Representation and Key i Related feature information value, i is the other feature number, Lx is the total number of other features, weight coefficient Similarity(Query, Key i ) indicates the calculation of Query and Key i The function of the similarity between .

3. The parking space detection method based on parking space corner points according to claim 1, characterized in that: For each positioning reference area, determine whether there is a first corner point of the parking space, including: For each of the positioning reference areas, the first corner point of the parking space is detected using the trained deep learning model; if the first corner point of the parking space is detected in a certain positioning reference area, it is determined that the first corner point of the parking space exists in the positioning reference area.

4. The parking space detection method based on parking space corner points according to claim 1, characterized in that: The method includes: obtaining an entry line offset in the x-axis and y-axis directions according to the first corner point position of the parking space, and outputting parking space detection basic information based on the first corner point position of the parking space and the entry line offset, including: Based on the position of the first corner point of the parking space, a regression model is used to predict the offset of the entry line, the angle between the dividing line and the positive direction of the x-axis, the position of the second corner point of the parking space, the position of the third corner point of the parking space, and the position of the fourth corner point of the parking space; wherein the dividing line represents a line segment that divides the parking space into different parts, and the dividing line includes a line segment between the second corner point of the parking space and the third corner point of the parking space, and a line segment between the first corner point of the parking space and the fourth corner point of the parking space, and the first corner point of the parking space, the second corner point of the parking space, the third corner point of the parking space, and the fourth corner point of the parking space constitute the parking space counterclockwise; Based on the information obtained, the basic information of the parking space detection is finally output, including the parking position reliability, the first corner point position of the parking space, the second corner point position of the parking space, the entry line offset, the dividing line offset, the sine value and cosine value of the angle, and whether the parking space is occupied.

5. The parking space detection method based on parking space corner points according to claim 4, characterized in that: The method further comprises: In the post-processing module, the vehicles are sorted from high to low according to their position confidence; Starting from the parking space with the highest confidence, calculate the intersection-and-union ratio with other parking spaces in turn. If the intersection-and-union ratio of two parking spaces exceeds the preset intersection-and-union ratio threshold, calculate the distance between the first corner points of the two parking spaces; If the distance is less than a preset distance threshold, the two parking spaces are considered to be adjacent parking spaces.

6. The parking space detection method based on parking space corner points according to claim 4, characterized in that: During the training process, the detection head calculates the loss using a loss function according to the first corner point position of the parking space, the second corner point position of the parking space, the third corner point position of the parking space, and the fourth corner point position of the parking space; and continuously iterates until the iteration stop condition is met; The expression of the loss function is as follows: ； Indicates the SIou value of the bbox-th rectangular box; The method for determining the rectangular frame is: determining the midline point based on the first corner point position of the parking space, the second corner point position of the parking space, the third corner point position of the parking space, and the fourth corner point position of the parking space; Four rectangular frames are made with the midline point and the first corner point position of the parking space, the second corner point position of the parking space, the third corner point position of the parking space and the fourth corner point position of the parking space as diagonals; The calculation formula is: ; in, ; ; ; B ptr is the area of ​​the predicted rectangular box, B gth is the area of ​​the real rectangular box composed of the first corner point position, the second corner point position, the third corner point position and the fourth corner point position of the parking space, γ is the adjustment factor, ρ is the distance measure between the predicted rectangular box and the real rectangular box, θ usually represents the rotation angle of the rectangular box, t represents the position label, x and y represent points on the coordinate axis, w represents the width of the rectangular box, and h represents the height of the rectangular box.

7. The parking space detection method based on parking space corner points according to claim 4, characterized in that: The method further comprises: If the first corner point of the parking space exists in the positioning reference area, then positioning the second corner point feature of the parking space and the confidence level of the second corner point of the parking space in the positioning reference element; After predicting the second corner point position RAW_B of the parking space, the predicted second corner point position RAW_B of the parking space is used as a radius with a set fixed value to search for second corner point features of other parking spaces within the radius. If the maximum confidence value of the second corner points of other parking spaces is greater than a preset confidence threshold, the second angle position of the parking space is updated according to the second corner point feature of the parking space corresponding to the maximum confidence value; otherwise, if the maximum confidence value of the second corner point of the parking space is less than or equal to the threshold, the predicted second corner point position of the parking space is still used.

8. A parking space detection device based on parking space corner points, characterized in that: The device comprises: An image processing module, used to obtain multiple environmental images of different directions around the vehicle, and stitch all the environmental images into a bird's-eye view of a preset size; The backbone network is used to extract features from the bird's-eye view to generate a feature map, and divide the feature map into a plurality of positioning reference areas; for each positioning reference area, determine whether there is a first corner point of a parking space, and if there is the first corner point of the parking space, locate the feature of the first corner point of the parking space; A feature pyramid module is used to fuse the feature maps of different scales, introduce an attention mechanism in the fusion stage to determine the weight of each feature, perform weighted summation on each feature, and obtain a fused feature map; A detection head is used to input the fused feature map, determine the position of the first corner point of the parking space according to the first corner point feature of the parking space; predict the offset of the entry line in the x-axis and y-axis directions according to the position of the first corner point of the parking space, and output basic parking space detection information based on the position of the first corner point of the parking space and the entry line offset; the entry line marks the line segment of the vehicle entering the parking space, and the two end points of the entry line are the first corner point feature of the parking space and the second corner point feature of the parking space respectively; The post-processing module is used to integrate the basic parking space detection information to generate final parking space detection result information.

Images