Automatic parking space detection method and device, computer equipment and medium

Abstract

The embodiment of the invention provides an automatic parking space detection method, and relates to the technical field of automatic parking, and the detection method comprises the steps: obtaining a multi-view image; based on the calibrated internal and external parameters of the plurality of image acquisition devices, splicing the multi-view images to obtain an all-round spliced image; inputting the look-around spliced graph into a multi-task network, and predicting position parameters of parking spaces; determining the type of the parking space according to the position parameters of the parking space; and based on the distortion stretching characteristic of the look-around spliced graph, dividing the look-around spliced graph into a plurality of areas, and determining an empty parking space in each area. According to the detection method, a multi-task network model is designed and covers semantic segmentation, key point detection and direction angle prediction, so that the information of predicted parking spaces is more comprehensive, and a region division and multi-strategy fusion method is provided. The method is more stable and effective in detection of empty parking spaces in complex scenes such as parking space shielding, discontinuous spaced parking spaces, non-vertical inclined parking spaces and the like caused by distortion of the surround view.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic parking, and in particular to an automatic parking space detection method, device, computer equipment and medium. Background Art

[0002] With the rapid development of autonomous driving technology, automatic parking has become an important part of smart cars. In the automatic parking system, accurate detection of empty parking spaces is a prerequisite for automatic parking. However, existing automatic parking systems still have certain limitations in parking space detection. Most of the existing automatic parking parking space detection methods obtain the corner points of the parking space through the detection network, or obtain the parking space detection frame to determine the location of the parking space. This method is relatively effective for parking spaces in simple scenes, but it is often less effective for complex scenes such as parking space occlusion caused by surround view distortion, non-continuous interval parking spaces, and non-vertical diagonal parking spaces.

[0003] Patent CN202310670523.6 proposes a single-stage parking space detection method based on a surround view. By constructing a detection network, the left and right corner point information of the parking space entry line, as well as the type of parking space, the occupancy of the parking space, etc. are obtained. The network structure designed by this patent belongs to a multi-task network, which can obtain more information about the parking space. It not only directly predicts the corner point information of the parking space, but also predicts the type of parking space and whether there are vehicles occupying the parking space, etc., so that the detected empty parking space can be directly obtained. However, this end-to-end method of directly predicting empty parking spaces in a single stage does not solve the parking space occlusion caused by the distortion of the surround view, and the detection problem of non-vertical oblique parking spaces.

[0004] Therefore, it is necessary to provide a parking space detection method that can also be used in non-standard parking spaces or complex environments. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides an automatic parking space detection method based on a surround view to solve the problem that the parking space detection method in the prior art cannot be applied to complex environments. The detection method includes:

[0006] Acquire multi-view images using multiple image acquisition devices arranged at different positions of the vehicle;

[0007] Based on the calibrated internal and external parameters of multiple image acquisition devices, the multi-view images are stitched together using image stitching technology to obtain a surround stitching image from a bird's-eye view.

[0008] Inputting the surround stitching image into a multi-task network to predict the position parameters of the parking space, wherein the position parameters include corner point information, semantic segmentation information of the edge line, edge line direction angle, and occupancy information;

[0009] Determine the entry line width of the parking space according to the location parameters of the parking space, and determine the parking space type according to the entry line width or the sideline direction angle;

[0010] Based on the distortion and stretching characteristics of the surround view mosaic map, the surround view mosaic map is divided into multiple regions. In each region, the truncation line of the parking space edge line is determined according to the semantic segmentation line information of the two corner points and edge line of the entry line. According to the different relationships of the truncation lines of the parking space edge line, the empty parking spaces in each region are determined.

[0011] The automatic parking space detection method provided by the present invention also has the following characteristics: the multi-view images are obtained based on multiple image acquisition devices arranged at different positions of the vehicle, including installing four fisheye cameras at four positions of the vehicle to obtain images of the surrounding areas.

[0012] The automatic parking space detection method provided by the present invention also has the following characteristics: the surround view splicing graph is input into a multi-task network to predict the location parameters of the parking space, including:

[0013] The surround view stitching image is input into a multi-task network. For the input surround view stitching image, prediction information of each branch is obtained through a multi-layer convolutional neural network, the corner point information is output through the key point branch of the multi-task network, the semantic segmentation information of the edge line is output through the segmentation branch of the multi-task network, the edge line direction angle is output through the direction angle branch of the multi-task network, and the occupancy situation information is output through the occupancy classification branch of the multi-task network.

[0014] The automatic parking space detection method provided by the present invention also has the following characteristics: determining the entry line width of the parking space according to the position parameters of the parking space, and determining the parking space type according to the entry line width or the sideline direction angle, including:

[0015] Calculate the width of the entry line according to the coordinates of the left and right corner points of the entry line;

[0016] Comparing the entry line width with a preset width and a preset length of the parking space respectively to determine whether the current parking space is a vertical parking space or a parallel parking space; or,

[0017] According to the difference between the sideline direction angle and 90 degrees, it is determined whether the current parking space is an oblique parking space, wherein the sideline direction angle is the angle between the entrance line and the left boundary line of the parking space.

[0018] The automatic parking space detection method provided by the present invention also has the following characteristics, and further comprises:

[0019] If it is determined based on the semantic segmentation information of the edge line of the parking space that there is no line between the left and right corner points of the entry line, and the width of the entry line calculated based on the left and right corner points of the entry line is smaller than the preset width of the parking space, then it is determined that the two parking spaces to which the left and right corner points of the entry line respectively belong are interval parking spaces.

[0020] The automatic parking space detection method provided by the present invention also has the following characteristics: based on the distortion stretching characteristics of the surround view splicing image, the surround view splicing image is divided into multiple areas, including:

[0021] Assuming that the vehicle is always located in the center of the surround view mosaic map, the surround view mosaic map is divided into four areas: the first area is located in the left front of the vehicle, the second area is located in the right front of the vehicle, the third area is located in the right rear of the vehicle, and the fourth area is located in the left rear of the vehicle.

[0022] The automatic parking space detection method provided by the present invention also has the following characteristics: in each area, according to the semantic segmentation line information of the two corner points of the entry line and the edge line, the truncation line of the parking space edge line is determined, and according to the judgment method of different relationships of the truncation lines of the parking space edge line, the empty parking space in each area is determined, including:

[0023] Determine that the area where the width w / 2 or more of the entry line of the current parking space falls is the area to which the current parking space belongs;

[0024] In the first area and the third area, a parking space whose truncation line of the parking space boundary line satisfies the following formula is determined as an empty parking space:

[0025] l0>max(l1,c)

[0026] Wherein, c is the preset length of the parking space, l0 is the cutoff line of the left boundary line of the parking space, and l1 is the cutoff line of the right boundary line of the parking space;

[0027] In the second area and the fourth area, a parking space whose truncation line of the parking space boundary line satisfies the following formula is determined as an empty parking space:

[0028] l1>max(l0,c).

[0029] A second object of the present invention is to provide an automatic parking space detection device, comprising:

[0030] An image acquisition module is used to acquire multi-view images of different positions of the vehicle;

[0031] An image processing module is used to stitch the multi-view images by image stitching technology based on the internal and external parameters of the calibrated multiple image acquisition devices to obtain a surround stitching image of a bird's-eye view;

[0032] A calculation module, used for outputting the surround stitching graph to a multi-task network, predicting the position parameters of the parking space, and determining the type of the parking space according to the position parameter information;

[0033] The judgment module is used to divide the surround view splicing image into multiple areas based on the distortion and stretching characteristics of the surround view splicing image, and determine the empty parking spaces in each area.

[0034] A third object of the present invention is to provide a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the aforementioned automatic parking space detection methods when executing the computer program.

[0035] A fourth object of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores instructions for executing any one of the automatic parking space detection methods described above.

[0036] Compared with the prior art, the beneficial effects that can be achieved by at least one of the above-mentioned technical solutions adopted in the embodiments of this specification include at least the following: the technical solution provided by the present invention designs a multi-task network model, covering semantic segmentation, key point detection and direction angle prediction, so that the information of predicted parking spaces is more comprehensive, and proposes a method of area division and multi-strategy fusion, which is more stable and effective for the detection of empty parking spaces in complex scenarios such as parking space occlusion caused by surround view distortion, non-continuous interval parking spaces, and non-vertical diagonal parking spaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 is a flow chart of a parking space detection method provided by an embodiment of the present invention;

[0039] Figure 2 is a structural diagram of empty parking space detection based on a surround view in an embodiment of the present invention;

[0040] Figure 3 is a structural block diagram of a multi-task network provided by an embodiment of the present invention;

[0041] Figure 4Schematic diagram of the structure of three commonly used parking space types in the embodiments of the present invention;

[0042] Figure 5 is a schematic diagram of the structure of two special parking space types in an embodiment of the present invention;

[0043] Figure 6 is a schematic diagram of an error detection parking space in an interval area of ​​an interval parking space in an embodiment of the present invention;

[0044] Figure 7 is a schematic diagram of the area division of the ring view in an embodiment of the present invention;

[0045] Figure 8 yes Figure 7 Schematic diagram of vacant parking spaces blocked in different areas;

[0046] Fig. 9 yes Figure 7 Schematic diagram of the semantic segmentation edge of the empty parking space in area I.

[0047] Fig.10 is a structural block diagram of a computer device provided by an embodiment of the present invention;

[0048] Fig.11 It is a structural block diagram of an automatic parking space detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0050] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0051] In an embodiment of the present invention, a method for detecting an automatic parking space based on a surround view is provided. Figure 1 As shown, the detection method comprises:

[0052] S1: Acquire multi-view images using multiple image acquisition devices set at different positions of the vehicle;

[0053] S2: Based on the calibrated internal and external parameters of multiple image acquisition devices, the multi-view images are stitched together using image stitching technology to obtain a surround stitching image from a bird's-eye view;

[0054] S3: Inputting the surround stitching image into a multi-task network to predict the location parameters of the parking space, where the location parameters include corner point information, semantic segmentation information of the edge line, edge line direction angle, and occupancy information;

[0055] S4: determining the entry line width of the parking space according to the position parameters of the parking space, and determining the parking space type according to the entry line width or the sideline direction angle;

[0056] S5: Based on the distortion and stretching characteristics of the surround view mosaic map, the surround view mosaic map is divided into multiple regions. In each region, the truncation line of the parking space edge line is determined according to the semantic segmentation line information of the two corner points and edge line of the entry line. The empty parking space in each region is determined according to the different relationships of the truncation lines of the parking space edge line.

[0057] In the above embodiment, each information is listed as follows:

[0058] (1) Information about the corner points of the parking space: the coordinates of the four corner points of the parking space and the confidence scores corresponding to the corner points. When the confidence score of a corner point is low, it means that the corner point is blocked. Generally, the two corner points near the parking space entry line have higher confidence scores, while the two corner points behind the parking space have lower confidence scores.

[0059] (2) Parking space occupancy information: A probability value output by the occupancy classification branch in the multi-task network needs to be compared with the threshold to reflect whether the parking space is occupied by a vehicle and is used to obtain a preliminary estimate of empty parking spaces.

[0060] (3) Parking space segmentation information: The semantic segmentation lines corresponding to the four edges of the parking space are used to assist in the determination of continuous parking spaces and intermittent parking spaces, and also to assist in the detection of empty parking spaces that are blocked by parking spaces due to the distortion of the surround view.

[0061] (4) Direction angle information of the parking space: the angle between the entrance line and the boundary line of the parking space, which is defined here as the angle on the left side of the parking space.

[0062] The technical solution provided in the above embodiment obtains more information about parking spaces based on a multi-task network, and constructs regional divisions in combination with the surround view characteristics. The integration of multiple logic strategies can make empty parking space detection more accurate and have a wide range of applications.

[0063] In some embodiments, the multi-view images are acquired using a plurality of image acquisition devices disposed at different positions of the vehicle, including installing four fisheye cameras at four positions of the vehicle, front, back, left, and right, to acquire images of the surrounding areas.

[0064] In some embodiments, inputting the surround view stitching graph into a multi-task network to predict the location parameters of the parking space includes:

[0065] The surround view stitching image is input into a multi-task network. For the input surround view stitching image, prediction information of each branch is obtained through a multi-layer convolutional neural network, the corner point information is output through the key point branch of the multi-task network, the semantic segmentation information of the edge line is output through the segmentation branch of the multi-task network, the edge line direction angle is output through the direction angle branch of the multi-task network, and the occupancy situation information is output through the occupancy classification branch of the multi-task network.

[0066] In the above embodiment, if Figure 2 As shown in the figure, a multi-task network is constructed. For the input surround stitching image, the prediction information of each branch is obtained through a multi-layer convolutional neural network (including Backbone network structure and Head network structure). Among them:

[0067] (1) The key point branch consists of the "parking space corner point output header" and the "parking space corner point confidence score output header". The "parking space corner point output header" is used to obtain the coordinates of the four corner points of the parking space, and the "parking space corner point confidence score output header" is used to obtain the confidence score of the corner point, which can measure the credibility of the corner point. The visibility information of the corner point can be obtained by whether it meets the threshold.

[0068] (2) The segmentation branch is the “edgeline semantic segmentation output head”, which is used to detect the edges of parking spaces (including entry lines and boundary lines) and obtain semantic segmentation information of the edges.

[0069] (3) The direction angle branch is the "edge line direction angle output head". It is used to measure the angle between the boundary line and the entry line, which can be used to assist in determining the type of parking space.

[0070] (4) The occupancy classification branch is the “parking space occupancy classification output head”. It is used to obtain preliminary vacant parking space information, and then combined with the area division strategy to obtain more accurate vacant parking spaces.

[0071] For the key point branch, define the four corner points of the parking space. Starting from the left end point of the parking space entrance line in a clockwise direction, the order of the four corner points is: upper left corner, upper right corner, lower right corner, and lower left corner. Each point is represented by ((x0, y0, v0), (x1, y1, v1), (x2, y2, v2), (x3, y3, v3)), where x i ,y i are the horizontal and vertical coordinates of the image of the i-th key point, v i It is the visible attribute, 0 means the key point is invisible, 1 means the key point is visible.

[0072] Assume the predicted value of “parking space corner output head” is z a , the MSE loss function is used for training, the specific formula is as follows:

[0073]

[0074] Where N is the number of training samples, z gt is the true value label of the corner point of the parking space in the training sample, which is composed of the true value coordinates of the four corner points (x0, y0, x1, y1, x2, y2, x3, y3). In actual training, z can be encoded gt Converted into the form of difference offset with the anchor point, then the predicted z a It will also be an offset, and the subsequent decoding process is required to add the offset to the anchor point to get the final predicted coordinate value.

[0075] Assume that the predicted value of “parking space corner confidence output head” is s a , the Smooth L1 loss function is used for training. The specific formula is as follows:

[0076]

[0077] where s gt is the confidence score true value label of the parking space corner point in the training sample, which is in the form of (v0, v1, v2, v3). i The value of (i=1,2,3,4) is 0 or 1, and the confidence score during prediction is a probability value between 0 and 1, which needs to be compared with the threshold and is subsequently used to determine the visibility of the corner point.

[0078] For the segmentation branch, let the predicted value of the "edge semantic segmentation output head" be X, and use Dice Loss training for training. The specific formula is as follows:

[0079]

[0080] Where X is the set of pixel categories of the predicted segmented image, Y is the set of pixel categories of the real segmented image, |X∩Y| represents the common elements between sets X and Y, |X| represents the number of elements in set X, and |Y| represents the number of elements in set Y. The above formula can be used to measure the overlap between the predicted semantic segmentation pixels X and the real semantic segmentation pixels Y, and its value is between 0 and 1. The closer the predicted semantic segmentation result is to the real semantic segmentation, the closer DiceLoss is to 0.

[0081] For the direction angle branch, let the predicted value of the "edge direction angle output head" be p a, the Smooth L1 loss function is also used for training. The formula is the same as the loss function formula of the confidence score above, and the formula is expressed as:

[0082]

[0083] where p gt It is the angle between the parking space entry line and the boundary line, which is defined as the angle with the left end point of the entry line as the vertex and the entry line and the left boundary line as the two sides. Its true value label is generally calculated from the annotated parking space corner points. When the parking space corner points are incomplete and the true value of the angle cannot be obtained, its true value label is calculated from the semantic segmentation line of the boundary line.

[0084] For the occupancy classification branch, let the prediction value of the “parking space occupancy classification output head” be y a , the cross entropy loss function is used for training. The specific formula is as follows:

[0085] H(y a )=-(y gt log(y a )+(1-y gt )log(1-y a ))

[0086] where y gt is the true value of the parking space occupancy, and its occupancy classification value is 0 or 1, 0 means that the parking space is not occupied (i.e., an empty parking space), and 1 means that the parking space is occupied. a It is a probability value between 0 and 1, which needs to be compared with the threshold to determine whether the predicted current parking space is occupied by a vehicle.

[0087] The overall loss function is defined as:

[0088] Loss = α1MSE(z a )+α2SLL1(s a )+α3DL(X)+α4SLL1(p a )+α5H(y a )

[0089] Among them, α1, α2, α3, α4, and α5 represent the weight ratio of each loss task, and the weight value is set according to the experience value during actual model training.

[0090] In some embodiments, Figure 4 As shown,

[0091] Determining the entry line width of the parking space according to the position parameters of the parking space, and determining the parking space type according to the entry line width or the sideline direction angle, includes:

[0092] Calculate the width of the entry line according to the coordinates of the left and right corner points of the entry line;

[0093] Comparing the entry line width with a preset width and a preset length of the parking space respectively to determine whether the current parking space is a vertical parking space or a parallel parking space; or,

[0094] According to the difference between the sideline direction angle and 90 degrees, it is determined whether the current parking space is an oblique parking space, wherein the sideline direction angle is the angle between the entrance line and the left boundary line of the parking space.

[0095] In some embodiments, the four corner points of the parking space are recorded as ((x0, y0, v0), (x1, y1, v1), (x2, y2, v2), (x3, y3, v3)), where (x i ,y i ) represents the corner point coordinates, v i Represents the corner point confidence score. According to the corner point sequence defined above, the coordinates of the left and right endpoints of the parking space entry line are (x0, y0) and (x1, y1), from which the parking space entry line width w can be obtained, and its calculation formula is:

[0096]

[0097] The a priori parking space width w0 and length d0 are preset. According to the specific value of the calculated entry line width w, the difference between it and w0 and d0 is compared respectively, so as to determine whether the current parking space belongs to a vertical parking space or a parallel parking space.

[0098] In addition, let the above-obtained edge direction angle be θ. In the case of vertical parking spaces and parallel parking spaces, the predicted value of θ will be around 90 degrees; in the case of diagonal parking spaces, the predicted value of θ will be far from 90 degrees. Therefore, it is possible to determine whether the current parking space is a diagonal parking space by the edge direction angle.

[0099] Therefore, the type of parking space can be determined based on the width w of the parking space entry line and the angle θ of the edge line. In order to increase the robustness of the judgment, according to the parking space similarity principle, the w and θ of multiple parking spaces near the current parking space are counted here and stored in the sets W and θ respectively, and then outliers and abnormal values ​​are removed, and the average is calculated after filtering. and Then use the above method to determine the type of parking space.

[0100] The detection method also includes:

[0101] If it is determined based on the semantic segmentation information of the edge line of the parking space that there is no line between the left and right corner points of the entry line, and the width of the entry line calculated based on the left and right corner points of the entry line is smaller than the preset width of the parking space, then it is determined that the two parking spaces to which the left and right corner points of the entry line respectively belong are interval parking spaces.

[0102] For example, in scenes such as basements, in addition to conventional connected parking spaces, there are also spaced parking spaces. Due to the isolation of basement support columns, such spaced parking spaces often appear in garages. Figure 5 As shown:

[0103] In order to prevent the middle interval area of ​​the parking space from being mistakenly identified as a parking space, Figure 6 As shown, it is necessary to combine the parking space prior width w0 and the parking space semantic segmentation line for judgment and filtering. First, according to the left and right endpoints (x0, y0) and (x1, y1) of the parking space entry line, combined with the parking space semantic segmentation line predicted by the multi-task network, it can be obtained that there is no line between point (x0, y0) and point (x1, y1). Secondly, the width w of the entry line calculated by points (x0, y0) and (x1, y1) must be significantly smaller than the prior width w0. Combining these two conditions, the false detection of the interval area between the parking spaces can be filtered out.

[0104] In the above embodiment, the parking space corner point information, segmentation information and direction angle information obtained by the multi-task network can be used to obtain the coordinates of the parking space, the width of the entry line and the type of the parking space. In addition, the multi-task network also predicts the occupancy of the parking space, that is, it can be obtained whether each parking space in the surround view is occupied by a vehicle, which can be used to preliminarily determine whether the parking space is an empty parking space.

[0105] In some embodiments, the surround view is obtained by converting the images of the fisheye cameras around the vehicle to a bird's-eye view and stitching them together. In this case, objects with height in the original image will be distorted and stretched in the surround view, which will cause the stretched vehicles in the surround view to block the parking spaces in the distance, resulting in missed detection of empty parking spaces. According to the characteristics of the surround view, that is, the vehicle is always located in the center of the surround view, the surround view can be divided into regions to handle the missed detection of empty parking spaces caused by the parking space occlusion caused by the distortion of the vehicles in the surround view.

[0106] In some embodiments, Figure 3 As shown, based on the distortion stretching characteristics of the surround view stitching image, the surround view stitching image is divided into multiple areas, including:

[0107] Assuming that the vehicle is always located in the center of the surround view mosaic map, the surround view mosaic map is divided into four areas: the first area is located in the left front of the vehicle, the second area is located in the right front of the vehicle, the third area is located in the right rear of the vehicle, and the fourth area is located in the left rear of the vehicle.

[0108] In some embodiments, in each area, according to the semantic segmentation line information of the two corner points of the entry line and the edge line, a truncation line of the parking space edge line is determined, and according to the judgment method of different relationships of the truncation lines of the parking space edge line, an empty parking space in each area is determined, including:

[0109] Determine the width of the entry line of the current parking space w The area where / 2 or above falls is the area to which the current parking space belongs;

[0110] In the first area and the third area, a parking space whose truncation line of the parking space boundary line satisfies the following formula is determined as an empty parking space:

[0111] l0>max(l1,c)

[0112] Wherein, c is the preset length of the parking space, l0 is the cutoff line of the left boundary line of the parking space, and l1 is the cutoff line of the right boundary line of the parking space;

[0113] In the second area and the fourth area, a parking space whose truncation line of the parking space boundary line satisfies the following formula is determined as an empty parking space:

[0114] l1>max(l0,c).

[0115] As described in the above embodiment, the area division of the ring view is as follows: Figure 7 As shown:

[0116] Different logics are used to judge empty parking spaces in the divided areas I, II, III and IV, mainly combining the endpoint coordinates of the parking space entry line and the semantic segmentation line of the edge line. The parking space occlusion caused by vehicle distortion in the four areas is sketched as follows: Figure 8 As shown:

[0117] Define the region to which the parking space belongs. For any parking space, calculate the width w of its entry line based on the two endpoints (x0, y0) and point (x1, y1). When the entry line has a length of more than w / 2 segments falling into a certain region, the parking space is defined as belonging to this region. In other words, determine which region R the parking space belongs to. i (i=I,II,III,IV), then one of the following two conditions must be met:

[0118] ①(x0,y0)∈R i ,and

[0119] ②(x1,y1)∈R i ,and

[0120] For region R i The parking spaces in (i=I,II,III,IV) can be further truncated by combining the two endpoints of the entry line and the edge semantic segmentation line. Taking area I as an example, the truncation line of the parking space edge is as follows: Fig. 9 As shown,

[0121] To determine the empty parking spaces in area I, the following formula needs to be satisfied:

[0122] l0>max(l1,c)

[0123] The constant c is generally taken as 1 / 3 of the prior length of the parking space, which is used to indicate the lower limit of the longest edge threshold blocked by the current parking space. It can be adjusted and set according to experiments. Here, c=d0 / 3, where d0 is the prior length of the parking space.

[0124] Similarly, for area II, according to the definition of the order of the left and right endpoints of the entry line, l1 is the long side and l0 is the short side. The formula that needs to be satisfied to determine the empty parking space is:

[0125] l1>max(l0,c)

[0126] For area III, the formula for determining an empty parking space is the same as that for area I; for area IV, the formula for determining an empty parking space is the same as that for area II.

[0127] Combining the logical judgment of the above area division and the judgment results of the parking space occupancy information output by the previous multi-task network, all the empty parking spaces in the current ring view can be obtained.

[0128] In the above embodiments, accurate and effective detection of empty parking spaces is crucial for constructing parking space maps, which is a critical part of automatic parking. In the above, more parking space information is obtained by constructing a multi-task network, and then the location of the parking space, the width of the entry line, the angle of the edge line, the type of parking space, etc. are obtained based on the logic strategy, and then combined with the area division to obtain a more comprehensive location result of the empty parking space. The detection result is also more robust and accurate, which is suitable for parking space detection in complex scenes.

[0129] In some embodiments, a computer device is provided, such as Fig.10As shown, it includes a memory 201, a processor 202, and a computer program stored in the memory 201 and executable on the processor 202. When the processor 202 executes the computer program, the automatic parking space detection method of any of the above embodiments is implemented.

[0130] Specifically, the computer device may be a computer terminal, a server or a similar computing device.

[0131] In some embodiments, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions for executing the automatic parking space detection method described in any of the above embodiments.

[0132] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0133] Based on the same inventive concept, an automatic parking space detection device is also provided in an embodiment of the present invention, as described in the following embodiments. Since the principle of solving the problem by an automatic parking space detection device is similar to that of an automatic parking space detection method, the implementation of an automatic parking space detection device can refer to the implementation of the automatic parking space detection method, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0134] An automatic parking space detection device, the structural block diagram is as follows Fig.11 As shown, it includes: an image acquisition module, an image processing module, a calculation module and a judgment module.

[0135] An image acquisition module 301 is used to acquire multi-view images of different positions of the vehicle;

[0136] An image processing module 302 is used to stitch the multi-view images by image stitching technology based on the internal and external parameters of the calibrated multiple image acquisition devices to obtain a surround stitching image of a bird's-eye view;

[0137] A calculation module 303 is used to output the surround stitching graph to a multi-task network, predict the location parameters of the parking space, and determine the parking space type according to the location parameter information;

[0138] The judgment module 304 is used to divide the surround view spliced ​​image into multiple areas based on the distortion and stretching characteristics of the surround view spliced ​​image, and determine the empty parking spaces in each area.

[0139] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0140] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting an automatic parking space, characterized in that: The detection method comprises: Acquire multi-view images using multiple image acquisition devices arranged at different positions of the vehicle; Based on the calibrated internal and external parameters of multiple image acquisition devices, the multi-view images are stitched together using image stitching technology to obtain a surround stitching image from a bird's-eye view. Inputting the surround stitching image into a multi-task network to predict the position parameters of the parking space, wherein the position parameters include corner point information, semantic segmentation information of the edge line, edge line direction angle, and occupancy information; Determine the entry line width of the parking space according to the position parameters of the parking space, and determine the parking space type according to the entry line width or the sideline direction angle; Based on the distortion and stretching characteristics of the surround view mosaic map, the surround view mosaic map is divided into multiple regions. In each region, the truncation line of the parking space edge line is determined according to the semantic segmentation line information of the two corner points and edge line of the entry line. According to the different relationships of the truncation lines of the parking space edge line, the empty parking spaces in each region are determined.

2. The automatic parking space detection method according to claim 1, characterized in that: The method of acquiring multi-view images based on a plurality of image acquisition devices arranged at different positions of the vehicle includes installing four fisheye cameras at four positions of the vehicle, front, back, left, and right, to acquire images of the surrounding areas.

3. The automatic parking space detection method according to claim 1, characterized in that: The step of inputting the surround stitching graph into a multi-task network and predicting the location parameters of the parking space comprises: The surround view stitching image is input into a multi-task network. For the input surround view stitching image, prediction information of each branch is obtained through a multi-layer convolutional neural network, the corner point information is output through the key point branch of the multi-task network, the semantic segmentation information of the edge line is output through the segmentation branch of the multi-task network, the edge line direction angle is output through the direction angle branch of the multi-task network, and the occupancy situation information is output through the occupancy classification branch of the multi-task network.

4. The automatic parking space detection method according to claim 1, characterized in that: Determining the entry line width of the parking space according to the position parameters of the parking space, and determining the parking space type according to the entry line width or the sideline direction angle, includes: Calculate the width of the entry line according to the coordinates of the left and right corner points of the entry line; Comparing the entry line width with a preset width and a preset length of the parking space respectively to determine whether the current parking space is a vertical parking space or a parallel parking space; or, According to the difference between the sideline direction angle and 90 degrees, it is determined whether the current parking space is an oblique parking space, wherein the sideline direction angle is the angle between the entrance line and the left boundary line of the parking space.

5. The automatic parking space detection method according to claim 1, characterized in that: Also includes: If it is determined based on the semantic segmentation information of the edge line of the parking space that there is no line between the left and right corner points of the entry line, and the width of the entry line calculated based on the left and right corner points of the entry line is smaller than the preset width of the parking space, then it is determined that the two parking spaces to which the left and right corner points of the entry line respectively belong are interval parking spaces.

6. The automatic parking space detection method according to any one of claims 1 to 5, characterized in that: Based on the distortion stretching characteristics of the surround view mosaic image, the surround view mosaic image is divided into multiple areas, including: Assuming that the vehicle is always located in the center of the surround view mosaic map, the surround view mosaic map is divided into four areas: the first area is located in the left front of the vehicle, the second area is located in the right front of the vehicle, the third area is located in the right rear of the vehicle, and the fourth area is located in the left rear of the vehicle.

7. The automatic parking space detection method according to claim 5, characterized in that: In each area, according to the semantic segmentation line information of the two corner points of the entry line and the edge line, the truncation line of the parking space edge line is determined, and according to the judgment method of different relationships of the truncation lines of the parking space edge line, the empty parking space in each area is determined, including: Determine the width of the entry line of the current parking space w The area where / 2 or above falls is the area to which the current parking space belongs; In the first area and the third area, a parking space whose truncation line of the parking space boundary line satisfies the following formula is determined as an empty parking space: l0>max(l1,) Wherein, c is the preset length of the parking space, l0 is the cutoff line of the left boundary line of the parking space, and l1 is the cutoff line of the right boundary line of the parking space; In the second area and the fourth area, a parking space whose truncation line of the parking space boundary line satisfies the following formula is determined as an empty parking space: l1>max(l0,).

8. An automatic parking space detection device, characterized in that: include: An image acquisition module is used to acquire multi-view images of different positions of the vehicle; An image processing module is used to stitch the multi-view images by image stitching technology based on the internal and external parameters of the calibrated multiple image acquisition devices to obtain a surround stitching image of a bird's-eye view; A calculation module, used for outputting the surround stitching graph to a multi-task network, predicting the position parameters of the parking space, and determining the type of the parking space according to the position parameter information; The judgment module is used to divide the surround view splicing image into multiple areas based on the distortion and stretching characteristics of the surround view splicing image, and determine the empty parking spaces in each area.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the automatic parking space detection method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions for executing the automatic parking space detection method according to any one of claims 1 to 7.

Images