Parking space recognition method, medium, electronic device and vehicle

Abstract

The present disclosure relates to a parking space recognition method, medium, electronic device and vehicle, the method comprising: acquiring an overhead view of the environment around the vehicle; using a pre-trained parking space recognition model to divide the overhead view into a predetermined number of grids, and acquiring a first offset of a first corner point, a second offset of a second corner point and a third offset of a third corner point in the grids; and determining a parking space according to the first offset, the second offset, the third offset and the coordinates of the reference point of the grid where the first corner point is located, and outputting; wherein the first offset is the offset of the first corner point relative to the reference point of the grid, the second offset is the offset of the second corner point relative to the first corner point, and the third offset is the offset of the third corner point relative to the first corner point. In this way, the parking space information around the vehicle can be more accurately determined.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of vehicles and computers, in particular, to a parking space recognition method, medium, electronic device and vehicle. BACKGROUND

[0002] How to recognize the parking space around the vehicle is a hot issue for technical personnel to study. In the related art, the coordinates of the four points of the parking space frame are directly detected, and the parking space frame of the whole vehicle is inferred through post-processing. In the related art, when recognizing the parking space around the vehicle, the four corner points of the parking space need to be clearly visible, and the inference needs to be accurate in the inference process, otherwise it may cause the parking space entrance line to be inferred incorrectly, thereby causing incorrect parking space information to be determined. SUMMARY

[0003] The purpose of the present disclosure is to provide a parking space recognition method, medium, electronic device and vehicle, which can more accurately determine the parking space information around the vehicle.

[0004] To achieve the above purpose, the present disclosure provides a parking space recognition method, the parking space includes four corner points, the method comprises:

[0005] Obtaining an overhead view of the environment around the vehicle;

[0006] Using a pre-trained parking space recognition model to divide the overhead view into a predetermined number of grids, and obtaining a first bias of a first corner point, a second bias of a second corner point and a third bias of a third corner point in the plurality of grids; and

[0007] Determining a parking space according to the first bias, the second bias, the third bias and the coordinates of the reference point of the grid where the first corner point is located, and outputting;

[0008] The first bias is the bias of the first corner point relative to the reference point of the grid, the second bias is the bias of the second corner point relative to the first corner point, and the third bias is the bias of the third corner point relative to the first corner point.

[0009] Optionally, the method further comprises:

[0010] For each grid in the plurality of grids, obtaining a confidence degree of the grid where the first corner point exists;

[0011] The determining a parking space according to the first bias, the second bias, the third bias and the coordinates of the reference point of the grid where the first corner point is located, and outputting, comprises:

[0012] For each of the grids whose confidence is greater than the preset confidence threshold, a parking space is determined according to the first offset of the first corner point in the grid, the second offset of the second corner point corresponding to the first corner point in the grid, the third offset of the third corner point corresponding to the first corner point in the grid, and the coordinates of the reference point of the grid, and the parking space is output.

[0013] Optionally, the determination of the parking space according to the first offset of the first corner point in the grid, the second offset of the second corner point corresponding to the first corner point in the grid, the third offset of the third corner point corresponding to the first corner point in the grid, and the coordinates of the reference point of the grid comprises:

[0014] determining the coordinates of the first corner point in the grid according to the first offset of the first corner point in the grid and the coordinates of the reference point of the grid;

[0015] determining the coordinates of the second corner point corresponding to the first corner point in the grid according to the coordinates of the first corner point in the grid and the second offset of the second corner point corresponding to the first corner point in the grid;

[0016] determining the coordinates of the third corner point corresponding to the first corner point in the grid according to the coordinates of the first corner point in the grid and the third offset of the third corner point corresponding to the first corner point in the grid;

[0017] determining the coordinates of the fourth corner point corresponding to the first corner point in the grid according to the coordinates of the first corner point in the grid, the coordinates of the second corner point corresponding to the first corner point in the grid, and the coordinates of the third corner point corresponding to the first corner point in the grid.

[0018] Optionally, the parking space recognition model comprises a feature extraction module and a decoupling module, the input of the feature extraction module is the bird's eye view, the output of the feature extraction module is a feature map obtained by feature extraction on the bird's eye view, and the input of the decoupling module is the output of the feature extraction module. The decoupling module regresses the first offset of the first corner point in each grid, the second offset of the second corner point corresponding to the first corner point relative to the first corner point, and the third offset of the third corner point corresponding to the first corner point relative to the first corner point in 3 branches according to the feature map.

[0019] The 3 branches are respectively used for regressing the first offset, the second offset, and the third offset.

[0020] Optionally, the determination of the coordinates of the first corner point in the grid according to the first offset of the first corner point in the grid and the coordinates of the reference point of the grid comprises:

[0021] The horizontal coordinate of the first corner point is determined according to the following formula:

[0022] p0_x = (x_val + col) / L

[0023] The vertical coordinate of the first corner point is determined according to the following formula:

[0024] p0_y = (y_val + row) / L

[0025] wherein p0_x is the horizontal coordinate of the first corner point, x_val is the horizontal coordinate offset of the first corner point relative to the reference point of the grid, col is the horizontal coordinate of the reference point of the grid, p0_y is the vertical coordinate of the first corner point, y_val is the vertical coordinate offset of the first corner point relative to the reference point of the grid, row is the vertical coordinate of the reference point of the grid, and L is the grid size.

[0026] Optionally, the second offset and the third offset are normalized offsets, and the determining the coordinates of the second corner point corresponding to the first corner point in the grid according to the coordinates of the first corner point in the grid and the second offset of the second corner point corresponding to the first corner point in the grid comprises:

[0027] The horizontal coordinate of the second corner point is determined according to the following formula:

[0028] p1_x = ps_delta_x1 * 2 - 1 + p0_x

[0029] The vertical coordinate of the second corner point is determined according to the following formula:

[0030] p1_y = ps_delta_y1 * 2 - 1 + p0_y

[0031] wherein p1_x is the horizontal coordinate of the second corner point, ps_delta_x1 is the horizontal coordinate offset in the second offset of the second corner point, p0_x is the horizontal coordinate of the first corner point, p1_y is the vertical coordinate of the second corner point, ps_delta_y1 is the vertical coordinate offset in the second offset of the second corner point, and p0_y is the vertical coordinate of the first corner point.

[0032] The determining the coordinates of the third corner point corresponding to the first corner point in the grid according to the coordinates of the first corner point in the grid and the third offset of the third corner point corresponding to the first corner point in the grid comprises:

[0033] The horizontal coordinate of the third corner point is determined according to the following formula:

[0034] p2_x = ps_delta_x2 * 2 - 1 + p0_x

[0035] The longitudinal coordinate of the third corner point is determined according to the following formula:

[0036] p2_y = ps_delta_y2 * 2 - 1 + p0_y

[0037] wherein p2_x is the longitudinal coordinate of the third corner point, ps_delta_x2 is the longitudinal coordinate offset in the third offset of the third corner point, p2_y is the longitudinal coordinate of the third corner point, and ps_delta_y2 is the longitudinal coordinate offset in the third offset of the third corner point.

[0038] Optionally, the determining the coordinates of the fourth corner point corresponding to the first corner point in the grid according to the coordinates of the first corner point in the grid, the coordinates of the second corner point corresponding to the first corner point in the grid, and the coordinates of the third corner point corresponding to the first corner point in the grid comprises:

[0039] The longitudinal coordinate of the fourth corner point is determined according to the following formula:

[0040] p3_x = p1_x + p2_x - p0_x

[0041] The longitudinal coordinate of the fourth corner point is determined according to the following formula:

[0042] p3_y = p1_y + p2_y - p0_y

[0043] wherein p3_x is the longitudinal coordinate of the fourth corner point, p1_x is the longitudinal coordinate of the second corner point, p2_x is the longitudinal coordinate of the third corner point, p0_x is the longitudinal coordinate of the first corner point, p3_y is the longitudinal coordinate of the fourth corner point, p1_y is the longitudinal coordinate of the second corner point, p2_y is the longitudinal coordinate of the third corner point, and p0_y is the longitudinal coordinate of the first corner point.

[0044] The present disclosure also provides a non-transitory computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the parking space recognition method described above.

[0045] The present disclosure also provides an electronic device comprising:

[0046] a memory having stored thereon a computer program;

[0047] a processor configured to execute the computer program in the memory to implement the steps of the parking space recognition method described above.

[0048] The present disclosure also provides a vehicle comprising a controller, which, when executed, implements the steps of the parking space recognition method described above.

[0049] By the above technical solution, the overhead view of the vehicle surrounding environment is divided into a predetermined number of grids using the pre-trained parking space recognition model, and the first bias amount of the first corner point, the second bias amount of the second corner point, and the third bias amount of the third corner point in each grid are taken as the features output by the parking space recognition model. In this way, in the case that the individual parking space corner points are not clear in the overhead view, the accurate parking space information is determined according to the output of the parking space recognition model. Moreover, since the parking space recognition model outputs the bias amount of the first corner point relative to the reference point of the grid, the bias amount of the second corner point relative to the first corner point, and the bias amount of the third corner point relative to the first corner point, the correlation between the parking space corner point coordinates is greatly utilized, and the probability of inferring an incorrect parking space entrance line is reduced.

[0050] Other features and advantages of the present disclosure will be described in detail in the following detailed description section. BRIEF DESCRIPTION OF DRAWINGS

[0051] The accompanying drawings are included to provide a further understanding of the present disclosure and constitute a part of the specification, and are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation of the present disclosure. In the drawings:

[0052] Figure 1 is a flowchart of a parking space recognition method according to an exemplary embodiment of the present disclosure.

[0053] Figure 2 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0054] The detailed description of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the detailed description described herein is only used to explain and illustrate the present disclosure, and does not limit the present disclosure.

[0055] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in accordance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization of the corresponding device owner. Figure 1 is a flowchart of a parking space recognition method according to an exemplary embodiment of the present disclosure. The parking space recognized by applying the parking space recognition method provided by the present disclosure includes four corner points. As shown in Figure 1 the parking space recognition method includes steps S101 to S103.

[0056] In step S101, an overhead view of a vehicle surrounding environment is acquired.

[0057] In step S102, the overhead view is divided into a predetermined number of grids using a pre-trained parking space recognition model, and a first offset of a first corner point, a second offset of a second corner point, and a third offset of a third corner point in the plurality of grids are obtained.

[0058] In step S103, the parking space is determined according to the first offset, the second offset, the third offset, and the coordinates of the reference point of the grid where the first corner point is located, and is output.

[0059] The first offset is the offset of the first corner point relative to the reference point of the grid where the first corner point is located, the second offset is the offset of the second corner point relative to the first corner point, and the third offset is the offset of the third corner point relative to the first corner point.

[0060] In an embodiment, the overhead view of the vehicle surrounding environment obtained can be a 360-degree panoramic overhead view of the vehicle surrounding environment. The parking space recognition model can be pre-trained. The parking space recognition model can be a model for recognizing the parking space around the vehicle. For example, the parking space recognition model can recognize and output the position information of the first corner point, the position information of the second corner point, and other information for recognizing the parking space around the vehicle in the environment image. After the overhead view of the vehicle surrounding environment obtained is input into the parking space recognition model, the output of the parking space recognition model can be obtained.

[0061] The overhead view can be divided into a predetermined number of grids using the parking space recognition model. In an embodiment, the overhead view of the vehicle surrounding environment is an image with a resolution of 384*384, and the overhead view of the vehicle surrounding environment can be divided into 12*12 grids using the parking space recognition model. In another embodiment, the overhead view of the vehicle surrounding environment can be divided into 16*16 grids using the parking space recognition model.

[0062] The first corner point and the second corner point refer to two outer corner points (i.e., two corner points on a parking space line for entering and exiting a vehicle) of the four corner points of the parking space. In the overhead view of the parking space, the four parking space lines of the parking space form a closed loop. The direction of the first corner point along the parking space line for entering and exiting a vehicle to the second corner point is counterclockwise.

[0063] The third corner point is an inner corner point (i.e., two corner points not on a parking space line for entering and exiting a vehicle) of the four corner points of the parking space. In an embodiment, the third corner point is an inner corner point of a parking space line formed by the line connecting the first corner point.

[0064] The offset is a quantity for characterizing the relative position. In an embodiment, the offset can include a horizontal coordinate offset and a vertical coordinate offset. The horizontal coordinate offset is used to characterize the relative position between the horizontal coordinates of two points, and the vertical coordinate offset is used to characterize the relative position between the vertical coordinates of two points.

[0065] The first offset amount is an offset amount of the first corner point relative to a reference point of the grid in which the first corner point is located. The reference point of the grid can be preset by the technician, for example, for each grid, the center point of the grid can be selected as the reference point of the grid, for another example, for each grid, the vertex of one of the four corners of the grid can also be selected as the reference point of the grid. In an embodiment, when selecting the reference point of the grid, the technician can select the reference point of each grid so that the coordinates of the reference point of each grid are equal to the row and column in which the grid is located among all the grids. For example, the technician can select the reference point of each grid so that the coordinates of the reference point of the grid located at the 3rd row and the 5th column among all the grids are (3, 5), and the coordinates of the reference point of the grid located at the 5th row and the 6th column among all the grids are (5, 6).

[0066] In an embodiment, the first offset amount of the first corner point relative to the reference point of the grid in which the first corner point is located can include a difference between the horizontal coordinate of the first corner point and the horizontal coordinate of the reference point of the grid in which the first corner point is located, and a difference between the vertical coordinate of the first corner point and the vertical coordinate of the reference point of the grid in which the first corner point is located.

[0067] In an embodiment, the offset amount of the second corner point relative to the first corner point (second offset amount) can include a difference between the horizontal coordinate of the second corner point and the horizontal coordinate of the first corner point, and a difference between the vertical coordinate of the second corner point and the vertical coordinate of the first corner point; the offset amount of the third corner point relative to the first corner point (third offset amount) can include a difference between the horizontal coordinate of the third corner point and the horizontal coordinate of the first corner point, and a difference between the vertical coordinate of the third corner point and the vertical coordinate of the first corner point. For each grid, the coordinates of the reference point of the grid are known. For each parking space, the coordinates of all the corner points of the parking space can be determined based on the coordinates of the reference point of the grid in which the first corner point of the parking space is located, the first offset amount of the first corner point of the parking space, the second offset amount of the second corner point of the parking space, and the third offset amount of the third corner point of the parking space, and the parking space information of the parking space is determined. The parking space information can be information indicating the situation of the parking space, such as the position of the parking space and the arrangement form (such as parallel parking space, vertical parking space, and diagonal parking space).

[0068] In an embodiment, after inputting the overhead view of the surrounding environment of the vehicle into the pre-trained parking space recognition model, all the parking spaces in the overhead view of the surrounding environment of the vehicle can be determined and output based on the output of the parking space recognition model (including the first offset amount of the first corner point, the second offset amount of the second corner point, and the third offset amount of the third corner point in each grid).

[0069] By the technical solution, the overhead view of the vehicle surrounding environment is divided into a predetermined number of grids by using the pre-trained parking space recognition model, the first bias amount of the first corner point, the second bias amount of the second corner point, and the third bias amount of the third corner point in each grid are taken as the features output by the parking space recognition model, so that in the case that the individual parking space corner points are not clear in the overhead view, the accurate parking space information is determined according to the output of the parking space recognition model. Moreover, since the parking space recognition model outputs the bias amount of the first corner point relative to the reference point of the grid where the first corner point is located, the bias amount of the second corner point relative to the first corner point, and the bias amount of the third corner point relative to the first corner point, the correlation between the coordinates of the parking space corner points is greatly utilized, and the probability of reasoning out the wrong parking space entrance line is reduced.

[0070] In yet another embodiment, the method further comprises:

[0071] For each of the plurality of grids, obtaining a confidence of the grid in which the first corner point exists;

[0072] determining the parking space according to the first bias amount, the second bias amount, the third bias amount, and the coordinates of the reference point of the grid in which the first corner point is located, and outputting, comprising:

[0073] For each of the grids in which the confidence is greater than the preset confidence threshold, determining the parking space according to the first bias amount of the first corner point in the grid, the second bias amount of the second corner point corresponding to the first corner point in the grid, the third bias amount of the third corner point corresponding to the first corner point in the grid, and the coordinates of the reference point of the grid, and outputting.

[0074] The confidence of the grid in which the first corner point exists is an index for characterizing the possibility of the existence of the first corner point in the grid. For example, the greater the confidence of the grid in which the first corner point exists, the greater the possibility of the existence of the first corner point in the grid. The confidence threshold can be preset by a technician according to experience. When the confidence of the grid is greater than the confidence threshold, it can be considered that the first corner point exists in the grid. When the confidence of the grid is less than the confidence threshold, it can be considered that the first corner point does not exist in the grid.

[0075] In this embodiment, for each of the grids in which the confidence is greater than the confidence threshold, the parking space is determined according to the first bias amount of the first corner point in the grid, the second bias amount of the second corner point corresponding to the first corner point in the grid, the third bias amount of the third corner point corresponding to the first corner point in the grid, and the coordinates of the reference point of the grid, and outputting. In this way, the determination of the parking space information for the grid corresponding to the confidence less than the confidence threshold is avoided, the computer computing power is reduced, and the parking space existing in the vehicle surrounding environment can be determined more quickly.

[0076] In an embodiment, the parking space recognition model can output through 7 output channels. The 7 output channels of the parking space recognition model can be respectively used to output the following information:

[0077] (1) the confidence of the grid in which the first corner point exists;

[0078] (2) the horizontal coordinate offset of the first corner point in the grid relative to the reference point of the grid (the horizontal coordinate offset can be the difference between the horizontal coordinates);

[0079] (3) the vertical coordinate offset of the first corner point in the grid relative to the reference point of the grid (the vertical coordinate offset can be the difference between the vertical coordinates);

[0080] (4) the horizontal coordinate offset of the second corner point corresponding to the first corner point in the grid relative to the first corner point in the grid;

[0081] (5) the vertical coordinate offset of the second corner point corresponding to the first corner point in the grid relative to the first corner point in the grid;

[0082] (6) the horizontal coordinate offset of the third corner point corresponding to the first corner point in the grid relative to the first corner point in the grid;

[0083] (7) the vertical coordinate offset of the third corner point corresponding to the first corner point in the grid relative to the first corner point in the grid.

[0084] In yet another embodiment, determining the parking space according to the first offset of the first corner point in the grid, the second offset of the second corner point corresponding to the first corner point in the grid, the third offset of the third corner point corresponding to the first corner point in the grid, and the coordinates of the reference point of the grid comprises:

[0085] determining the coordinates of the first corner point in the grid according to the first offset of the first corner point in the grid and the coordinates of the reference point of the grid;

[0086] determining the coordinates of the second corner point corresponding to the first corner point in the grid according to the coordinates of the first corner point in the grid and the second offset of the second corner point corresponding to the first corner point in the grid;

[0087] determining the coordinates of the third corner point corresponding to the first corner point in the grid according to the coordinates of the first corner point in the grid and the third offset of the third corner point corresponding to the first corner point in the grid;

[0088] determining the coordinates of the fourth corner point corresponding to the first corner point in the grid according to the coordinates of the first corner point in the grid, the coordinates of the second corner point corresponding to the first corner point in the grid, and the coordinates of the third corner point corresponding to the first corner point in the grid.

[0089] In an embodiment, a first offset of the first corner point relative to a reference point of a grid in which the first corner point is located can be added to coordinates of the reference point of the grid in which the first corner point is located to determine coordinates of the first corner point. After the coordinates of the first corner point are determined, a second offset of a second corner point corresponding to the first corner point in the grid can be added to the coordinates of the first corner point in the grid to determine coordinates of the second corner point; a third offset of a third corner point corresponding to the first corner point in the grid can be added to the coordinates of the first corner point in the grid to determine coordinates of the third corner point.

[0090] Generally, the quadrilateral formed by the four corner points of the parking space is a rectangle or a parallelogram. In a case where the positions of the first corner point, the second corner point, and the third corner point of the parking space are determined, the coordinates of the fourth corner point of the parking space can be inferred, and thus the positions of the four corner points of the parking space can be obtained, and the parking space information of the parking space can be obtained. It should be noted that the fourth corner point is another internal corner point of the four corner points of the parking space except the third corner point.

[0091] In this embodiment, the coordinates of the first corner point are determined first, and then the coordinates of the second corner point and the third corner point are determined according to the coordinates of the first corner point, and then the coordinates of the fourth corner point are determined according to the coordinates of the first corner point, the second corner point, and the third corner point, and thus the parking space information of the parking space is determined. In this way, the determined parking space information is accurate and is not prone to inference errors, and the method has good use effect.

[0092] In yet another embodiment, the parking space recognition model includes a feature extraction module and a decoupling module. The input of the feature extraction module is the top view, and the output of the feature extraction module is a feature map obtained by feature extraction on the top view. The input of the decoupling module is the output of the feature extraction module. The decoupling module regresses a first offset of a first corner point in each grid, a second offset of a second corner point corresponding to the first corner point relative to the first corner point, and a third offset of a third corner point corresponding to the first corner point relative to the first corner point in 3 branches according to the feature map.

[0093] The 3 branches are respectively used to regress the first offset, the second offset, and the third offset.

[0094] The network structure of the feature extraction module can adopt a darknet network structure. In an embodiment, the feature extraction module can include five convolutional blocks, and each convolutional block can include three convolutional layers. The first convolutional layer of the first convolutional block takes the overhead view as input. In the first convolutional layer, the input can be convolved with a convolutional kernel of size 3*3, and the step size can be set to 1 and the padding to 1. The output of the first convolutional layer can be taken as the input of the second convolutional layer, and in the second convolutional layer, a convolutional kernel of size 1*1 can be used, and the step size can be set to 0 and the padding to 1. The output of the second convolutional layer can be taken as the input of the third convolutional layer, and in the third convolutional layer, a convolutional kernel of size 3*3 can be used, and the step size can be set to 1 and the padding to 1. After each convolutional layer in each convolutional block, a normalization module and an activation function can be arranged. The normalization module can be used to normalize the parameters obtained after convolutional operation. In an embodiment, after the overhead view is input into the parking space recognition module, the overhead view can pass through the five convolutional blocks of the feature extraction module to obtain a feature map corresponding to the overhead view (output of the feature extraction module).

[0095] In the parking space recognition model, the output of the feature extraction module can be taken as the input of the decoupling module. The decoupling module can regress the first bias of the first corner point in each grid, the second bias of the second corner point corresponding to the first corner point relative to the first corner point, and the third bias of the third corner point corresponding to the first corner point relative to the first corner point in the three branches according to the feature map output by the feature extraction module.

[0096] In an embodiment, in the first branch, the feature map is convolved with a convolutional kernel of size 3*3 to obtain the first bias of the first corner point in each grid (output of the first branch). In the convolutional operation, the step size can be set to 1 and the padding to 0.

[0097] In the second branch, the feature map is convolved with a convolutional kernel of size 3*3 to obtain the second bias of the second corner point corresponding to the first corner point relative to the first corner point (output of the second branch). In the convolutional operation, the step size can be set to 1 and the padding to 0.

[0098] In the third branch, the feature map is convolved with a convolutional kernel of size 3*3 to obtain the third bias of the third corner point corresponding to the first corner point relative to the first corner point (output of the third branch). In the convolutional operation, the step size can be set to 1 and the padding to 0. In an embodiment, a sigmoid function is used as the activation function in the three branches of the decoupling module.

[0099] In an embodiment, the decoupling module is configured to regress the first offset of the first corner point in each grid in a branch, and is also configured to regress the confidence of the grid in which the first corner point exists. For example, in the first branch, the first offset of the first corner point in each grid and the confidence of the grid in which the first corner point exists can be regressed.

[0100] After obtaining the outputs of each branch, the outputs of each branch can be spliced to obtain the output of the parking space recognition model. For example, the outputs of each branch are spliced, and the output of the parking space recognition model is output through 7 output channels. The information output by the 7 output channels of the parking space recognition model has been described in detail above, and will not be described again here.

[0101] In yet another embodiment, the coordinates of the first corner point in the grid are determined according to the first offset of the first corner point in the grid and the coordinates of the reference point of the grid, comprising:

[0102] The horizontal coordinate of the first corner point is determined according to the following formula:

[0103] p0_x = (x_val + col) / L (1)

[0104] The vertical coordinate of the first corner point is determined according to the following formula:

[0105] p0_y = (y_val + row) / L (2)

[0106] Where p0_x is the horizontal coordinate of the first corner point, x_val is the horizontal coordinate offset of the first corner point relative to the reference point of the grid, col is the horizontal coordinate of the reference point of the grid, p0_y is the vertical coordinate of the first corner point, y_val is the vertical coordinate offset of the first corner point relative to the reference point of the grid, row is the vertical coordinate of the reference point of the grid, and L is the grid size. As shown in formula (1) and formula (2), when determining the horizontal coordinate p0_x of the first corner point, the sum of x_val and col is divided by the grid size L, and the calculated horizontal coordinate p0_x of the first corner point is the normalized horizontal coordinate of the first corner point; when determining the vertical coordinate p0_y of the first corner point, the sum of y_val and row is divided by the grid size L, and the calculated vertical coordinate of the first corner point is the normalized vertical coordinate of the first corner point. That is, the values of x_val, col, y_val, row, x_val+col, and y_val+row are less than the value of the grid size. For example, when the value of the grid size is 16, x_val, col, y_val, row, x_val+col, and y_val+row are all less than 16.

[0107] The grid size is the number of rows or columns of the divided grid after the top view is divided into a predetermined number of grids. It should be noted that the above formula (1) and formula (2) can be applied to the case where the number of rows and columns of the divided grid is equal after the top view is divided into a predetermined number of grids. For example, when the top view is divided into 16*16 grids, the value of the grid size L is 16. For another example, when the top view is divided into 12*12 grids, the value of the grid size L is 12.

[0108] In this embodiment, the coordinates of the first corner point can be quickly determined according to formula (1) and formula (2), the method is simple, and the implementation effect is good.

[0109] In an implementation, when training the parking space recognition model, the label marked by the training data can be calculated according to the following formula (3)-formula (8).

[0110] The horizontal coordinate offset of the first corner point relative to the reference point of the grid can be calculated according to the following formula:

[0111] x_val=x*map_size-col (3)

[0112] Wherein, x is the horizontal coordinate of the first corner point in each grid in the training data, and map_size is the size of the feature map. It should be noted that in an implementation, the size of the feature map is equal to the numerical value of the grid size.

[0113] The vertical coordinate offset of the first corner point relative to the reference point of the grid can be calculated according to the following formula:

[0114] y_val=y*map_size-row (4)

[0115] Wherein, y is the vertical coordinate of the first corner point in each grid in the training data.

[0116] The horizontal coordinate offset of the second corner point relative to the first corner point can be calculated according to the following formula:

[0117] ps_delta_x1=(x1-x+1) / 2 (5)

[0118] Wherein, x1 is the horizontal coordinate of the second corner point corresponding to the first corner point in the training data, and ps_delta_x1 is the horizontal coordinate offset in the second offset of the second corner point (i.e., the horizontal coordinate offset of the second corner point relative to the first corner point).

[0119] The vertical coordinate offset of the second corner point relative to the first corner point can be calculated according to the following formula:

[0120] ps_delta_y1=(y1-y+1) / 2 (6)

[0121] wherein y1 is a vertical coordinate of the second corner corresponding to the first corner in the training data, and ps_delta_y1 is a vertical coordinate offset (i.e., a vertical coordinate offset of the second corner relative to the first corner) in the second offset of the second corner.

[0122] A horizontal coordinate offset of the third corner relative to the first corner can be calculated according to the following formula:

[0123] ps_delta_x2 = (x2 - x + 1) / 2 (7)

[0124] wherein x2 is a horizontal coordinate of the third corner corresponding to the first corner in the training data, and ps_delta_x2 is a horizontal coordinate offset (i.e., a horizontal coordinate offset of the third corner relative to the first corner) in the third offset of the third corner.

[0125] A vertical coordinate offset of the third corner relative to the first corner can be calculated according to the following formula:

[0126] ps_delta_y2 = (y2 - y + 1) / 2 (8)

[0127] wherein y2 is a vertical coordinate of the third corner corresponding to the first corner in the training data, and ps_delta_y2 is a vertical coordinate offset (i.e., a vertical coordinate offset of the third corner relative to the first corner) in the third offset of the third corner.

[0128] It should be noted that the parameters x, y, x1, y1, x2, y2 in the formula (3) - formula (8) are normalized parameters, that is, the values of x, y, x1, y1, x2, y2 are between 0 and 1. For example, in an embodiment, when the normalized horizontal coordinate is 0, it corresponds to the leftmost side of the top view, and when the normalized horizontal coordinate is 1, it corresponds to the rightmost side of the top view; when the normalized vertical coordinate is 0, it corresponds to the lowermost side of the top view, and when the normalized vertical coordinate is 1, it corresponds to the uppermost side of the top view.

[0129] In yet another embodiment, the second offset and the third offset are normalized offsets, and determining a coordinate of the second corner corresponding to the first corner in the grid according to the coordinate of the first corner in the grid and the second offset of the second corner corresponding to the first corner in the grid comprises:

[0130] determining the horizontal coordinate of the second corner according to the following formula:

[0131] p1_x = ps_delta_x1 * 2 - 1 + p0_x (9)

[0132] determining the vertical coordinate of the second corner according to the following formula:

[0133] p1_y = ps_delta_y1 * 2 - 1 + p0_y (10)

[0134] wherein p1_x is the horizontal coordinate of the second corner point, p0_x is the horizontal coordinate of the first corner point, p1_y is the vertical coordinate of the second corner point, and p0_y is the vertical coordinate of the first corner point.

[0135] According to the coordinate of the first corner point in the grid and the third offset of the third corner point corresponding to the first corner point in the grid, the coordinate of the third corner point corresponding to the first corner point in the grid is determined, comprising:

[0136] The horizontal coordinate of the third corner point is determined according to the following formula:

[0137] p2_x = ps_delta_x2 * 2 - 1 + p0_x (11)

[0138] The vertical coordinate of the third corner point is determined according to the following formula:

[0139] p2_y = ps_delta_y2 * 2 - 1 + p0_y (12)

[0140] wherein p2_x is the horizontal coordinate of the third corner point, and p2_y is the vertical coordinate of the third corner point.

[0141] In this embodiment, after obtaining the output of the parking space recognition model and the coordinate of the first corner point, the coordinates of the second corner point and the third corner point can be calculated according to the formulas (9)-(12), which is simple and has good implementation effect.

[0142] In yet another embodiment, according to the coordinate of the first corner point in the grid, the coordinate of the second corner point corresponding to the first corner point in the grid, and the coordinate of the third corner point corresponding to the first corner point in the grid, the coordinate of the fourth corner point corresponding to the first corner point in the grid is determined, comprising:

[0143] The horizontal coordinate of the fourth corner point is determined according to the following formula:

[0144] p3_x = p1_x + p2_x - p0_x (13)

[0145] The vertical coordinate of the fourth corner point is determined according to the following formula:

[0146] p3_y = p1_y + p2_y - p0_y (14)

[0147] wherein p3_x is the horizontal coordinate of the fourth corner point, p3_y is the vertical coordinate of the fourth corner point, and p1_y is the vertical coordinate of the second corner point.

[0148] In this embodiment, the coordinates of the fourth corner point can be determined through the coordinates of the first corner point, the second corner point and the third corner point, so that the reasoning error is not prone to occur, and the coordinates of the four corner points of the parking space can be accurately determined, and the method has good implementation effect.

[0149] The present disclosure also provides a non-transitory computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the parking space recognition method.

[0150] The present disclosure also provides an electronic device, comprising:

[0151] a memory having stored thereon a computer program;

[0152] a processor configured to execute the computer program in the memory to implement the steps of the parking space recognition method.

[0153] The present disclosure also provides a vehicle comprising a controller, which, when executed, implements the steps of the parking space recognition method.

[0154] Figure 2 is a block diagram of an electronic device 700 according to an exemplary embodiment. As shown in Figure 2 the electronic device 700 can include a processor 701 and a memory 702. The electronic device 700 can also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0155] The processor 701 is configured to control overall operations of the electronic device 700 to complete all or part of the steps of the parking space recognition method described above. The memory 702 is configured to store various types of data to support operations of the electronic device 700, which can include, for example, instructions for operating any application or method on the electronic device 700, and application-related data, such as contact data, transmitted and received messages, pictures, audio, video, and the like. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk, or an optical disk. The multimedia component 703 can include a screen and an audio component. The screen can be, for example, a touch screen, and the audio component is configured to output and / or input audio signals. For example, the audio component can include a microphone configured to receive external audio signals. The received audio signals can be further stored in the memory 702 or transmitted through the communication component 705. The audio component further includes at least one speaker configured to output audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, which can be a keyboard, a mouse, a button, and the like. The buttons can be virtual buttons or physical buttons. The communication component 705 is configured to perform wired or wireless communication between the electronic device 700 and other devices. The wireless communication, such as Wi-Fi, Bluetooth, near field communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, and the like, or a combination of one or more of them, is not limited herein. Therefore, the communication component 705 can include, for example, a Wi-Fi module, a Bluetooth module, an NFC module, and the like.

[0156] In an exemplary embodiment, the electronic device 700 can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements for performing the parking space recognition method described above.

[0157] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the parking space recognition method described above. For example, the computer-readable storage medium can be the memory 702 described above including program instructions, which can be executed by the processor 701 of the electronic device 700 to complete the parking space recognition method described above.

[0158] In another exemplary embodiment, a computer program product is also provided, which contains a computer program capable of being executed by a programmable device, and the computer program has code portions for executing the parking space recognition method described above when executed by the programmable device.

[0159] The preferred embodiments of the present disclosure are described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to the specific details in the above-described embodiments. Within the technical concept scope of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.

[0160] In addition, it should be noted that each specific technical feature described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, various possible combinations are not described again in the present disclosure.

[0161] Furthermore, any combination of the various different embodiments of the present disclosure can also be made, as long as it does not deviate from the idea of the present disclosure, and it should also be considered as the disclosed content of the present disclosure.

Claims

1. A parking space recognition method, the parking space including four corner points, characterized by, The method comprises: obtaining an overhead view of a surrounding environment of a vehicle; dividing the overhead view into a predetermined number of grids using a pre-trained parking space recognition model, and obtaining a first offset of a first corner point, a second offset of a second corner point and a third offset of a third corner point in the grids; for each grid, obtaining a confidence of the grid in which the first corner point exists; for each grid whose confidence is greater than a preset confidence threshold, determining a parking space according to the first offset of the first corner point in the grid, the second offset of the second corner point corresponding to the first corner point in the grid, the third offset of the third corner point corresponding to the first corner point in the grid and coordinates of a reference point of the grid, and outputting the parking space; and determining a parking space according to the first offset of the first corner point in the grid, the second offset of the second corner point corresponding to the first corner point in the grid, the third offset of the third corner point corresponding to the first corner point in the grid and coordinates of a reference point of the grid, comprises: determining coordinates of the first corner point in the grid according to the first offset of the first corner point in the grid and the coordinates of the reference point of the grid; determining coordinates of the second corner point corresponding to the first corner point in the grid according to the coordinates of the first corner point in the grid and the second offset of the second corner point corresponding to the first corner point in the grid; determining coordinates of the third corner point corresponding to the first corner point in the grid according to the coordinates of the first corner point in the grid and the third offset of the third corner point corresponding to the first corner point in the grid; determining coordinates of a fourth corner point corresponding to the first corner point in the grid according to the coordinates of the first corner point in the grid, the coordinates of the second corner point corresponding to the first corner point in the grid and the coordinates of the third corner point corresponding to the first corner point in the grid; wherein the first offset is an offset of the first corner point relative to a reference point of the grid, the second offset is an offset of the second corner point relative to the first corner point, and the third offset is an offset of the third corner point relative to the first corner point. The parking space recognition model comprises a feature extraction module and a decoupling module, an input of the feature extraction module is the overhead view, an output of the feature extraction module is a feature map obtained by feature extraction on the overhead view, an input of the decoupling module is the output of the feature extraction module, and the decoupling module regresses a first offset of a first corner point in each grid, a second offset of a second corner point corresponding to the first corner point relative to the first corner point and a third offset of a third corner point corresponding to the first corner point relative to the first corner point in three branches according to the feature map.

2. The method of claim 1, wherein, The three branches are respectively used to regress the first offset, the second offset and the third offset. The determination of the coordinates of the first corner point in the grid according to the first offset of the first corner point in the grid and the coordinates of the reference point of the grid comprises:

3. The method of claim 1, wherein, determining the abscissa of the first corner point according to the following formula: ​ A longitudinal coordinate of the first corner point is determined according to the following formula: wherein, is a horizontal coordinate of the first corner point, is a horizontal coordinate offset of the first corner point relative to a reference point of the grid, is a horizontal coordinate of the reference point of the grid, is a vertical coordinate of the first corner point, is a vertical coordinate offset of the first corner point relative to a reference point of the grid, is a vertical coordinate of the reference point of the grid, and L is a grid size.

4. The method of claim 1, wherein, The second offset and the third offset are normalized offsets, and the determination of the coordinate of the second corner point corresponding to the first corner point in the grid according to the coordinate of the first corner point in the grid and the second offset of the second corner point corresponding to the first corner point in the grid comprises: A horizontal coordinate of the second corner point is determined according to the following formula: A longitudinal coordinate of the second corner point is determined according to the following formula: wherein, is a horizontal coordinate of the second corner point, is a horizontal coordinate offset in the second offset amount for the second corner point, is a horizontal coordinate of the first corner point, is a vertical coordinate of the second corner point, is a vertical coordinate offset in the second offset amount for the second corner point, is a vertical coordinate of the first corner point; The determination of the coordinate of the third corner point corresponding to the first corner point in the grid according to the coordinate of the first corner point in the grid and the third offset of the third corner point corresponding to the first corner point in the grid comprises: A horizontal coordinate of the third corner point is determined according to the following formula: A longitudinal coordinate of the third corner point is determined according to the following formula: wherein, is a horizontal coordinate of the third corner point, is a horizontal coordinate offset in the third offset amount for the third corner point, is a vertical coordinate of the third corner point, is a vertical coordinate offset in the third offset amount for the third corner point.

5. The method of claim 1, wherein, The determination of the coordinate of the fourth corner point corresponding to the first corner point in the grid according to the coordinate of the first corner point in the grid, the coordinate of the second corner point corresponding to the first corner point in the grid and the coordinate of the third corner point corresponding to the first corner point in the grid comprises: A horizontal coordinate of the fourth corner point is determined according to the following formula: A longitudinal coordinate of the fourth corner point is determined according to the following formula: wherein is a horizontal coordinate of the fourth corner point, is a horizontal coordinate of the second corner point, is a horizontal coordinate of the third corner point, is a horizontal coordinate of the first corner point, is a vertical coordinate of the fourth corner point, is a vertical coordinate of the second corner point, is a vertical coordinate of the third corner point, is a vertical coordinate of the first corner point.

6. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the steps of the method of any one of claims 1-5.

7. An electronic device, comprising: comprises: a memory having a computer program stored thereon; a processor configured to execute the computer program in the memory to implement the steps of the method of any one of claims 1-5.

8. A vehicle characterized by comprising: comprises a controller configured to implement the steps of the method of any one of claims 1-5 when executed.

Images