A method and system for identifying an empty parking space, a computer device, and a storage medium

By using semantic segmentation model and regular processing technology in the automatic parking system, the problem of poor robustness and accuracy of empty parking space recognition is solved, and the accurate identification and stability of multiple parking space types is achieved. Output.

CN114120259BActive Publication Date: 2024-11-08GUANGZHOU AUTOMOBILE GROUP CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202010900989.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-09-01
Publication Date
2024-11-08
Estimated Expiration
2040-09-01

Smart Images

  • Figure CN114120259B_ABST
    Figure CN114120259B_ABST
Patent Text Reader

Abstract

The present invention relates to an empty parking space identification method and its system, computer equipment, and storage medium, the method comprising: in response to receiving a start identification signal, obtaining a panoramic image of the surrounding environment of the current vehicle; using a pre-trained semantic segmentation model to perform semantic segmentation and identification on the panoramic image to obtain a drivable area; regularizing the boundary of the drivable area to obtain boundary information, the boundary information including multiple boundary points; screening the multiple boundary points according to a preset screening rule to obtain multiple empty parking space corner point pre-selected points; randomly selecting any 4 points as a group of pre-selected points to enumerate the multiple empty parking space corner point pre-selected points to obtain multiple groups of pre-selected points, respectively judging whether the multiple parking spaces formed by the multiple groups of pre-selected points meet the preset parking space conditions, and generating corresponding candidate empty parking spaces according to one or more groups of pre-selected points that meet the parking space conditions, and determining the target empty parking space that is finally output according to the candidate empty parking spaces. The present invention can solve the technical problem that the current recognition of empty parking spaces has poor robustness and accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of automatic parking, and in particular to an empty parking space identification method and a system, computer equipment, and storage medium thereof. Background Art

[0002] Empty parking space recognition is the basis of automatic parking technology. The parking space detection methods used in most automatic parking systems can be divided into three categories: camera-based parking space detection methods, ultrasonic radar-based parking space detection methods, and camera and ultrasonic radar fusion-based parking space detection methods. Camera-based parking space detection methods can usually only detect linear parking spaces, that is, the parking spaces must be regular lined parking spaces; ultrasonic radar-based empty parking space detection methods can sense non-linear parking spaces (spatial parking spaces), but this type of method is difficult to obtain accurate information such as parking space size and posture, and has poor robustness and accuracy in application; the method based on camera and ultrasonic radar fusion combines the advantages of the first two methods and can detect both linear parking spaces and spatial parking spaces at the same time, but the recognition of spatial parking spaces still has the disadvantages of poor robustness and accuracy. Summary of the invention

[0003] The purpose of the present invention is to propose a method and system for identifying empty parking spaces for automatic parking, a computer device, and a computer-readable storage medium, so as to solve the technical problem that the current identification of empty parking spaces has poor robustness and accuracy.

[0004] To achieve the above objective, according to a first aspect, an embodiment of the present invention provides a method for identifying an empty parking space, comprising:

[0005] Step S1, in response to receiving a start identification signal, obtaining a panoramic image of the current vehicle surrounding environment;

[0006] Step S2, using a pre-trained semantic segmentation model to perform semantic segmentation and recognition on the surround panoramic image to obtain a drivable area;

[0007] Step S3, regularizing the boundary of the drivable area to obtain boundary information, wherein the boundary information includes a plurality of boundary points;

[0008] Step S4, screening the plurality of boundary points according to a preset screening rule to obtain a plurality of pre-selected vacant parking space corner points;

[0009] Step S5, randomly selecting any 4 points as a group of pre-selected points to enumerate the multiple empty parking space corner pre-selected points to obtain multiple groups of pre-selected points, and respectively judging whether the multiple parking spaces formed by the multiple groups of pre-selected points meet the preset parking space conditions, and generating corresponding candidate empty parking spaces according to one or more groups of pre-selected points that meet the parking space conditions, and determining the target empty parking space that is finally output according to the candidate empty parking spaces.

[0010] Optionally, extracting boundary information of the drivable area includes:

[0011] Extracting the boundary of the drivable area to form a closed curve;

[0012] Expand the closed curve by one pixel point inside the curve to form a boundary search area;

[0013] Selecting any one adjacent pixel point on the boundary search area as an initial search point for the boundary search;

[0014] Performing a boundary search in the boundary search area according to the initial search point; during the search process, calculating the cost distance of other pixel points adjacent to the current search point according to a preset cost function, and selecting the pixel point with the smallest cost distance as the next search point to continue searching until the initial search point is found;

[0015] Regularization is performed on multiple search points found through boundary search to obtain the boundary information.

[0016] Optionally, the calculating the cost distance of searching for other pixel points adjacent to the current search point according to a preset cost function includes:

[0017] d=d 1 ×ɑ+d 2 ×(1-ɑ)

[0018] d 1 =dis min

[0019] d 2 = abs(theta i-2,i-1 -theta i-1,i )

[0020] Among them, d is the cost distance; d 1 is the shortest distance between the pixel and the non-empty parking space; d 2 Theta is the angle between the direction of the previous search path and the direction of the current search path; i-1,i Represents a vector (x i -x i-1 ,y i -y i-1 ) and the x-axis in the vehicle coordinate system in radians; ɑ is the preset scaling factor for d 1 With d 2 The proportion in the cost calculation.

[0021] Optionally, the step of performing regularization processing on a plurality of search points obtained through boundary search to obtain boundary information of the drivable area includes:

[0022] A straight line is fitted between three or more adjacent search points; if the fitting error is less than or equal to the preset error threshold, the two end points of the fitted straight line are used as boundary points; if the fitting error is greater than the preset error threshold, three or more adjacent search points are used as boundary points; finally, multiple boundary points of the regularized empty parking spaces are obtained.

[0023] Optionally, the step S4 includes:

[0024] Step S41, calculate the path direction and path turning angle of each boundary point; wherein the path direction a of the i-th boundary point is i The angle between the directed line segment formed between the ith boundary point and the i+1th boundary point and the positive direction of the x-axis in the vehicle coordinate system rotated clockwise; the path angle theta of the ith boundary point i is the path direction a of the i-th boundary point i The path direction a with the i-1th boundary point i-1 The absolute value of the difference;

[0025] Step S42: transform the path angle theta in the plurality of boundary points i The boundary points that meet the preset value range are output as the pre-selected corner points of the empty parking space.

[0026] Optionally, the step S5 includes:

[0027] Step S51, randomly selecting any 4 points as a group of pre-selected points to enumerate the plurality of pre-selected points of the corner points of the empty parking spaces to obtain a plurality of groups of pre-selected points;

[0028] Step S52: respectively determining whether the multiple parking spaces formed by the multiple groups of pre-selected points meet the preset parking space conditions, and generating corresponding candidate empty parking spaces according to one or more groups of pre-selected points that meet the parking space conditions, and calculating the credibility of the candidate empty parking spaces;

[0029] Among them, for any set of pre-selected points (P k,1 , P k,2 , P k,3 , P k,4 ), calculate point P k,1 Solstice P k,4 The cumulative path length l k , the Euclidean distance between two adjacent points l k-12 , l k-23 , l k-34 , l k-41 , determine the calculated l k , l k-12 , l k-23 , l k-34 , l k-41Whether the preset parking space conditions are met, if the group of pre-selected points (P k,1 , P k,2 , P k,3 , P k,4 ) meets the parking space condition, then the group of pre-selected points (P k,1 , P k,2 , P k,3 , P k,4 ) is marked as a pre-selected empty parking space corner point sequence, and a candidate empty parking space is obtained by performing parking space prediction based on a closed quadrilateral formed by the four points in each sequence;

[0030] Step S53: determining the target empty parking space to be outputted finally according to the credibility of the candidate empty parking spaces.

[0031] Optionally, the parking space conditions are specifically:

[0032] (car w ×2+car h )×k cmin <l k <(car w +car h ×2)×k cmax

[0033] k min <l k-12 / l k-34 <k max

[0034] k min <l k-23 / l k-41 <k max

[0035] Among them, car w and car h are the width and height of the current vehicle, respectively, k cmin , k cmax , k min and k max is the preset value.

[0036] Optionally, where:

[0037] For any pre-selected empty parking space corner point sequence (P k,1 , P k,2 , P k,3 , P k,4 ), P k,1 P k,2 P k,3 P k,4 is the original closed quadrilateral; let P k,1 'P k,2 'Pk,3 'P k,4 ' is the predicted empty parking space, C 1 , C 2 The line segments P k,1 P k,2 , P k,3 P k,4 The midpoint of O 2 O 4 C 1 C 2 The perpendicular bisector of 2 O 4 With O 1 O 3 perpendicular to each other and bisecting each other; then a parking space prediction is performed according to a preset constraint condition to obtain a plurality of candidate empty parking spaces; the constraint condition is that the length and width of the candidate empty parking space are respectively m times the length and width of the current vehicle;

[0038] The credibility score of the candidate empty parking space k The calculation is as follows:

[0039] score k =w 1 ×S 1 / S+w 2 ×S 2 / S+w 3 ×S 3 / S

[0040] Where S is P k,1 'P k,2 'P k,3 'P k,4 ' area, S 1 P k,1 'P k,2 'P k,3 'P k,4 'With P k,1 P k,2 P k,3 P k,4 The area of ​​the intersection, S 2 P k,1 'P k,2 'P k,3 'P k,4 'With P k,1 P k,2 P k,3 P k,4 The intersection-over-union ratio, S 3 P k,1 'P k,2 'P k,3 'P k,4 ' is the area of ​​the drivable area, w 1 、w 2、w 3 is the preset weight value, and m is the preset value.

[0041] Optionally, the step S53 includes:

[0042] Determining whether the credibility of the plurality of candidate empty parking spaces is greater than the credibility threshold;

[0043] If there is only one candidate empty parking space whose credibility is greater than the credibility threshold, then the candidate empty parking space is determined as the empty parking space to be output;

[0044] If the credibility of at least two candidate empty parking spaces is greater than the credibility threshold, the candidate empty parking space with the highest credibility is selected as the empty parking space to be output;

[0045] If the credibility of the plurality of candidate empty parking spaces is less than the credibility threshold, a result indicating that no empty parking space is detected is output.

[0046] Optionally, it also includes:

[0047] Obtain the current time information and vehicle position information, and determine whether to perform image semantic segmentation based on the current time information and vehicle position information; wherein, if the time interval between the current time and the time of the last image semantic segmentation is greater than a preset time threshold, or if the distance between the current vehicle position and the vehicle position at the last time image semantic segmentation was performed is greater than a preset distance threshold, then determine to perform image semantic segmentation and generate the start recognition signal.

[0048] According to a second aspect, an embodiment of the present invention provides an empty parking space recognition system, which is used to execute the empty parking space recognition method described in the first aspect, and the system includes:

[0049] An image acquisition unit, configured to acquire a panoramic image of the current vehicle surroundings in response to receiving a start identification signal;

[0050] A semantic segmentation unit, used to perform semantic segmentation and recognition on the surround panoramic image using a pre-trained semantic segmentation model to obtain a drivable area;

[0051] A boundary extraction unit, used for performing regularization processing on the boundary of the drivable area to obtain boundary information, wherein the boundary information includes a plurality of boundary points;

[0052] A pre-selected point screening unit, configured to screen the plurality of boundary points according to a preset screening rule to obtain a plurality of pre-selected vacant parking space corner points; and

[0053] The empty parking space prediction unit is used to enumerate the multiple empty parking space corner pre-selected points by randomly selecting any 4 points as a group of pre-selected points to obtain multiple groups of pre-selected points, and respectively determine whether the multiple parking spaces formed by the multiple groups of pre-selected points meet the preset parking space conditions; if any group of pre-selected points meets the parking space conditions, the corresponding empty parking space is generated according to the group of pre-selected points.

[0054] Optionally, the system further comprises:

[0055] A trigger unit is used to obtain current time information and vehicle position information, and determine whether to perform image semantic segmentation based on the current time information and vehicle position information; wherein, if the time interval between the current time and the time of the last image semantic segmentation is greater than a preset time threshold, or if the distance between the current vehicle position and the vehicle position at the last image semantic segmentation is greater than a preset distance threshold, it is determined to perform image semantic segmentation, generate a start recognition signal, and send the start recognition signal to the image acquisition unit.

[0056] According to a third aspect, an embodiment of the present invention proposes a computer device, comprising: an empty parking space identification system according to the first aspect; or, a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the empty parking space identification method according to the second aspect.

[0057] According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the vacant parking space identification method according to the first aspect.

[0058] An embodiment of the present invention proposes an empty parking space recognition method and system, computer device, and computer-readable storage medium for automatic parking. When the embodiment is implemented, the automatic parking system of the vehicle is started to obtain a panoramic image of the surrounding environment of the current vehicle; a pre-trained semantic segmentation model is used to perform semantic segmentation and recognition on the panoramic image to obtain a drivable area; the boundary of the drivable area is regularized to obtain boundary information, and the boundary information includes multiple boundary points; the multiple boundary points are screened according to preset screening rules to obtain multiple empty parking space corner point pre-selected points; any 4 points are randomly selected as a group of pre-selected points to enumerate the multiple empty parking space corner point pre-selected points to obtain multiple groups of pre-selected points, and it is judged whether the multiple parking spaces formed by the multiple groups of pre-selected points meet the preset parking space conditions. If any group of pre-selected points meets the parking space conditions, the corresponding empty parking space is generated according to the group of pre-selected points. This embodiment predicts empty parking spaces based on multiple boundary points of the drivable area, and can effectively suppress the influence of lighting changes and weather changes on the accuracy of empty parking space positioning results in various parking scenarios. It is not restricted by the type of parking spaces and can simultaneously identify line parking spaces and space parking spaces (parking spaces without parking lines).

[0059] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be embodied by implementing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 The figure is a flow chart of a method for identifying an empty parking space in one embodiment of the present invention.

[0062] Figure 2 Schematic diagram of a drivable area in one embodiment of the present invention.

[0063] Figure 3 FIG. 1 is a schematic diagram of a closed curve in an embodiment of the present invention.

[0064] Figure 4 Schematic diagram of a boundary search area in an embodiment of the present invention.

[0065] Figure 5 It is a schematic diagram of boundary search results in one embodiment of the present invention.

[0066] Figure 6 Schematic diagram of candidate empty parking space prediction in one embodiment of the present invention.

[0067] Figure 7 It is a schematic diagram of the framework of an empty parking space recognition system in another embodiment of the present invention. DETAILED DESCRIPTION

[0068] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In addition, in order to better illustrate the present invention, numerous specific details are given in the specific embodiments below. It should be understood by those skilled in the art that the present invention can also be implemented without certain specific details. In some embodiments, means well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present invention.

[0069] See also Figure 1 An embodiment of the present invention provides a method for identifying an empty parking space, comprising the following steps S1 to S5:

[0070] Step S1, in response to receiving a start identification signal, obtaining a panoramic image of the current vehicle surrounding environment;

[0071] Specifically, the method of this embodiment is applied to an automatic parking system to realize the recognition of a drivable area. When the automatic parking is started, the automatic parking system is initialized and an image semantic segmentation is performed to obtain a drivable area.

[0072] Exemplarily, the surround panoramic image can be obtained in the following manner: first, four fisheye images are collected and distortion correction is performed on the four fisheye videos; then, the four outermost inner corner points of the chessboard are automatically selected as the perspective transformation source points, and the four corresponding target points of the target image are selected, thereby obtaining four pairs of corresponding points, and then the four fisheye images are perspectively transformed respectively by the calculated perspective transformation matrix to obtain four bird's-eye views; finally, the four fisheye bird's-eye views are stitched. Optionally, the surround panoramic image is a top-down image covering a rectangular area with a length of 18.6 meters and a width of 15.4 meters, with the center of the rear axle of the vehicle as the midpoint.

[0073] It should be noted that the above method is only an example, and the method of this embodiment is not limited to a certain means of obtaining the panoramic image.

[0074] Among them, there are obvious seams at the joints of two adjacent image areas, and some positions are inaccurate, which is considered to be a problem with the camera installation; the four-way video has a large difference in brightness, and there is uneven brightness. In order to deal with the seam problem, you can optionally add a gradual weight value to the overlapping area of ​​the two stitched images, and perform weighted fusion on the overlapping area to achieve a smooth transition effect.

[0075] The initialization of the parking system in this embodiment involves the determination of the world coordinate system and the initialization of the vehicle's initial posture. Once the world coordinate system is determined, all coordinate transformations are uniformly converted to the world coordinate system for calculation during the entire automatic parking process until the parking is terminated. The determination of the world coordinate system helps to simplify the complexity of the algorithm in the subsequent drivable area map update phase. The initialization information of the vehicle's posture and the determination of the world coordinate system are carried out synchronously. The directions of the x-axis and y-axis in the world coordinate system are determined by the posture of the vehicle at the current moment of system initialization. The expression of the vehicle's posture may be synchronously converted to the world coordinate system, and the coordinates of the center of the vehicle's rear axle and the posture of the vehicle are updated in real time during the entire stage of the automatic parking system operation.

[0076] Step S2, using a pre-trained semantic segmentation model to perform semantic segmentation and recognition on the surround panoramic image to obtain a drivable area;

[0077] Specifically, semantic segmentation is a very important field in computer vision. It refers to the pixel-level recognition of images, that is, marking the object category to which each pixel in the image belongs, and assigning different pixel values ​​to different object categories in the image. Specifically, in this embodiment, the drivable area is segmented from the non-drivable area. Through semantic segmentation, a drivable area under the field of view of the current vehicle camera can be obtained.

[0078] Step S3, regularizing the boundary of the drivable area to obtain boundary information, wherein the boundary information includes a plurality of boundary points;

[0079] Step S4, screening the plurality of boundary points according to a preset screening rule to obtain a plurality of pre-selected vacant parking space corner points;

[0080] Step S5, randomly selecting any 4 points as a group of pre-selected points to enumerate the multiple empty parking space corner pre-selected points to obtain multiple groups of pre-selected points, and respectively judging whether the multiple parking spaces formed by the multiple groups of pre-selected points meet the preset parking space conditions, and generating corresponding candidate empty parking spaces according to one or more groups of pre-selected points that meet the parking space conditions, and determining the target empty parking space that is finally output according to the candidate empty parking spaces.

[0081] As one of the optional solutions, step S3 in this embodiment includes the following steps S31-236:

[0082] Step S31, extracting the boundary of the drivable area to form a closed curve;

[0083] For example, the drivable area is as follows: Figure 2 As shown, the closed curve is Figure 3 shown.

[0084] Step S32, expanding the closed curve by one pixel point inside the curve to form a boundary search area;

[0085] For example, based on Figure 3 The closed curve of Figure 4 shown.

[0086] Step S34, selecting any one adjacent pixel point on the boundary search area as an initial search point for the boundary search;

[0087] Step S35, performing boundary search in the boundary search area according to the initial search point; during the search process, calculating the cost distance of other pixel points adjacent to the current search point according to a preset cost function, and selecting the pixel point with the smallest cost distance as the next search point to continue searching until the initial search point is found;

[0088] For example, based on Figure 4 The boundary search area, the corresponding boundary search results are as follows Figure 5 As shown, Figure 5 The mid-boundary search results include multiple search points.

[0089] Step S36: Regularize the multiple search points found during the boundary search to obtain boundary information of the drivable area.

[0090] Optionally, the calculating the cost distance of searching for other pixel points adjacent to the current search point according to a preset cost function includes:

[0091] d=d 1 ×ɑ+d 2 ×(1-ɑ)

[0092] d 1 =dis min

[0093] d 2 = abs(theta i-2,i-1 -theta i-1,i )

[0094] Among them, d is the cost distance; d 1 is the shortest distance between the pixel and the non-drivable area; d 2 theta is the angle between the direction of the previous search path and the direction of the current search path. For example, the two search points obtained in the previous search are D1 and D2, and D3 is the other pixel point adjacent to the current search point D2. Then the direction of the previous search path is D1→D2, and the direction of the current search path is D2→D3. i-1,i Represents a vector (x i -x i-1 ,yi -y i-1 ) and the x-axis in the vehicle coordinate system in radians; ɑ is the preset scaling factor used for d 1 With d 2 The proportion in the cost calculation is preferably, but not limited to, 0.3 in this embodiment.

[0095] Optionally, the step of performing regularization processing on a plurality of search points obtained through boundary search to obtain boundary information of the drivable area includes:

[0096] A straight line is fitted between three or more adjacent search points; if the fitting error is less than or equal to the preset error threshold, the two end points of the fitted straight line are used as boundary points; if the fitting error is greater than the preset error threshold, three or more adjacent search points are used as boundary points; finally, multiple boundary points of the regularized empty parking spaces are obtained.

[0097] Specifically, the fitting error refers to the difference between the fitted straight line and the line segment formed by connecting the original search points. This difference can be determined by, for example, calculating the similarity of the two line segments. It can be understood that there are many ways to calculate the similarity, and this embodiment is not limited to a certain calculation method. Specifically, for example, it is the difference in the coordinate position information of the two line segments.

[0098] It should be noted that, based on the method of this embodiment and a large number of sample tests, the boundary of the drivable area is composed of an average of 200 to 300 pixels before the boundary points are regularized, and the boundary of the drivable area can be represented by 20 to 50 pixels after the boundary regularization extraction, and the error after regularization is controlled within the error range allowed by the accuracy. The significance of regularizing the boundary points of the drivable area is to simplify the complexity of the subsequent algorithms in the automatic parking system and improve the overall operating efficiency of the system.

[0099] Optionally, the step S4 includes steps S41 to S42:

[0100] Step S41, calculate the path direction and path turning angle of each boundary point; wherein the path direction a of the i-th boundary point is i The angle between the directed line segment formed between the ith boundary point and the i+1th boundary point and the positive direction of the x-axis in the vehicle coordinate system rotated clockwise; the path angle theta of the ith boundary point i is the path direction a of the i-th boundary point i The path direction a with the i-1th boundary point i-1 The absolute value of the difference;

[0101] Specifically, the path direction in the step refers to the search direction during the above boundary search, which is determined by the current boundary point (x i ,y i) points to the next boundary point (x i+1 ,y i+1 ), where theta i = abs(a i -a i-1 ).

[0102] Step S42: transform the path angle theta in the plurality of boundary points i The boundary points that meet the preset value range are output as the pre-selected corner points of the empty parking space.

[0103] For example, in this embodiment, the boundary points of 30≤path turning angle≤150, or 210≤path turning angle≤330 are output as the pre-selected corner points of the empty parking space. Of course, in actual application, the above value range can be adjusted appropriately.

[0104] Optionally, the step S5 includes:

[0105] Step S51, randomly selecting any 4 points as a group of pre-selected points to enumerate the plurality of pre-selected points of the corner points of the empty parking spaces to obtain a plurality of groups of pre-selected points;

[0106] Step S52: respectively determining whether the multiple parking spaces formed by the multiple groups of pre-selected points meet the preset parking space conditions, and generating corresponding candidate empty parking spaces according to one or more groups of pre-selected points that meet the parking space conditions, and calculating the credibility of the candidate empty parking spaces;

[0107] Among them, for any set of pre-selected points (P k,1 , P k,2 , P k,3 , P k,4 ), calculate point P k,1 Solstice P k,4 The cumulative path length l k , the Euclidean distance between two adjacent points l k-12 , l k-23 , l k-34 , l k-41 , determine the calculated l k , l k-12 , l k-23 , l k-34 , l k-41 Whether the preset parking space conditions are met, if the group of pre-selected points (P k,1 , P k,2 , P k,3 , P k,4 ) meets the parking space condition, then the group of pre-selected points (P k,1 , P k,2 , P k,3 , P k,4) is marked as a pre-selected empty parking space corner point sequence, and a candidate empty parking space is obtained by performing parking space prediction based on a closed quadrilateral formed by the four points in each sequence;

[0108] Step S53: determining the target empty parking space to be outputted finally according to the credibility of the candidate empty parking spaces.

[0109] Optionally, the parking space conditions are specifically:

[0110] (car w ×2+car h )×k cmin <l k <(car w +car h ×2)×k cmax

[0111] k min <l k-12 / l k-34 <k max

[0112] k min <l k-23 / l k-41 <k max

[0113] Among them, car w and car h are the width and height of the current vehicle, respectively, k cmin , k cmax , k min and k max is the preset value;

[0114] Preferably, in this embodiment, k cmin and k cmax The values ​​of k are 1.1 and 1.5, min and k max The values ​​of are 0.8 and 1.3; the enumeration sequence that meets the conditions is marked as the pre-selected empty parking space corner point sequence.

[0115] Optionally, where:

[0116] Figure 6 This is a schematic diagram of candidate empty parking space prediction, see Figure 6 , for any pre-selected empty parking space corner point sequence (P k,1 , P k,2 , P k,3 , P k,4 ), P k,1 P k,2 P k,3 P k,4 is the original closed quadrilateral; let P k,1'P k,2 'P k,3 'P k,4 ' is the predicted empty parking space, C 1 , C 2 The line segments P k,1 P k,2 , P k,3 P k,4 The midpoint of O 2 O 4 C 1 C 2 The perpendicular bisector of 2 O 4 With O 1 O 3 perpendicular to each other and bisecting each other; then a parking space prediction is performed according to a preset constraint condition to obtain a plurality of candidate empty parking spaces; the constraint condition is that the length and width of the candidate empty parking space are respectively m times the length and width of the current vehicle;

[0117] Specifically, according to the length and width of the set candidate empty parking spaces, an empty parking space P corresponding to the original closed quadrilateral can be predicted. k,1 'P k,2 'P k,3 'P k,4 '.

[0118] Furthermore, the credibility score of the candidate vacant parking space is k The calculation is as follows:

[0119] score k =w 1 ×S 1 / S+w 2 ×S 2 / S+w 3 ×S 3 / S

[0120] Where S is P k,1 'P k,2 'P k,3 'P k,4 ' area, S 1 P k,1 'P k,2 'P k,3 'P k,4 'With P k,1 P k,2 P k,3 P k,4 The area of ​​the intersection, S 2 P k,1 'P k,2 'P k,3 'P k,4 'With P k,1 Pk,2 P k,3 P k,4 The intersection-over-combination ratio, S 3 P k,1 'P k,2 'P k,3 'P k,4 ' is the area of ​​the drivable area, w 1 、w 2 、w 3 is the preset weight value, weight w 1 、w 2 、w 3 In the embodiment, preferably, they are 0.3, 0.2, and 0.5 respectively; m is a preset value, and in this embodiment, m is preferably 1.2.

[0121] Optionally, the step S53 includes:

[0122] Determining whether the credibility of the plurality of candidate empty parking spaces is greater than the credibility threshold;

[0123] in:

[0124] (1) If there is only one candidate empty parking space whose credibility is greater than the credibility threshold, then the candidate empty parking space is determined as the empty parking space to be output;

[0125] (2) If the credibility of at least two candidate empty parking spaces is greater than the credibility threshold, the candidate empty parking space with the highest credibility is selected as the empty parking space to be output;

[0126] (3) If the credibility of the multiple candidate empty parking spaces is less than the credibility threshold, output a result that no empty parking space is detected.

[0127] Specifically, in this embodiment, the credibility threshold is preferably 0.9.

[0128] Optionally, the method of this embodiment further includes:

[0129] Obtain the current time information and vehicle position information, and determine whether to perform image semantic segmentation based on the current time information and vehicle position information; wherein, if the time interval between the current time and the time of the last image semantic segmentation is greater than a preset time threshold, or if the distance between the current vehicle position and the vehicle position at the last time image semantic segmentation was performed is greater than a preset distance threshold, then determine to perform image semantic segmentation and generate the start recognition signal.

[0130] Specifically, during the automatic parking process, the method of this embodiment will always update the vehicle's location information in real time, and only when the semantic segmentation conditions are triggered will the semantic segmentation model be used to identify the drivable area within the current field of view. The purpose of setting the semantic segmentation trigger conditions in this embodiment is to ensure the efficiency and effectiveness of drivable area identification. Because when the same image data is input, the semantic segmentation model based on deep learning with strong robustness will feedback the same or similar recognition output. If the recognition of the drivable area is repeated when the characteristics of the vehicle's surrounding environment do not change much at the same location, a large amount of unnecessary computing resources will be occupied, affecting the overall computing efficiency of the algorithm. However, redundant recognition of the drivable area does not bring significant performance improvements. Therefore, after the first image semantic segmentation is performed to output the drivable area, this embodiment determines whether to perform image semantic segmentation in real time based on the current time information and vehicle location information to save computing resources and improve computing efficiency.

[0131] Another embodiment of the present invention provides an empty parking space recognition system, which is used to execute the empty parking space recognition method described in the above embodiment, and the system includes:

[0132] The image acquisition unit 1 is used to acquire a panoramic image of the surrounding environment of the current vehicle in response to receiving the start recognition signal;

[0133] A semantic segmentation unit 2 is used to perform semantic segmentation and recognition on the surround panoramic image using a pre-trained semantic segmentation model to obtain a drivable area;

[0134] A boundary extraction unit 3, used for performing regularization processing on the boundary of the drivable area to obtain boundary information, wherein the boundary information includes a plurality of boundary points;

[0135] A pre-selected point screening unit 4 is used to screen the plurality of boundary points according to a preset screening rule to obtain a plurality of pre-selected vacant parking space corner points; and

[0136] The empty parking space prediction unit 5 is used to enumerate the plurality of empty parking space corner pre-selected points by randomly selecting any 4 points as a group of pre-selected points to obtain a plurality of groups of pre-selected points, and to respectively determine whether the plurality of parking spaces formed by the plurality of groups of pre-selected points meet the preset parking space conditions, and to generate corresponding candidate empty parking spaces according to one or more groups of pre-selected points that meet the parking space conditions, and to determine the target empty parking space that is finally output according to the candidate empty parking spaces.

[0137] Optionally, the system further comprises:

[0138] A trigger unit is used to obtain current time information and vehicle position information, and determine whether to perform image semantic segmentation based on the current time information and vehicle position information; wherein, if the time interval between the current time and the time of the last image semantic segmentation is greater than a preset time threshold, or if the distance between the current vehicle position and the vehicle position at the last image semantic segmentation is greater than a preset distance threshold, it is determined to perform image semantic segmentation, generate a start recognition signal, and send the start recognition signal to the image acquisition unit.

[0139] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] It should be noted that the system described in the above embodiment corresponds to the method described in the above embodiment. Therefore, the undetailed parts of the system described in the above embodiment can be obtained by referring to the contents of the method described in the above embodiment, and will not be repeated here.

[0141] Furthermore, if the vacant parking space recognition system described in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0142] Exemplarily, the computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0143] In addition, another embodiment of the present invention further proposes another computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the method for identifying empty parking spaces described in the above embodiment.

[0144] Of course, the computer device may also have components such as a wired or wireless network interface, a keyboard, and an input / output interface for input and output. The computer device may also include other components for realizing device functions, which will not be described in detail here.

[0145] Exemplarily, the computer program may be divided into one or more units, which are stored in the memory and executed by the processor to implement the present invention. The one or more units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the computer device.

[0146] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the computer device, and uses various interfaces and lines to connect various parts of the entire computer device.

[0147] The memory can be used to store the computer program and / or unit, and the processor realizes various functions of the computer device by running or executing the computer program and / or unit stored in the memory, and calling the data stored in the memory. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0148] Another embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the empty parking space identification method described in the above embodiment are implemented.

[0149] Specifically, the computer-readable storage medium may include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0150] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for identifying empty parking spaces, It is characterized in that include: Step S1, in response to receiving a start identification signal, obtaining a panoramic image of the current vehicle surrounding environment; Step S2, using a pre-trained semantic segmentation model to perform semantic segmentation and recognition on the surround panoramic image to obtain a drivable area; Step S3, regularizing the boundary of the drivable area to obtain boundary information, wherein the boundary information includes a plurality of boundary points; Step S4, screening the plurality of boundary points according to a preset screening rule to obtain a plurality of pre-selected empty parking space corner points, including obtaining a path direction and a path turning angle of each boundary point, and outputting a boundary point whose path turning angle among the plurality of boundary points satisfies a preset numerical range as a pre-selected empty parking space corner point; Step S5, randomly selecting any 4 points as a group of pre-selected points to enumerate the multiple empty parking space corner pre-selected points to obtain multiple groups of pre-selected points, and respectively judging whether the multiple parking spaces formed by the multiple groups of pre-selected points meet the preset parking space conditions, and generating corresponding candidate empty parking spaces according to one or more groups of pre-selected points that meet the parking space conditions, and determining the target empty parking space that is finally output according to the candidate empty parking spaces.

2. The vacant parking space recognition method according to claim 1, It is characterized in that The step of regularizing the boundary of the drivable area to obtain boundary information includes: Extracting the boundary of the drivable area to form a closed curve; Expand the closed curve by one pixel point inside the curve to form a boundary search area; Selecting any one adjacent pixel point on the boundary search area as an initial search point for the boundary search; Performing a boundary search in the boundary search area according to the initial search point; during the search process, calculating the cost distance of other pixel points adjacent to the current search point according to a preset cost function, and selecting the pixel point with the smallest cost distance as the next search point to continue searching until the initial search point is found; Regularization is performed on multiple search points found through boundary search to obtain the boundary information.

3. The vacant parking space recognition method according to claim 2, It is characterized in that The step of calculating the cost distance of other pixel points adjacent to the current search point according to a preset cost function includes: d = d 1 × ɑ + d 2 ×(1 - ɑ ) d 1 = dis min d 2 = abs ( theta i-2,i-1 - theta i-1,i ) in, d For the cost distance; d 1 is the shortest distance between the pixel and the non-empty parking space; d 2 is the angle between the direction of the previous search path and the direction of the current search path; theta i-1,i Represents a vector ( x i - x i-1 , y i - y i-1 ) and the vehicle coordinate system x The radian value of the angle between the axis and the ground; ɑ is the preset scaling factor for d 1 and d 2 The proportion in the cost calculation.

4. The vacant parking space recognition method according to claim 2, It is characterized in that The step of performing regular processing on the multiple search points obtained by boundary search to obtain boundary information of the drivable area includes: A straight line is fitted between three or more adjacent search points; if the fitting error is less than or equal to the preset error threshold, the two end points of the fitted straight line are used as boundary points; if the fitting error is greater than the preset error threshold, three or more adjacent search points are used as boundary points; finally, multiple boundary points of the regularized empty parking spaces are obtained.

5. The vacant parking space recognition method according to any one of claims 1 to 4, It is characterized in that The step S4 comprises: No. i The path direction of the boundary points a i For the i The boundary point and i +1 The directed line segment between the boundary points and the vehicle coordinate system x The angle of rotation of the positive direction of the axis in clockwise direction; i Path turning angles at boundary points theta i For the i Boundary point path direction a i With i -1 boundary point path direction a i-1 The absolute value of the difference.

6. The vacant parking space identification method according to claim 1, It is characterized in that The step S5 comprises: Step S51, randomly selecting any 4 points as a group of pre-selected points to enumerate the plurality of pre-selected points of the corner points of the empty parking spaces to obtain a plurality of groups of pre-selected points; Step S52: respectively determining whether the multiple parking spaces formed by the multiple groups of pre-selected points meet the preset parking space conditions, and generating corresponding candidate empty parking spaces according to one or more groups of pre-selected points that meet the parking space conditions, and calculating the credibility of the candidate empty parking spaces; Among them, for any set of pre-selected points ( P k,1 , P k,2 , P k,3 , P k,4 ), calculation point P k,1 Solstice P k,4 The cumulative path length l k , the Euclidean distance between two adjacent points l k-12 , l k-23 , l k-34 , l k-41 , determine the calculated l k , l k-12 , l k-23 , l k-34 , l k-41 Whether the preset parking space conditions are met, if the group of pre-selected points ( P k,1 , P k,2 , P k,3 , P k,4 ) meets the parking space conditions, then the group of pre-selected points ( P k,1 , P k,2 , P k,3 , P k,4 ) is marked as a pre-selected empty parking space corner point sequence, and a candidate empty parking space is obtained by predicting the parking space based on the closed quadrilateral formed by the four points in each sequence; Step S53: determining the target empty parking space to be outputted finally according to the credibility of the candidate empty parking spaces.

7. The method for identifying an empty parking space according to claim 6, It is characterized in that The parking space conditions are specifically: ( car w ×2+ car h ) × k cmin < l k < ( car w + car h ×2) × k cmax k min < l k-12 / l k-34 < k max k min < l k-23 / l k-41 < k max in, car w and car h are the width and height of the current vehicle respectively, k cmin , k cmax , k min and k max is the preset value.

8. The method for identifying an empty parking space according to claim 6, It is characterized in that in: For any pre-selected empty parking space corner point sequence ( P k,1 , P k,2 , P k,3 , P k,4 ), P k,1 P k,2 P k,3 P k,4 is the original closed quadrilateral; let P k,1 ' P k,2 ' P k,3 ' P k,4 ' is the predicted empty parking space, C 1 , C 2 Line segments P k,1 P k,2 , P k,3 P k,4 The midpoint of O 2 O 4 for C 1 C 2 The perpendicular bisector of O 2 O 4 and C 1 C 2 perpendicular to each other and bisect each other; then, a parking space prediction is performed according to the preset constraint conditions to obtain multiple candidate empty parking spaces; the constraint conditions are that the length and width of the candidate empty parking space are respectively the length and width of the current vehicle m times; The credibility of the candidate empty parking space score k The calculation is as follows: score k = w 1 × S 1 / S + w 2 × S 2 / S + w 3 × S 3 / S in, S for P k,1 ' P k,2 ' P k,3 ' P k,4 'The area, S 1 for P k,1 ' P k,2 ' P k,3 ' P k,4 'and P k,1 P k,2 P k,3 P k,4 The area of ​​the intersection, S 2 for P k,1 ' P k,2 ' P k,3 ' P k,4 'and P k,1 P k,2 P k,3 P k,4 The intersection ratio of S 3 for P k,1 ' P k,2 ' P k,3 ' P k,4 'The area of ​​the drivable area, w 1 、w 2 、w 3 is the preset weight value, m is the preset value.

9. The method for identifying an empty parking space according to claim 8, It is characterized in that The step S53 comprises: Determining whether the credibility of the plurality of candidate empty parking spaces is greater than a credibility threshold; If there is only one candidate empty parking space whose credibility is greater than the credibility threshold, then the candidate empty parking space is determined as the empty parking space to be output; If the credibility of at least two candidate empty parking spaces is greater than the credibility threshold, the candidate empty parking space with the highest credibility is selected as the empty parking space to be output; If the credibility of the plurality of candidate empty parking spaces is less than the credibility threshold, a result indicating that no empty parking space is detected is output.

10. The vacant parking space identification method according to claim 1, It is characterized in that Also includes: Obtain the current time information and vehicle position information, and determine whether to perform image semantic segmentation based on the current time information and vehicle position information; wherein, if the time interval between the current time and the time of the last image semantic segmentation is greater than a preset time threshold, or if the distance between the current vehicle position and the vehicle position at the last time image semantic segmentation was performed is greater than a preset distance threshold, then determine to perform image semantic segmentation and generate the start recognition signal.

11. A vacant parking space recognition system, It is characterized in that The method for identifying an empty parking space according to any one of claims 1 to 10 is characterized in that the system comprises: An image acquisition unit, configured to acquire a panoramic image of the current vehicle surroundings in response to receiving a start identification signal; A semantic segmentation unit, used to perform semantic segmentation and recognition on the surround panoramic image using a pre-trained semantic segmentation model to obtain a drivable area; A boundary extraction unit, used for performing regularization processing on the boundary of the drivable area to obtain boundary information, wherein the boundary information includes a plurality of boundary points; A pre-selected point screening unit, configured to screen the plurality of boundary points according to a preset screening rule to obtain a plurality of pre-selected vacant parking space corner points; and The empty parking space prediction unit is used to enumerate the multiple empty parking space corner pre-selected points by randomly selecting any 4 points as a group of pre-selected points to obtain multiple groups of pre-selected points, and respectively determine whether the multiple parking spaces formed by the multiple groups of pre-selected points meet the preset parking space conditions, and generate corresponding candidate empty parking spaces according to one or more groups of pre-selected points that meet the parking space conditions, and determine the target empty parking space that is finally output according to the candidate empty parking spaces.

12. The vacant parking space recognition system according to claim 11, It is characterized in that The system further comprises: A trigger unit is used to obtain current time information and vehicle position information, and determine whether to perform image semantic segmentation based on the current time information and vehicle position information; wherein, if the time interval between the current time and the time of the last image semantic segmentation is greater than a preset time threshold, or if the distance between the current vehicle position and the vehicle position at the last image semantic segmentation is greater than a preset distance threshold, it is determined to perform image semantic segmentation, generate a start recognition signal, and send the start recognition signal to the image acquisition unit.

13. A computer device, include: The vacant parking space recognition system as described in any one of claims 11-12; or, comprising: a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the vacant parking space recognition method as described in any one of claims 1-10.

14. A computer-readable storage medium having a computer program stored thereon, Features: When the computer program is executed by a processor, the vacant parking space identification method according to any one of claims 1 to 10 is implemented.

Citation Information

Patent Citations

  • Vehicle information identification method and system

    CN105574485A

  • Method and system for identifying empty parking space and vehicle

    CN109871745A

  • Automatic parking method and device

    WO2020019930A1