Parking space inspection methods and storage media, parking space inspection equipment, vehicles

Abstract

This invention discloses a parking space detection method, storage medium, parking space detection equipment, and vehicle. The parking space detection method includes: acquiring an environmental image surrounding the vehicle; inputting the environmental image into a pre-trained target detection model, outputting a parking space entry line target bounding box and a parking space entrance line corner target bounding box; and determining the parking space based on the parking space entry line target bounding box and the parking space entrance line corner target bounding box. This method utilizes a pre-trained target detection model to directly output the parking space entry line target bounding box and the parking space entrance line corner target bounding box from the environmental image, improving parking space detection efficiency. Furthermore, by determining the parking space based on these target bounding boxes, the method becomes applicable to various scenarios and parking space types, capable of detecting parking spaces of any color and angle.

Description

Technical Field

[0001] This invention relates to the field of parking space detection technology, and in particular to a parking space detection method and storage medium, parking space detection equipment, and vehicle. Background Technology

[0002] For parking space detection, related technologies propose first using a network to detect corner points, and then matching these corner points. However, the process of obtaining corner points requires multiple image processing steps, such as binarization, corner point extraction, and feature matching, which is cumbersome and results in low detection efficiency. Summary of the Invention

[0003] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, one objective of this invention is to propose a parking space detection method that improves parking space detection efficiency and is applicable to various scenarios and types of parking spaces, including those of any color and angle.

[0004] A second objective of this invention is to provide a computer-readable storage medium.

[0005] The third objective of this invention is to provide a parking space detection device.

[0006] The fourth objective of this invention is to provide a vehicle.

[0007] To achieve the above objectives, a first aspect of the present invention provides a parking space detection method, the method comprising: acquiring an environmental image of the area surrounding a vehicle; inputting the environmental image into a pre-trained target detection model, and outputting a parking space entry line target box and a parking space entrance line corner target box; and determining a parking space based on the parking space entry line target box and the parking space entrance line corner target box.

[0008] According to the parking space detection method of the present invention, a pre-trained target detection model is used to directly output the target bounding box of the parking space entry line and the target bounding box of the parking space entrance line corner in the environmental image, which improves the parking space detection efficiency. Then, the parking space is determined based on the target bounding box of the parking space entry line and the target bounding box of the parking space entrance line, making the parking space detection method applicable to various scenarios and various types of parking spaces, and able to detect parking spaces of any color and angle.

[0009] In addition, the parking space detection method proposed in the above embodiments of the present invention may also have the following additional technical features:

[0010] According to an embodiment of the present invention, determining a parking space based on the target frame of the parking space entry line and the target frame of the corner point of the parking space entrance line includes: extracting corner point pair information of the parking space entrance line based on the target frame of the parking space entry line and the target frame of the corner point of the parking space entrance line to obtain a corner point information array and a corner point matching information array, wherein the corner point information array includes the position information and angle information of each corner point, and the corner point matching information array includes the numbering information and pointing information of the corner points that constitute the complete parking space entrance line in the corner point information array; determining the parking space based on the corner point information array and the corner point matching information array.

[0011] According to an embodiment of the present invention, the step of extracting corner point pair information of the parking space entrance line based on the parking space entry line target frame and the parking space entrance line corner point target frame to obtain a corner point information array and a corner point matching information array includes: obtaining the corner point position based on the parking space entrance line corner point target frame, and recording the corner point corresponding to the corner point position as the detected corner point; taking the corner point position of each detected corner point and the angle information attached to the corresponding parking space entrance line corner point target frame as an element of the corner point information array; for each parking space entry line target frame, determining the parking space entrance line corner point pair within the parking space entry line target frame based on the detected corner point, and determining the pointing information of the two corner points in the parking space entrance line corner point pair based on the angle information attached to the parking space entry line target frame, and taking the numbering information and pointing information of the two corner points in the parking space entrance line corner point pair in the corner point information array as an element of the corner point matching information array.

[0012] According to an embodiment of the present invention, determining the parking space entrance line corner point pair within the target frame of the parking space entrance line based on the detected corner points includes: determining whether two detected corner points are simultaneously within the target frame of the parking space entrance line; if two detected corner points are simultaneously within the target frame of the parking space entrance line, then the two detected corner points are taken as the parking space entrance line corner point pair; if one detected corner point is within the target frame of the parking space entrance line, then based on the position information of the detected corner point, the position information of the target frame of the parking space entrance line, and a preset position conversion relationship, the position information of another corner point within the target frame of the parking space entrance line is obtained, and the angle information of the other corner point is determined as the angle information of the detected corner point, and the detected corner point and the other corner point are taken as the parking space entrance line corner point pair; wherein, the method further includes: taking the position information and angle information of the other corner point as an element in the corner point information array.

[0013] According to an embodiment of the present invention, determining a parking space based on the corner information array and the corner matching information array includes: for each element in the corner matching information array, determining the parking space type of the corresponding parking space based on the angle information and pointing information of the two corner points aligned with the parking space entrance line in that element; obtaining the position information of the internal corner points of the corresponding parking space based on the parking space type and the position information and angle information of the two corner points aligned with the parking space entrance line; and determining the parking space based on the position information of the two corner points aligned with the parking space entrance line and the position information of the internal corner points of the corresponding parking space.

[0014] According to an embodiment of the present invention, the corner matching information array further includes parking space occupancy information of the corresponding parking space, the occupancy information being obtained based on the parking space occupancy information attached to the target box of the corresponding parking space entry line, and the method further includes: obtaining the occupancy status of the corresponding parking space based on the occupancy information.

[0015] According to an embodiment of the present invention, the training process of the target detection model includes: acquiring training data, wherein the training data includes a vehicle surround-view top view and target box real information in a parking lot environment, wherein the target box real information includes the real target box category, target box position information, cosine and sine values ​​of the target box's attached angle, and parking space occupancy information, and the target box category includes parking space entry line target boxes and parking space entrance line corner point target boxes; constructing a target detection model, inputting the training data into the target detection model, and outputting target box prediction information, wherein the target box prediction information includes the predicted target box position information, the probability of the target box appearing, and the target box... The target detection model employs a YOLO network structure. It utilizes a sigmoid function to activate the target box based on its location, probability of occurrence, and the angle values ​​(cosine and sine) of the target box's location and the angle values ​​(cosine and sine) of the target box's location. A tanh activation function is used for the angle values ​​of the target box. Based on the true and predicted information, an overall loss function is constructed, and the parameters of the target detection model are adjusted using this overall loss function.

[0016] According to one embodiment of the present invention, the overall loss function is as follows:

[0017] loss = γ coord L EIOU +γ obj L obj +γ cls L cls +γ θ L θ +γvalid L valid

[0018] Where loss is the overall loss function, L EIOU L obj L cls L θ L valid These are the bounding box loss function, confidence loss function, category loss function, angle loss function, and occupancy information loss function, respectively. coord γ obj γ cIs γ θ γ valid Loss function L EIOU L obj L cls L θ L valid The weight, L EIOU Using the crossover ratio (EIOU) loss function, L obj L cls Using the cross-entropy loss function, cos and sin are the predicted cosine and sine values ​​of the angle, respectively; sin and cos are the actual cosine and sine values ​​of the angle, respectively; and L1() is the L1 norm. v represents the predicted and actual values ​​of the occupied information, respectively, and m represents the number of samples.

[0019] According to an embodiment of the present invention, before determining the parking space based on the parking space entry line target box and the parking space entrance line corner point target box, the method further includes: performing non-maximum suppression (NMS) processing on the parking space entry line target box and the parking space entrance line corner point target box.

[0020] To achieve the above objectives, a second aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the parking space detection method as proposed in the first aspect of the present invention.

[0021] To achieve the above objectives, a third aspect of the present invention provides a parking space detection device, including a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the parking space detection method as proposed in the first aspect of the present invention.

[0022] To achieve the above objectives, a fourth aspect of the present invention provides a vehicle including a parking space detection device as described in the third aspect of the present invention.

[0023] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] Figure 1 This is a flowchart of a parking space detection method according to an embodiment of the present invention;

[0025] Figure 2 This is a flowchart of a training target detection model according to an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram of the target frame for the parking space entry line and the target frame for the corner point of the parking space entrance line according to an embodiment of the present invention;

[0027] Figure 4 This is a schematic diagram of an image pixel coordinate system according to an embodiment of the present invention;

[0028] Figure 5 This is a diagram illustrating the inference structure of a target detection model according to an embodiment of the present invention.

[0029] Figure 6 This is a flowchart illustrating how a parking space is determined based on a target frame of the parking space entry line and a target frame of the corner point of the parking space entrance line, according to one embodiment of the present invention.

[0030] Figure 7 This is a flowchart illustrating the process of obtaining a corner information array and a corner matching information array according to an embodiment of the present invention;

[0031] Figure 8(a) , 8(b) Figures 8(c), 8(d), and 8(e) are schematic diagrams illustrating the connection sequence of the corner points of the parking space entrance line according to an embodiment of the present invention.

[0032] Figure 9 This is a schematic diagram illustrating the detection of any corner point according to an embodiment of the present invention;

[0033] Figure 10 is a diagram showing the occupancy of corresponding parking spaces according to an embodiment of the present invention;

[0034] Figure 11 This is a flowchart illustrating how a parking space is determined based on a corner information array and a corner matching information array, according to an embodiment of the present invention.

[0035] Figure 12(a) , 12(b) 12(c) are schematic diagrams of vertical parking spaces, horizontal parking spaces, and angled parking spaces, respectively;

[0036] Figure 13(a) , 13(b) 13(c) Schematic diagrams showing the positions of the vehicle body relative to the perpendicular, horizontal and angled parking spaces, respectively;

[0037] Figure 14 This is a schematic diagram of a parking space entry line and a parking space entrance line according to an embodiment of the present invention;

[0038] Figure 15 The image shows a parking space detection method provided in this embodiment of the invention.

[0039] Figure 16 This is a schematic diagram of a parking space detection device according to an embodiment of the present invention;

[0040] Figure 17 This is a schematic diagram of a vehicle according to an embodiment of the present invention. Detailed Implementation

[0041] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0042] The parking space detection method, storage medium, parking space detection equipment, and vehicle of the present invention will be described in detail below with reference to Sections 1-17 and specific implementation methods.

[0043] Figure 1 This is a flowchart of a parking space detection method according to an embodiment of the present invention. Figure 1 As shown, parking space detection methods may include:

[0044] S1, acquire an image of the environment surrounding the vehicle.

[0045] Specifically, when a vehicle is detecting parking spaces, the vehicle body camera is used to acquire environmental images around the vehicle (vehicle surround view top view), and parking spaces around the vehicle are detected based on the acquired vehicle surround view top view.

[0046] S2 inputs the environmental image into the pre-trained target detection model and outputs the target bounding boxes of the parking space entry line and the corner points of the parking space entrance line.

[0047] Specifically, a pre-trained target detection model can be used to detect objects in the environment image, obtaining the target bounding boxes of the parking space entry line and the corner points of the parking space entrance line, so that vehicles can determine the target parking space based on the target bounding boxes of the parking space entry line and the corner points of the parking space entrance line.

[0048] In one embodiment of the present invention, such as Figure 2 As shown, the training process of an object detection model may include:

[0049] S21, acquire training data, which includes the vehicle surround view top view and target box real information in the parking lot environment. The target box real information includes the real target box category, the target box position information, the cosine and sine values ​​of the target box's attached angle, and the parking space occupancy information. The target box category includes the parking space entry line target box and the parking space entrance line corner point target box.

[0050] Specifically, when training the object detection model, it is necessary to acquire a large number of RGB images of the vehicle's surround-view overhead view in a parking lot environment, and to annotate the actual bounding box information in the vehicle's surround-view overhead view in the parking lot environment for use in training the object detection model. The annotated actual bounding box information may include the actual bounding box category (parking space entry line bounding box and parking space entrance line corner bounding box), the position information of the bounding box, the cosine and sine values ​​of the bounding box's attached angle, and parking space occupancy information.

[0051] In embodiments of the present invention, there are two types of target boxes: one is the target box for the corner point of the parking space entrance line, and the other is the target box for the parking space entry line (a parking space entry line can be formed between the corner points of two parking space entrance lines). The target box for the parking space entry line can be referred to as the ps target box. The target boxes for the parking space entry line and the corner point of the parking space entrance line can be found in [reference needed]. Figure 3 , Figure 3 The dark gray rectangle on the right is the target box for the parking space entry line. Figure 3 The two light gray rectangles in the upper right and lower right corners are the target boxes for the corner points of the parking space entrance line.

[0052] To ensure the synchronized output of the binding information between parking space entrance line corner points and their corresponding parking space line angles, the actual target bounding box information can be recorded as: (cls_id, x, y, w, h, cos, sin, occupied). Here, cls_id represents the probability of the category; cls_id = 0 indicates an entrance line corner point; cls_id = 1 indicates the entrance line itself. (x, y, w, h) represents the position information of the target bounding box. (cos, sin) represents the sine and cosine information of the angles attached to the target bounding box. The line angle information considered for the parking space entrance line corner point target bounding box is detailed in [link to relevant documentation]. Figure 3 The light gray arrow line within the light gray rectangle indicates the angle information considered by the target box for the parking space entry line. Figure 3 The dark gray arrow line within the dark gray rectangular frame (this angle points from one entrance line corner to another within the same parking space; these two entrance line corners must be corners of the same parking space's entrance line, and the arrows must point in the same direction following the rule of entrance line corner -> entrance line corner -> inner corner -> inner corner). It should be noted that the angle attached to the target box refers to the angle between the arrow line and the u-axis in the image pixel coordinate system of the vehicle surround view top view. See the image pixel coordinate system... Figure 4The UV direction in the image. "Occupied" indicates whether the parking space where the target is located is occupied in the corresponding vehicle surround view top view. When occupied, occupied = 1; otherwise, occupied = 0.

[0053] After obtaining the location information of the target bounding box, the corresponding target bounding box can be obtained based on its location. For the target bounding box of the parking space entrance line corner, the target bounding box (rectangle) of the parking space entrance line corner can be determined based on the (x,y) of the parking space entrance line corner. Specifically, the (x,y) of the parking space entrance line corner is directly used as the center point of the target bounding box, and a fixed parameter (delta_W,delta_H) is given as the size of the target bounding box. For the target bounding box of the parking space entrance line, the target bounding box (rectangle) of the parking space entrance line can be determined based on the parking space entrance line (the coordinates of the edge points of the line are set to (x1,y1,x2,y2)). Specifically, the center point of the parking space entrance line target bounding box is set... Let xmin = min(x1,x2), ymin = min(y1,y2), xmax = max(x1,x2), and ymax = max(y1,y2). To ensure that the rectangle completely contains all the information of the parking space entrance line, set the rectangle w = xmax - xmin + W_e, h = ymax - ymin + H_e, where W_e and H_e are pre-given thresholds greater than 0.

[0054] S22, construct an object detection model and input the training data into the object detection model to output object bounding box prediction information. The object bounding box prediction information includes the predicted object bounding box location information, the probability of the object bounding box appearing, the probability that the object bounding box belongs to the corner point of the parking space entrance line and the probability that it belongs to the parking space entry line, the cosine and sine values ​​of the angle attached to the object bounding box, and the parking space occupancy information attached to the object bounding box. The object detection model adopts a YOLO network structure, and the sigmoid function is used to activate the object bounding box location information, the probability of the object bounding box appearing, the probability that the object bounding box belongs to the corner point of the parking space entrance line and the probability that it belongs to the parking space entry line, and the parking space occupancy information. The tanh activation function is used for the cosine and sine values ​​of the angle attached to the object bounding box.

[0055] The target detection model in this embodiment of the invention uses a YOLO network structure as the main framework. This model detects parking space entrance line corner points as independent targets, while avoiding complex corner point matching issues later. This embodiment transforms the parking space entrance line corner point matching process into a PS (Parking Position Frame) target box recognition process; corner points of the same parking space entrance line are always within a single PS target box. This embodiment directly changes the parking space entrance line corner point matching logic into a PS target box detection process, greatly simplifying subsequent processing steps. In the YOLO network structure, in addition to outputting the predicted target box position information, the probability of the target box appearing, the probability that the target box belongs to a parking space entrance line corner point target box, and the probability that it belongs to a parking space entry line target box, it also needs to output the angle cosine and sine values ​​of the target box, and the parking space occupancy information attached to the target box. To enable the YOLO network structure to output the angle cosine and sine values ​​of the target box, and the parking space occupancy information attached to the target box, adjustments are made to the YOLO network structure.

[0056] Specifically, the dimensional information output by the YOLO network structure was expanded, making the output information (x, y, w, h, conf, cls1_p, cls2_p, cos, sin, valid, occupied). Here, (x, y, w, h) represents the location information of the bounding box, conf represents the probability that the bounding box is a target, (cls1_p, cls2_p) represents the probability that the target belongs to one of two categories, and (cos, sin) represents the sine and cosine values ​​of the angle of the parking space line where the corner point (i.e., the center point of the bounding box) is located. Using sine and cosine calculations instead of directly recording the angle is because angle is a periodic function, and directly regressing an angle involves a series of complex normalization operations, which sine and cosine calculations avoid. (valid, occupied) uses a classification approach to determine whether the parking space is occupied; (1, 0) indicates vacant, and (0, 1) indicates occupied. It should be noted that: for the corner point category of parking space entrance line, there is no (valid, occupied) information content, but for the purpose of dimension alignment, this embodiment of the invention directly assigns all (valid, occupied) values ​​of (1, 0) to the corner points of the entrance line.

[0057] Since the values ​​of (x, y, w, h, conf, cls1_p, cls2_p) and (valid, occupied) are in the range [0, 1], the sigmoid function can be used directly to activate them, which meets the requirements. The values ​​of (cos, sin) are in the range [-1, 1], so the tanh activation function can be used. The expression for the sigmoid function is as follows:

[0058]

[0059] The expression for the tanh activation function is as follows:

[0060]

[0061] S23. Construct an overall loss function based on real and predicted information, and use the overall loss function to adjust the parameters of the target detection model.

[0062] In one embodiment of the present invention, the overall loss function is as follows:

[0063] loss = γ coord L EIOU +γ obj L obj +γ cls L cls +γ θ L θ +γ valid L valid

[0064] Where loss is the overall loss function, L EIOU L obj L cls L θ L valid These are the bounding box loss function, confidence loss function, category loss function, angle loss function, and occupancy information loss function, respectively. coord γ obj γ cls γ θ γ valid Loss function L EIOU L obj L cls L θ L valid The weight, L EIOU Using the crossover ratio (EIOU) loss function, L obj L cls Using the cross-entropy loss function, Let be the predicted cosine and sine values ​​of the angle, and sin and cos be the actual cosine and sine values ​​of the angle, respectively. Let L1() be the L1 norm. v represents the predicted and actual values ​​of the occupied information, respectively, and m represents the number of samples.

[0065] For loss based on target category and confidence level, the BCELoss (Binary CrossEntropy Loss) loss function can be used. The expression for the BCELoss loss function is as follows:

[0066]

[0067] in, y represents the predicted value and the actual value, respectively.

[0068] For (cos, sin), in order to pay more attention to the loss of angle, the L1 norm can be used directly for calculation, and strong constraints are also imposed on whether the sum of its squares is 1.

[0069] For calculating the location loss of (x,y,w,h), EIOU loss (Efficient Intersectionover Union) can be used.

[0070] Therefore, the overall loss function of the target detection model in this embodiment of the invention is roughly implemented as follows:

[0071] loss = γ coord L EIOU +γ obj L obj +γ cls L cls +γ θ L θ +γ valid L valid

[0072] Among them, L EIOU Let L be the target bounding box loss function. obj Let L be the confidence loss function. cls The loss function is γ, where γ represents the different weights involved in different loss functions.

[0073] L θ It is a loss function (angle loss function) added to the (cos, sin) dimension information, and its specific calculation method is as follows:

[0074]

[0075] L θ Constraint angle loss. Utilizing L θ By adding constraints to the angle loss function, corner point binding relationships can be obtained, thereby enabling the calibration of parking space frames. Among these, Represents the network prediction value, while sin and cos are the true values ​​of the input.

[0076] L valid The loss function (occupied information loss function) is designed based on the (valid, occupied) dimension information. Since these two pieces of information are considered as a special classification structure, the applicable loss function is BCELoss:

[0077]

[0078] in, v and v represent the predicted value and the actual value of occupied or idle status, respectively.

[0079] By training the object detection model using the steps described above until the overall loss function of the object detection model converges to a minimum, a well-trained object detection model can be obtained. Figure 5 The inference structure diagram of the target detection model according to an embodiment of the present invention is shown.

[0080] S3, determine the parking space based on the target box of the parking space entry line and the target box of the corner point of the parking space entrance line.

[0081] In one embodiment of the present invention, before determining the parking space based on the parking space entry line target box and the parking space entrance line corner point target box, the parking space detection method may further include: performing non-maximum suppression (NMS) processing on the parking space entry line target box and the parking space entrance line corner point target box.

[0082] In some embodiments, before determining the parking space based on the parking space entry line target box and the parking space entrance line corner target box, non-maximum suppression (NMS) processing is applied to the parking space entry line target box and the parking space entrance line corner target box to filter the parking space entry line target boxes and parking space entrance line corner target boxes in the environmental image, obtaining parking space entry line target boxes and parking space entrance line corner target boxes that meet the filtering conditions. Simultaneously, the sine and cosine values ​​corresponding to the parking space entry line target boxes and the parking space entrance line corner target boxes are directly converted into angle information and recorded.

[0083] In one embodiment of the present invention, such as Figure 6 As shown, determining a parking space based on the target bounding box of the parking space entry line and the target bounding box of the corner point of the parking space entrance line can include:

[0084] S31. Based on the target box of the parking space entry line and the target box of the corner point of the parking space entrance line, extract the corner point pair information of the parking space entrance line to obtain a corner point information array and a corner point matching information array. The corner point information array includes the position information and angle information of each corner point, and the corner point matching information array includes the number information and pointing information of the corner points that constitute the complete parking space entrance line in the corner point information array.

[0085] In one embodiment of the present invention, such as Figure 7 As shown, based on the target bounding box of the parking space entry line and the target bounding box of the corner points of the parking space entrance line, corner point pair information of the parking space entrance line is extracted, resulting in a corner point information array and a corner point matching information array, which may include:

[0086] S311, obtain the corner point position based on the target box of the corner point of the parking space entrance line, and record the corner point corresponding to the corner point position as the detected corner point.

[0087] Specifically, the center point of the target box of the corner point of the parking space entrance line is recorded as the corner point position. The center point of the target box of the corner point of the parking space entrance line retained after non-maximum suppression (NMS) filtering is obtained, the corner point position of the corresponding target box of the corner point of the parking space entrance line is obtained, and the corner point corresponding to the obtained corner point position is recorded as the detected corner point.

[0088] S312, take the corner position of each detected corner point and the angle information attached to the target box of the corresponding parking space entrance line corner point as an element of the corner information array.

[0089] Specifically, the angle information attached to the target bounding box of the parking space entrance line corner point corresponding to the corner point position of each detected corner point is obtained, and the corner point position of each detected corner point and the angle information attached to its corresponding target bounding box of the parking space entrance line corner point are used as an element of the corner point information array. Here, marks[i] = [x, y, theta] can be used to represent the corner point information array, where i represents the number of the detected corner point, x and y are the corner point positions of the detected corner points, and theta is the angle information attached to the target bounding box of the corresponding parking space entrance line corner point.

[0090] S313, for each parking space entry line target frame, determine the parking space entrance line corner point pair within the parking space entry line target frame based on the detected corner points, determine the pointing information of the two corner points in the parking space entrance line corner point pair based on the angle information attached to the parking space entry line target frame, and take the numbering information and pointing information of the two corner points in the corner point information array as an element in the corner point matching information array.

[0091] In one embodiment of the present invention, determining the pair of parking space entrance line corner points within the target frame of the parking space entrance line based on the detected corner points may include:

[0092] Determine if two detected corner points simultaneously enter the target frame of the line at the parking space;

[0093] If two detected corner points are simultaneously within the target frame of the parking space entry line, then the two detected corner points are regarded as a parking space entry line corner point pair.

[0094] If a detected corner point exists within the target frame of the parking space entry line, then based on the position information of the detected corner point, the position information of the target frame of the parking space entry line, and the preset position transformation relationship, the position information of another corner point within the target frame of the parking space entry line is obtained, and the angle information of the other corner point is determined as the angle information of the detected corner point. The detected corner point and the other corner point are then used as a parking space entry line corner point pair.

[0095] The method also includes using the position and angle information of another corner point as an element in the corner point information array.

[0096] Specifically, based on the corner position of each detected corner, the system iterates through the parking space entry line target boxes (the parking space entry line target boxes retained after non-maximum suppression NMS filtering) to determine whether two detected corners are simultaneously within the same parking space entry line target box.

[0097] If two detected corner points are simultaneously within the target frame of the parking space entry line, then the two corner points A and B within the same target frame are considered as a parking space entry line corner point pair. Then, based on the angle information attached to the target frame, the parking space entry line corner point pair is determined. As an example, the connection order of corner points A and B can form a vector. sum vector The angle information attached to the target box closer to the parking space entry line is the correct connection direction. The numbering and pointing information of the two corner points aligned with the parking space entry line corner points in the corner point information array are used as an element in the corner point matching information array. Implementably, slots can be used to represent the corner point matching information array, recording the corner point matching information (recording the numbering information of the corner points in the corner point information array marks that can form a complete parking space entry line). For example, the corner point matching information array slots[j] = [1, 5, 0] indicates that the 1st and 5th detected corner points in the corner point information array marks can form a parking space entry line, and the pointing information is from 1 to 5. A 0 in the third bit of the corner point matching information array slots[j] indicates that the parking space is idle.

[0098] in, Figure 8(a)-8(e) The arrows in the diagram indicate the connection order of the corner points of the parking space entrance line. The arrows in 8(a)-8(e) represent the order in which the corner points are connected. 8(a)-8(e) only take perpendicular parking spaces as an example, and other types of parking spaces can be deduced by analogy.

[0099] If a detected corner point exists within the target bounding box of the parking space entry line, it indicates that one corner point of the parking space entry line may have failed to be detected due to occlusion or wear. There is a fixed positional transformation relationship between the setting of the parking space entry line target bounding box and the position of the parking space entry line corner point. When the target detection model can only detect one parking space entry line corner point target bounding box and one parking space entry line target bounding box, the positional information of the other corner point within the target bounding box can be inferred based on the positional information of the detected corner point, the positional information of the parking space entry line target bounding box, and the preset positional transformation relationship, thus eliminating missed detections due to occlusion or the target detection model. After determining that the angle information of the other corner point is the same as that of the detected corner point, the detected corner point and the other corner point can be combined to form a parking space entry line corner point pair.

[0100] As a concrete example, as follows: Figure 9 Therefore, when the target box of the parking space entry line is obtained ( Figure 9When the gray box is in the middle, if either corner point A or B is detected, we can directly infer the position information of the other corner point based on the fixed positional relationship between corner point A and corner point B and the target box of the parking space entry line. Then, by using the parallel relationship of the parking space line, we can directly assign the angle of the detected corner point to the angle information of the inferred corner point.

[0101] If there is a detected corner point within the target frame of the parking space entry line, the parking space detection method in this embodiment of the invention can also use the position information and angle information of another corner point as an element in the corner point information array.

[0102] In one embodiment of the present invention, the corner matching information array further includes parking space occupancy information of the corresponding parking space. The occupancy information is obtained based on the parking space occupancy information attached to the target box of the corresponding parking space entry line. The parking space detection method further includes: obtaining the occupancy status of the corresponding parking space based on the occupancy information.

[0103] in, Figures 10(a)-10(c) The system uses YES and NO to indicate the occupancy status of the corresponding parking spaces. YES indicates that the parking space is occupied, and NO indicates that the parking space is not occupied.

[0104] Specifically, the occupancy information of the parking space can be assigned to the corner point pair of the entrance line to mark the occupancy information of the parking space.

[0105] S32, determine the parking space based on the corner information array and the corner matching information array.

[0106] In one embodiment of the present invention, such as Figure 11 As shown, determining parking spaces based on the corner information array and the corner matching information array can include:

[0107] S321, For each element in the corner matching information array, determine the parking space type of the corresponding parking space based on the angle information and pointing information of the two corner points of the parking space entrance line corner point in that element;

[0108] S322, based on the parking space type and the position and angle information of the two corner points aligned with the corner point of the parking space entrance line, obtain the position information of the corresponding internal corner point;

[0109] S323, determine the parking space based on the position information of the two corner points aligned with the corner point of the parking space entrance line and the position information of the corresponding internal corner points of the parking space.

[0110] Since the corner information inside a parking space is generally not visible in the environmental image (the top view of the vehicle surround view), in order to infer a complete parking space, the length and width requirements of a standard parking space can be used to infer the complete parking space.

[0111] It should be noted that parking space types include perpendicular parking spaces, horizontal parking spaces, and angled parking spaces. The specific forms of perpendicular parking spaces are shown in Figure 12(a), horizontal parking spaces in Figure 12(b), and angled parking spaces in Figure 12(c). The standard length of a perpendicular parking space is 5.3 meters and the width is 2.4 meters; the standard length of a horizontal parking space is 6.0 meters and the width is 2.4 meters; and the standard length of an angled parking space is 6.0 meters and the width is 2.8 meters. Figure 13(a) shows the perpendicular relationship between the vehicle body and the parking space lines, with the front of the vehicle facing the parking space entrance line. Figure 13(b) shows the perpendicular relationship between the vehicle body and the parking space lines, with the front of the vehicle facing perpendicular to the parking space entrance line. Figure 13(c) shows a non-perpendicular relationship between the vehicle body and the parking space lines.

[0112] Therefore, setting Nearby floating angles are all considered right angles, with δ being a preset floating value. A parking space is defined as horizontal when the entry line length s > L = 4.5m or more; the standard length L of the edge line where the corner point of a vertical parking space is located is set. vertical = 5.3m, the standard length L of the edge line where the corner point of the horizontal parking space is located. horizontal = 2.4m, the standard length L of the edge line where the corner point of the angled parking space is located. oblique =6.0m.

[0113] When determining the parking space type, a slot and its two corresponding marks define three angular pieces of information: two are the angles associated with the marks, and the third is the connection order of the marks specified by the slot. From this data, three lines can be determined. (See...) Figure 14 Given two white lines and one black line in the diagram, calculate θ; determine if θ is within the range of the given lines. Within the range, no: angled parking space, yes: calculate the length s between the two corner points of the parking space entrance line, s>L: horizontal parking space, otherwise: vertical parking space.

[0114] When reasoning about the internal corner points of parking spaces, the position information of the internal corner points of the parking space corresponding to the mark is calculated based on the standard length information of the three types of parking spaces and the corresponding angle information of mark = [x, y, theta].

[0115]

[0116]

[0117] The value of L is determined by the parking space type determined in the first step.

[0118] The two markers identified the two interior corner points. At this point, the information for all four corner points of a parking space was completely determined. See the result below. Figure 15 .

[0119] The parking space detection method of this invention, firstly, compared to traditional algorithms, does not require multi-step image processing techniques such as binarization, corner extraction, and feature matching, effectively avoiding the influence of noise such as lighting and occlusion in traditional algorithms, ensuring algorithm stability. Moreover, it directly analyzes based on a top-down view, and can quickly convert image coordinates to world coordinates using the physical distance per unit pixel. Secondly, the network structure design of the target detection model can realize parking space corner detection, corner matching, and parking space occupancy information determination in one step, and can quickly determine the parking space type based on simple angle and length information. Thirdly, the parking space detection method of this invention has predictive reasoning capabilities. When one corner of the parking space entry line fails to be detected due to damage or slight occlusion, it can infer the complete coordinates and angle of the parking space entry line corner based on the information of the target box of the parking space entry line and the information of another known corner. Finally, the parking space detection method of this invention is highly adaptable and can be applied to the detection of various scenarios and various parking space types, and can be applied to parking spaces of any color and angle.

[0120] The parking space detection method of this invention utilizes a pre-trained target detection model to directly output the target bounding boxes of the parking space entry line and the corner points of the parking space entrance line in the environmental image. This reduces the impact of the environment on parking space detection and improves detection efficiency. The parking space is then determined based on these target bounding boxes, making the method applicable to various scenarios and parking space types, capable of detecting parking spaces of any color and angle.

[0121] The present invention also provides a computer-readable storage medium.

[0122] In this embodiment, a computer program is stored on a computer-readable storage medium, which, when executed by a processor, implements the parking space detection method described above.

[0123] The present invention also provides a parking space detection device.

[0124] Figure 16 This is a structural block diagram of the controller according to an embodiment of the present invention. Figure 16 As shown, the parking space detection device 500 includes a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, for example, via a bus 502. Optionally, the controller 500 may also include a transceiver 504. It should be noted that in practical applications, the transceiver 504 is not limited to one, and the structure of the controller 500 does not constitute a limitation on the embodiments of the present invention.

[0125] Processor 501 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 501 may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0126] Bus 502 may include a pathway for transmitting information between the aforementioned components. Bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 502 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0127] The memory 503 stores a computer program corresponding to the parking space detection method of the above embodiments of the present invention. This computer program is controlled and executed by the processor 501. The processor 501 executes the computer program stored in the memory 503 to implement the content shown in the aforementioned method embodiments. Figure 16 The controller 500 shown is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.

[0128] The present invention also provides a vehicle.

[0129] In this embodiment, such as Figure 17 As shown, vehicle 600 may include parking space detection equipment 500 as described above.

[0130] The computer-readable storage medium, parking space detection device, and vehicle in the embodiments of the present invention utilize the above-described parking space detection method to detect parking spaces. This method has high detection efficiency and is applicable to various scenarios and types of parking spaces, and can be applied to parking spaces of any color and angle.

[0131] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0132] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0133] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0134] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0135] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0136] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0137] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0138] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A parking space detection method, characterized in that, The method includes: Acquire images of the environment surrounding the vehicle; The environmental image is input into a pre-trained target detection model, which outputs the target bounding box of the parking space entry line and the target bounding box of the corner point of the parking space entrance line. The parking space is determined based on the target frame of the parking space entry line and the target frame of the corner point of the parking space entrance line. The training process of the target detection model includes: Acquire training data, wherein the training data includes an in-vehicle surround view top view and real target box information in a parking lot environment, wherein the real target box information includes the real target box category, the target box position information, the cosine and sine values ​​of the target box's attached angle, and the parking space occupancy information, and the target box category includes the parking space entry line target box and the parking space entrance line corner point target box; A target detection model is constructed, and the training data is input into the target detection model to output target box prediction information. The target box prediction information includes the predicted target box location information, the probability of the target box appearing, the probability that the target box belongs to the corner point of the parking space entrance line and the probability that it belongs to the parking space entry line target box, the cosine and sine values ​​of the angle attached to the target box, and the parking space occupancy information attached to the target box. The target detection model adopts a YOLO network structure, and the sigmoid function is used to activate the target box location information, the probability of the target box appearing, the probability that the target box belongs to the corner point of the parking space entrance line and the probability that it belongs to the parking space entry line target box, and the parking space occupancy information. The tanh activation function is used for the cosine and sine values ​​of the angle attached to the target box. An overall loss function is constructed based on the real information and the predicted information, and the parameters of the target detection model are adjusted using the overall loss function.

2. The parking space detection method according to claim 1, characterized in that, The step of determining the parking space based on the target frame of the parking space entry line and the target frame of the corner point of the parking space entrance line includes: Based on the target box of the parking space entry line and the target box of the corner point of the parking space entrance line, corner point information of the parking space entrance line is extracted to obtain a corner point information array and a corner point matching information array. The corner point information array includes the position information and angle information of each corner point, and the corner point matching information array includes the number information and pointing information of the corner points that constitute the complete parking space entrance line in the corner point information array. Parking spaces are determined based on the corner information array and the corner matching information array.

3. The parking space detection method according to claim 2, characterized in that, The step of extracting corner point information of the parking space entrance line based on the target bounding box of the parking space entry line and the target bounding box of the corner point of the parking space entrance line, to obtain a corner point information array and a corner point matching information array, includes: The corner point position is obtained based on the target box of the corner point of the parking space entrance line, and the corner point corresponding to the corner point position is recorded as the detected corner point; The corner position of each detected corner point and the angle information attached to the corresponding parking space entrance line corner point target box are used as an element of the corner information array; For each parking space entry line target frame, the parking space entrance line corner point pair within the target frame is determined based on the detected corner points, and the pointing information of the two corner points in the parking space entrance line corner point pair is determined based on the angle information attached to the target frame. The numbering information and pointing information of the two corner points in the corner point information array are used as an element in the corner point matching information array.

4. The parking space detection method according to claim 3, characterized in that, The step of determining the parking space entrance line corner point pair within the target frame of the parking space entrance line based on the detected corner points includes: Determine if two detected corner points simultaneously enter the target frame of the line at the parking space; If two detected corner points are simultaneously within the target frame of the parking space entry line, then the two detected corner points are regarded as the parking space entrance line corner point pair; If a detected corner point exists within the target frame of the parking space entry line, then based on the position information of the detected corner point, the position information of the target frame of the parking space entry line, and a preset position transformation relationship, the position information of another corner point within the target frame of the parking space entry line is obtained, and the angle information of the other corner point is determined as the angle information of the detected corner point. The detected corner point and the other corner point are then used as the corner point pair of the parking space entry line. The method further includes: using the position information and angle information of the other corner point as an element in the corner point information array.

5. The parking space detection method according to claim 3, characterized in that, The step of determining the parking space based on the corner information array and the corner matching information array includes: For each element in the corner matching information array, the parking space type of the corresponding parking space is determined based on the angle information and pointing information of the two corner points of the parking space entrance line corner point alignment in that element; Based on the parking space type and the position and angle information of the two corner points aligned with the corner point of the parking space entrance line, the position information of the corresponding internal corner point is obtained; The parking space is determined based on the position information of the two corner points aligned with the corner point of the parking space entrance line and the position information of the corresponding internal corner points of the parking space.

6. The parking space detection method according to claim 2, characterized in that, The corner matching information array also includes parking space occupancy information for the corresponding parking space. This occupancy information is obtained based on the parking space occupancy information attached to the target bounding box of the corresponding parking space entry line. The method further includes: Based on the occupancy information, the occupancy status of the corresponding parking spaces can be obtained.

7. The parking space detection method according to claim 1, characterized in that, The overall loss function is as follows: Wherein, loss is the overall loss function. , , , These are the target bounding box loss function, confidence loss function, category loss function, angle loss function, and occupancy information loss function, respectively. , , , , Loss functions , , , , The weight, Using the crossover ratio (EIOU) loss function, , Using the cross-entropy loss function, , , For the cosine and sine predicted values ​​of the angle, Let L1 be the true cosine and true sine values ​​of the angle, and L1() be the L1 norm. , , where are the predicted and actual values ​​of the occupied information, respectively, and m is the number of samples.

8. The parking space detection method according to claim 1, characterized in that, Before determining the parking space based on the target bounding box of the parking space entry line and the target bounding box of the corner point of the parking space entrance line, the method further includes: Non-maximum suppression (NMS) processing is applied to the target bounding boxes of the parking space entry line and the corner points of the parking space entrance line.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the parking space detection method as described in any one of claims 1-8.

10. A parking space detection device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the computer program is executed by the processor, it implements the parking space detection method as described in any one of claims 1-8.

11. A vehicle, characterized in that, Includes the parking space detection equipment as described in claim 10.

Images