Parking space detection method and device
By using a multi-stage model to detect parking space areas and line directions, the problem of missed detection and false detection in existing parking space detection algorithms is solved, and efficient parking space detection is achieved in complex environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG GEELY HLDG GRP CO LTD
- Filing Date
- 2023-05-25
- Publication Date
- 2026-04-17
AI Technical Summary
Existing parking space detection algorithms are easily affected by lighting conditions, changes in the shape of parking lines, and obstacles, leading to parking space detection failures. Furthermore, deep learning-based methods suffer from both missed detections and false detections.
A multi-stage model is adopted, including a pre-defined parking space area information detection model, a parking space line segmentation model, and a parking space direction detection model. By acquiring environmental images, labeling the training dataset, constructing an initial model, and training it using CenterNet, UNet, and RoI Transformer, a pre-defined model is obtained, which detects parking space corner points, line directions, and area information, reducing missed detections and false detections.
It improves the accuracy and robustness of parking space detection, reduces missed detections and false detections, and can effectively detect parking space information in ambiguous scenarios.
Smart Images
Figure CN116580385B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent driving technology, and more particularly to a parking space detection method and device. Background Technology
[0002] Currently, mainstream parking space detection algorithms are mainly divided into traditional image processing algorithms and deep learning-based algorithms. Traditional image processing-based algorithms primarily rely on the geometric rules of parking space edges and corners to detect parking spaces. However, these algorithms are easily affected by lighting conditions, changes in parking line shape, and obstacles, leading to detection failures. Deep learning-based parking line detection algorithms mainly focus on parking line segmentation and corner detection, or on parking line direction and corner regression. Post-processing is then used to obtain information such as corner locations, direction, and parking availability to complete the parking space detection.
[0003] Obtaining parking space corner points through direct regression and detecting them through heatmaps can lead to numerous missed and false detections in both methods. Furthermore, the accuracy of parking line directions obtained using direct regression is not high, and the method of obtaining parking line directions through post-processing of parking space direction heatmaps is logically complex and time-consuming. Summary of the Invention
[0004] This application provides a parking space detection method and device, which can reduce the occurrence of false detection and missed detection of parking space corners and parking spaces.
[0005] This application provides a parking space detection method, including: acquiring an environmental image of a target vehicle at the current moment; wherein the target vehicle is an autonomous vehicle driving in a parking lot;
[0006] Based on the current environmental image and the preset parking space area information detection model, parking space area information for multiple parking spaces is obtained; each parking space area information includes parking space corner area information, parking space entrance area information, and overall parking space area information.
[0007] A parking line segmentation image is obtained based on the current environmental image and the preset parking line segmentation model.
[0008] Multiple parking space line direction information is obtained based on the parking space line segmentation image and the preset parking space direction detection model; each parking space line direction information includes the smallest bounding rectangle surrounding the parking space segmentation line;
[0009] Based on the information of multiple parking space areas and multiple parking space line directions, the detection results of N parking spaces at the current moment are obtained.
[0010] In one exemplary embodiment, the parking space corner area information includes the parking space corner coordinates or a rectangular frame surrounding the parking space corner.
[0011] The parking space entrance area information includes a rectangular frame surrounding the corner points of the first number of parking space entrances;
[0012] The overall parking space area information includes the overall rectangular frame surrounding the second number of parking spaces, the parking space type, and parking availability.
[0013] In one exemplary embodiment, before obtaining parking space area information for multiple parking spaces based on the environmental image and the preset parking space area information detection model, the process includes establishing the preset parking space area information detection model.
[0014] The establishment of the preset parking space area information detection model includes:
[0015] Obtain an environmental image of the parking lot, and annotate the parking space corner area information, parking space entrance area information, and overall parking space area information on the environmental image to obtain a parking space area information training dataset.
[0016] Construct an initial parking space area information detection model;
[0017] The initial parking space area information detection model is trained based on the parking space area information training dataset to obtain the preset parking space area information detection model.
[0018] In one exemplary embodiment, the initial parking space area information detection model includes the target detection model CenterNet.
[0019] In one exemplary embodiment, before obtaining the parking line segmentation image based on the current environmental image and the preset parking line segmentation model, the preset parking line segmentation model is established.
[0020] The establishment of the preset parking space line segmentation model includes:
[0021] Obtain an environmental image of the parking lot, and label the parking space lines with segmentation labels to obtain a parking space line training dataset.
[0022] Construct an initial parking space line segmentation model;
[0023] The initial parking line segmentation model is trained based on the parking line training dataset to obtain the preset parking line segmentation model.
[0024] In one exemplary embodiment, the initial parking space area information detection model includes a neural network semantic segmentation model, UNet.
[0025] In one exemplary embodiment, before obtaining multiple parking space line direction information based on the parking space line segmentation image and the preset parking space direction detection model, the preset parking space direction detection model is established.
[0026] The establishment of the preset parking space direction detection model includes:
[0027] Obtain the parking space line segmentation image, and mark the minimum bounding rectangle of each parking space segmentation line on the parking space line segmentation image to obtain the parking space direction training dataset.
[0028] Construct an initial parking space orientation detection model;
[0029] The initial parking space direction detection model is trained based on the parking space direction training dataset to obtain the preset parking space direction detection model.
[0030] In one exemplary embodiment, the initial parking space orientation detection model includes a neural network-based rotating target detection model, RoI Transformer.
[0031] In one exemplary embodiment, obtaining multiple parking space detection results at the current moment based on the multiple parking space area information and multiple parking space line direction information includes:
[0032] The direction of each parking space line is obtained based on the information of each parking space area and the direction of each parking space line.
[0033] In one exemplary embodiment, the direction of each parking space line is obtained based on the information about each parking space area and the direction information of each parking space line; including:
[0034] Determine whether the parking space area information has a parking space entrance corner point and a parking space rear corner point; if so, take the direction of the symmetrical center line from the parking space entrance corner point to the parking space rear corner point along the smallest bounding rectangle surrounding each parking space dividing line as the direction of the parking space line corresponding to that dividing line.
[0035] In one exemplary embodiment, the direction of each parking space line is obtained based on the information about each parking space area and the direction information of each parking space line; including:
[0036] Determine whether the parking space area information has an entrance corner point and a rear corner point; if it does not have a rear corner point, calculate the coordinates of the rear corner point of the parking space based on the coordinates of the entrance corner point, the direction of the parking line, and the preset parking space length; take the direction of the symmetrical center line from the entrance corner point to the rear corner point of the parking space along the smallest bounding rectangle of each parking space dividing line as the direction of the parking line corresponding to that dividing line.
[0037] In one exemplary embodiment, the parking space dividing line of each parking space includes a first parking space dividing line and a second parking space dividing line; wherein, the length of the first parking space dividing line is greater than the length of the second parking space dividing line;
[0038] The direction of each parking space line is obtained based on the information of each parking space area and the direction of each parking space line; including:
[0039] Determine whether the first parking space dividing line and the second parking space dividing line are located within the overall rectangular frame of the parking space; if so, take the direction of the overall rectangular frame of the parking space that matches the first parking space dividing line as the direction of the parking space line corresponding to that dividing line.
[0040] In one exemplary embodiment, the parking space detection result includes the coordinates of the parking space corner point.
[0041] In one exemplary embodiment, the parking space detection result also includes parking space type and / or parking space availability.
[0042] In one exemplary embodiment, it further includes:
[0043] The detection results of N parking spaces at the current time are tracked and processed to obtain the M parking space results of the target vehicle at t times adjacent to the current time, where t is an integer greater than or equal to 1, and N and M are integers greater than or equal to 1.
[0044] This application provides a parking space detection device, including a memory and a processor.
[0045] The memory is used to store the program for parking space detection;
[0046] The processor is used to read and execute the program for parking space detection and to execute the above-described parking space detection method.
[0047] This application has the following advantages:
[0048] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the application. Other advantages of this application can be realized and obtained by means of the solutions described in the description and the accompanying drawings. Attached Figure Description
[0049] The accompanying drawings are used to provide an understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.
[0050] Figure 1 This is a flowchart of a parking space detection method according to an embodiment of this application;
[0051] Figure 2 This is a parking space diagram according to an embodiment of this application;
[0052] Figure 3This is a flowchart illustrating the process of establishing a parking space area information detection model according to an embodiment of this application;
[0053] Figure 4a This is an environmental image of an embodiment of this application;
[0054] Figure 4b for Figure 4a The corresponding semantic segmentation results for parking space lines;
[0055] Figure 5 This is a flowchart illustrating the process of establishing a parking space direction detection model according to an embodiment of this application;
[0056] Figure 6 This is a flowchart illustrating the process of obtaining parking space detection results according to an embodiment of this application;
[0057] Figure 7 This is a schematic diagram of a parking space detection device according to an embodiment of this application. Detailed Implementation
[0058] Figure 1 This is a flowchart of the parking space detection method according to an embodiment of this application, such as... Figure 1 As shown, the parking space detection method of this embodiment includes steps S11-S15:
[0059] S11. Obtain the environmental image of the target vehicle at the current moment;
[0060] S12. Obtain parking space area information for multiple parking spaces based on the current environmental image and the preset parking space area information detection model;
[0061] S13. Obtain a parking line segmentation image based on the current environmental image and the preset parking line segmentation model;
[0062] S14. Obtain multiple parking space line direction information based on the parking space line segmentation image and the preset parking space direction detection model;
[0063] S15. Based on the multiple parking space area information and multiple parking space line direction information, obtain the N parking space detection results at the current time.
[0064] The target vehicle is an autonomous vehicle driving in a parking lot.
[0065] The information for each parking space area includes information on the corner area of the parking space, information on the entrance area of the parking space, and information on the overall area of the parking space.
[0066] The direction information for each parking space line includes the smallest bounding rectangle surrounding the parking space dividing line.
[0067] In one exemplary embodiment, the environmental image may be a panoramic bird's-eye view (BEV).
[0068] A panoramic bird's-eye view can be obtained by stitching together fisheye images captured by at least four fisheye cameras using an image stitching algorithm and then removing distortion. Alternatively, it can be obtained by stitching together images captured by other types of cameras using the same image stitching algorithm.
[0069] In one exemplary embodiment, the parking space corner area information may include the coordinates of the parking space corner or a rectangular frame surrounding the parking space corner;
[0070] The parking space entrance area information may include a parking space entrance rectangle that surrounds the first number of entrance corners of the parking space.
[0071] The overall parking space area information may include the overall rectangular frame surrounding the second number of parking spaces, the parking space type, and the parking space availability.
[0072] The corner coordinates of a parking space can include the corner coordinates of the parking space entrance and the corner coordinates of the parking space rear.
[0073] In the corner region results output by the preset parking space area information detection model, if the tail corner is visible, it will be output. If the tail corner is not visible, the preset parking space area information detection model will not output the tail corner, because the tail corner may be outside the image or occluded.
[0074] In one exemplary embodiment, the first quantity can be 2 and the second quantity can be 1.
[0075] In one exemplary embodiment, the parking space type can be a perpendicular parking space, a parallel parking space, or a tilted parking space.
[0076] In one exemplary embodiment, parking availability may include parking space availability and parking space non-parking.
[0077] by Figure 2 For example, Figure 2 In the text, the parking space corner area information includes the rectangular box surrounding the parking space corner (e.g., ...). Figure 2 The coordinates of the center point of the parking space corner rectangle can be used as the coordinates of the parking space corner point. For example, if the coordinates of the top left corner of the parking space corner rectangle are (x1, y1) and the coordinates of the bottom right corner are (x2, y2), then the coordinates of the center point of the corner rectangle are ((x1+x2) / 2, y1+y2) / 2).
[0078] The parking space entrance area information includes the rectangular bounding box surrounding the two corner points of the parking space. The overall parking space area information is the rectangular bounding box surrounding the four corner points of the parking space. For scenarios where corner points are occluded, data augmentation techniques can be used to allow the model to learn the corner coordinates of that scenario. If the entrance corner point is not detected, parking space output is not supported. If the rear corner point is not detected, it can be inferred from the fixed length and direction of the parking space.
[0079] In one exemplary embodiment, before obtaining parking space area information for multiple parking spaces based on the environmental image and the preset parking space area information detection model, the process includes establishing the preset parking space area information detection model.
[0080] The establishment of the preset parking space area information detection model includes:
[0081] Obtain an environmental image of the parking lot, and annotate the parking space corner area information, parking space entrance area information, and overall parking space area information on the environmental image to obtain a parking space area information training dataset.
[0082] Construct an initial parking space area information detection model;
[0083] The initial parking space area information detection model is trained based on the parking space area information training dataset to obtain the preset parking space area information detection model.
[0084] In one exemplary embodiment, the initial parking space area information detection model includes the target detection model CenterNet.
[0085] by Figure 3 For example, the steps to establish the preset parking space area information detection model are as follows:
[0086] S31: Collect a large number of fisheye images and obtain BEV (Balance of Vehicle) images from them. Mark the corner rectangles of parking spaces, the entrance rectangles of parking spaces, and the overall rectangles of parking spaces on the BEV images to obtain a training dataset of parking space area information;
[0087] S32: Construct the object detection model CenterNet;
[0088] S33: Train the target detection model CenterNet based on the parking space area information training dataset to obtain the preset parking space area information detection model.
[0089] In one exemplary embodiment, before obtaining the parking line segmentation image based on the current environmental image and the preset parking line segmentation model, the preset parking line segmentation model is established.
[0090] The establishment of the preset parking space line segmentation model includes:
[0091] Obtain an environmental image of the parking lot, and label the parking space lines with segmentation labels to obtain a parking space line training dataset.
[0092] Construct an initial parking space line segmentation model;
[0093] The initial parking line segmentation model is trained based on the parking line training dataset to obtain the preset parking line segmentation model.
[0094] The parking space line segmentation image is a binary image, such as... Figure 4b (for Figure 4a The corresponding parking space line segmentation result is shown in the figure. The white area is the parking space line and the black area is the background.
[0095] In one exemplary embodiment, the initial parking space area information detection model includes a neural network semantic segmentation model, UNet.
[0096] The steps for creating the preset parking space line segmentation image are as follows:
[0097] Acquire an environmental image, and label the parking line segmentation on the environmental image to obtain a parking line segmentation training dataset;
[0098] Construct a parking space line segmentation model;
[0099] A parking line segmentation model is trained based on the parking line segmentation training dataset to obtain a preset parking line segmentation model.
[0100] The parking space line segmentation model includes, but is not limited to, a semantic segmentation model based on neural networks.
[0101] by Figure 5 For example, the specific implementation steps are as follows:
[0102] S41: Collect a large number of BEV images containing parking spaces, and label the BEV images with parking space line segmentation labels to obtain a parking space line segmentation training dataset.
[0103] S42: Construct a neural network semantic segmentation model UNet;
[0104] S43: Train the neural network semantic segmentation model UNet based on the parking line segmentation training dataset to obtain the preset parking line segmentation model.
[0105] In one exemplary embodiment, before obtaining multiple parking space line direction information based on the parking space line segmentation image and the preset parking space direction detection model, the preset parking space direction detection model is established.
[0106] The establishment of the preset parking space direction detection model includes:
[0107] Obtain the parking space line segmentation image, and mark the minimum bounding rectangle of each parking space segmentation line on the parking space line segmentation image to obtain the parking space direction training dataset.
[0108] Construct an initial parking space orientation detection model;
[0109] The initial parking space direction detection model is trained based on the parking space direction training dataset to obtain the preset parking space direction detection model.
[0110] The direction of the parking space line includes, but is not limited to, the direction of the symmetrical center line of the smallest circumscribed rectangle surrounding the parking space dividing line, from the corner of the parking space entrance to the corner of the parking space tail.
[0111] In one exemplary embodiment, the initial parking space orientation detection model includes a neural network-based rotating target detection model, RoI Transformer.
[0112] by Figure 6 For example, the specific steps of the parking space direction detection model are as follows:
[0113] S51: Obtain the parking space line segmentation image, and mark the minimum bounding rectangle of each parking space segmentation line on the parking space line segmentation image to obtain the parking space direction training dataset;
[0114] S52: Construct a parking space orientation detection model, namely the RoITransformer rotating target detection model based on neural networks;
[0115] S53: Train the RoI Transformer model using the parking space orientation training dataset to obtain the final parking space orientation detection model.
[0116] In one exemplary embodiment, obtaining multiple parking space detection results at the current moment based on the multiple parking space area information and multiple parking space line direction information includes:
[0117] The direction of each parking space line is obtained based on the information of each parking space area and the direction of each parking space line.
[0118] In one exemplary embodiment, the direction of each parking space line is obtained based on the information about each parking space area and the direction information of each parking space line; including:
[0119] Determine whether the parking space area information has a parking space entrance corner point and a parking space rear corner point; if so, take the direction of the symmetrical center line from the parking space entrance corner point to the parking space rear corner point along the smallest bounding rectangle surrounding each parking space dividing line as the direction of the parking space line corresponding to that dividing line.
[0120] For example, based on the direction information and parking area information of each parking space line in N parking spaces, the direction of each parking space line is obtained. Specifically: based on the direction information of each parking space line in N parking spaces, i.e., the smallest bounding rectangle surrounding the parking space dividing line; the parking space corner point area information, i.e., the center coordinates of the parking space entrance corner point rectangle; and along the smallest bounding rectangle of the dividing line, the direction of the symmetrical center line from the parking space entrance corner point to the parking space rear corner point is obtained, thus obtaining the direction of each parking space line.
[0121] In one exemplary embodiment, the direction of each parking space line is obtained based on the information about each parking space area and the direction information of each parking space line; including:
[0122] Determine whether the parking space area information has an entrance corner point and a rear corner point; if it does not have a rear corner point, calculate the coordinates of the rear corner point of the parking space based on the coordinates of the entrance corner point, the direction of the parking line, and the preset parking space length; take the direction of the symmetrical center line from the entrance corner point to the rear corner point of the parking space along the smallest bounding rectangle of each parking space dividing line as the direction of the parking line corresponding to that dividing line.
[0123] The coordinates of the parking space corner points can be obtained from the parking space corner point frame.
[0124] The matching method includes, but is not limited to, when the parking space dividing line is within the overall rectangular frame of the parking space, using the matched parking space dividing line as the parking space line of the parking space corresponding to the overall rectangular frame of the parking space.
[0125] For example, for the parking space area information of each parking space, a matching method is used to match the direction of the parking line corresponding to each parking space. Specifically: for the parking space area information of each parking space, when the parking space dividing line is within the overall rectangle of the parking space, the longer parking space dividing line is taken as the direction of the parking line corresponding to the parking space. It should be noted that a parking space has two parking space dividing lines, one long and one short.
[0126] Compared to methods that directly regress corner coordinates, corner region detection methods are more robust in blurred scenes because they can learn more regional information.
[0127] In one exemplary embodiment, the direction of each parking space line is obtained based on the information about each parking space area and the direction information of each parking space line; including:
[0128] Determine whether the parking space dividing line is located within the overall rectangular frame of the parking space; if so, take the direction of the overall rectangular frame of the parking space that matches the parking space dividing line as the direction of the parking space line corresponding to the dividing line.
[0129] For example, when the rear corner of a parking space is obscured or extends beyond the image, the coordinates of the rear corner are calculated using the parking space area information, parking line direction, and prior parking space length for each parking space. Specifically, when the rear corner of a parking space is obscured or extends beyond the image, the coordinates of the rear corner are calculated using the parking space area information, parking line direction, and prior parking space length for each parking space.
[0130] Assuming the preset parking space length is L, the parking line direction is θ, the coordinates of the entrance corner point are (x1, y1), and the coordinates of the rear corner point of the parking space are (x2, y2), then
[0131] x2 = x1 + L*cos(θ)
[0132] y2=y1+L*sin(θ)
[0133] In one exemplary embodiment, the parking space detection result includes the coordinates of the parking space corner point.
[0134] In one exemplary embodiment, the parking space detection result also includes parking space type and / or parking space availability.
[0135] The parking space type and parking availability can be detected using a parking space area information detection model. Based on the current environmental image and the preset parking space area information detection model, multiple parking space area information is obtained. Each parking space area information includes corner area information, entrance area information, and overall area information. The entrance area information detected by the parking space area information detection model includes the parking space type, such as perpendicular, horizontal, and angled parking spaces. The overall area information includes parking availability, such as whether parking is possible or not.
[0136] In one exemplary embodiment, the method further includes tracking the N parking space detection results at the current time to obtain the M parking space detection results of the target vehicle at t times adjacent to the current time, where t is an integer greater than or equal to 1, and N and M are integers greater than or equal to 1.
[0137] In one exemplary embodiment, the N parking space detection results at the current time are tracked and processed to obtain the M parking space detection results of the target vehicle at the current time and t times adjacent to the current time, including:
[0138] Obtain the driving data of the target vehicle from the current time to the t adjacent times;
[0139] The driving data mentioned includes, but is not limited to, the target vehicle's position, speed, acceleration, heading angle, and heading angular velocity.
[0140] The N parking space detection results at the current moment are updated based on the driving data to obtain the updated N parking space detection results;
[0141] Obtain the detection results of M parking spaces for the target vehicle at the current time and t times adjacent to the current time, where M is an integer greater than or equal to 1;
[0142] If the target parking space detection result does not match any of the updated N parking space detection results, the parking space status corresponding to the target parking space detection result is set to a newly added parking space, where the target parking space detection result is any one of the M parking space detection results.
[0143] If the target parking space detection result matches one of the updated N parking space detection results, then the parking space status corresponding to the target parking space detection result is set to the original parking space, where the target parking space detection result is any one of the M parking space detection results;
[0144] In this embodiment, the specific implementation steps are as follows:
[0145] Obtain the target vehicle's position, speed, acceleration, heading angle, and heading angular velocity from the current time to the next adjacent time.
[0146] Based on the target vehicle's position, speed, acceleration, heading angle, and heading angular velocity, the Kal's slow filtering algorithm is used to update the N parking space detection results at the current moment, resulting in updated N parking space detection results. The purpose of obtaining the vehicle's information is to ensure that the parking space coordinates are mapped more accurately from the image coordinate system to the world coordinate system.
[0147] Obtain the detection results of M parking spaces for the target vehicle at the next time interval adjacent to the current time, where M is an integer greater than or equal to 1;
[0148] If the target parking space detection result does not match any of the updated N parking space detection results, the parking space status corresponding to the target parking space detection result is set to a newly added parking space, where the target parking space detection result is any one of the M parking space detection results.
[0149] If the target parking space detection result matches one of the updated N parking space detection results, then the parking space status corresponding to the target parking space detection result is set to the original parking space, where the target parking space detection result is any one of the M parking space detection results.
[0150] Tracking parking spaces ensures that their coordinates are updated in real time without causing severe fluctuations.
[0151] Figure 7 This is a schematic diagram of a parking space detection device according to an embodiment of this application, as shown below. Figure 7 As shown, the parking space detection device in this embodiment includes a memory and a processor.
[0152] The memory is used to store the program for parking space detection;
[0153] The processor is used to read and execute the program for parking space detection and to execute the above-described parking space detection method.
[0154] This application describes several embodiments, but these descriptions are exemplary and not restrictive, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.
[0155] Any feature shown and / or discussed in this application may be implemented individually or in any suitable combination.
[0156] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that it does not depend on such a specific order. Other sequences of steps are possible, as will be understood by those skilled in the art.
[0157] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.
Claims
1. A parking space detection method, characterized in that, Acquire an environmental image of the target vehicle at the current moment; wherein the target vehicle is an autonomous vehicle driving in a parking lot; and the environmental image is a panoramic bird's-eye view. Based on the current environmental image and the preset parking space area information detection model, parking space area information for multiple parking spaces is obtained; each parking space area information includes parking space corner area information, parking space entrance area information, and overall parking space area information. A parking line segmentation image is obtained based on the current environmental image and the preset parking line segmentation model. Multiple parking space line direction information is obtained based on the parking space line segmentation image and the preset parking space direction detection model; each parking space line direction information includes the smallest bounding rectangle surrounding the parking space segmentation line; the preset parking space direction detection model is a model trained based on the parking space line segmentation image labeled with the smallest bounding rectangle of each parking space segmentation line. The current time N parking space detection results are obtained based on the multiple parking space area information and multiple parking space line direction information; the parking space detection results include at least the parking space line direction; the parking space line direction is determined based on the minimum circumscribed rectangle surrounding the parking space dividing line and the parking space corner point area information.
2. The parking space detection method as described in claim 1, characterized in that, The parking space corner point area information includes the coordinates of the parking space corner point or the rectangular frame surrounding the parking space corner point; The parking space entrance area information includes a rectangular frame surrounding the corner points of the first number of parking space entrances; The overall parking space area information includes the overall rectangular frame surrounding the second number of parking spaces, the parking space type, and parking availability.
3. The parking space detection method as described in claim 2, characterized in that, Before obtaining parking space area information for multiple parking spaces based on the environmental image and the preset parking space area information detection model, the process includes establishing the preset parking space area information detection model. The establishment of the preset parking space area information detection model includes: Obtain an environmental image of the parking lot, and annotate the parking space corner area information, parking space entrance area information, and overall parking space area information on the environmental image to obtain a parking space area information training dataset. Construct an initial parking space area information detection model; The initial parking space area information detection model is trained based on the parking space area information training dataset to obtain the preset parking space area information detection model.
4. The parking space detection method as described in claim 3, characterized in that, The initial parking space area information detection model includes the target detection model CenterNet.
5. The parking space detection method as described in claim 1, characterized in that, Before obtaining the parking line segmentation image based on the current environmental image and the preset parking line segmentation model, the process includes establishing the preset parking line segmentation model. The establishment of the preset parking space line segmentation model includes: Obtain an environmental image of the parking lot, and label the parking space lines with segmentation labels to obtain a parking space line training dataset. Construct an initial parking space line segmentation model; The initial parking line segmentation model is trained based on the parking line training dataset to obtain the preset parking line segmentation model.
6. The parking space detection method as described in claim 3, characterized in that, The initial parking space area information detection model includes the neural network semantic segmentation model UNet.
7. The parking space detection method as described in claim 1, characterized in that, Before obtaining multiple parking space line direction information based on the parking space line segmentation image and the preset parking space direction detection model, the process includes establishing the preset parking space direction detection model. The establishment of the preset parking space direction detection model includes: Obtain the parking space line segmentation image, and mark the minimum bounding rectangle of each parking space segmentation line on the parking space line segmentation image to obtain the parking space direction training dataset. Construct an initial parking space orientation detection model; The initial parking space direction detection model is trained based on the parking space direction training dataset to obtain the preset parking space direction detection model.
8. The parking space detection method as described in claim 7, characterized in that, The initial parking space orientation detection model includes the RoI Transformer, a neural network-based rotating target detection model.
9. The parking space detection method as described in claim 1, characterized in that, The parking space corner point area information includes the coordinates of the parking space corner point; The parking space corner coordinates include the entrance corner coordinates and the rear corner coordinates; The direction of each parking space line is obtained based on the information of each parking space area and the direction of each parking space line; including: Determine whether the parking space area information includes the parking space entrance corner and the parking space rear corner; If so, the direction of the symmetrical center line from the corner of the parking space entrance to the corner of the parking space tail is taken as the direction of the parking space line corresponding to the smallest bounding rectangle surrounding each parking space dividing line.
10. The parking space detection method as described in claim 1, characterized in that, The parking space corner point area information includes the coordinates of the parking space corner point; The parking space corner coordinates include the entrance corner coordinates and the rear corner coordinates; The direction of each parking space line is obtained based on the information of each parking space area and the direction of each parking space line; including: Determine whether the parking space area information includes the parking space entrance corner and the parking space rear corner; If there is no rear corner point of the parking space, the coordinates of the rear corner point of the parking space are calculated based on the coordinates of the entrance corner point of the parking space, the angle of the minimum bounding rectangle, and the preset length of the parking space. The direction of the symmetrical center line from the entrance corner point to the rear corner point of the parking space along the minimum bounding rectangle surrounding each parking space dividing line is taken as the direction of the parking space line corresponding to that dividing line.
11. The parking space detection method as described in claim 1, characterized in that, Each parking space is divided into a first parking space dividing line and a second parking space dividing line; wherein the length of the first parking space dividing line is greater than the length of the second parking space dividing line. The direction of each parking space line is obtained based on the information of each parking space area and the direction of each parking space line; including: Determine whether the first parking space dividing line and the second parking space dividing line are located within the overall rectangular frame of the parking space; if so, take the direction of the overall rectangular frame of the parking space that matches the first parking space dividing line as the direction of the parking space line corresponding to that dividing line.
12. The parking space detection method as described in claim 1, characterized in that, The parking space detection results include the coordinates of the parking space corner points.
13. The parking space detection method as described in claim 12, characterized in that, The parking space detection results also include parking space type and / or parking space availability.
14. The parking space detection method according to claim 1, characterized by, Also includes: The detection results of N parking spaces at the current time are tracked and processed to obtain the M parking space results of the target vehicle at t times adjacent to the current time, where t is an integer greater than or equal to 1, and N and M are integers greater than or equal to 1.
15. A parking space detection device, comprising a memory and a processor, characterized in that: The memory is used to store the program for parking space detection; The processor is configured to read and execute the program for parking space detection, and to execute the method as described in any one of claims 1-14.
Citation Information
Patent Citations
Parking space detection method and device, electronic equipment and storage medium
CN114782928A
Real-time panoramic parking space detection method and device based on dual-network deep learning
CN116012817A