Parking space detection method, device, electronic device and readable storage medium

Abstract

The present application relates to the field of automotive technology and provides a parking space detection method, device, electronic device, and readable storage medium. The method comprises: obtaining a panoramic surround view monitoring (AVM) image within a first preset range of a target vehicle; obtaining a target detection result within the first preset range using a pre-selected and trained parking space detection model based on the AVM image, the target detection result including first parking space frame information of the parking space within the first preset range and grounding point position information of obstacles; for each parking space, converting the first parking space frame information corresponding to the parking space to obtain second parking space frame information, wherein at least one side of the border line corresponding to the converted second parking space frame information overlaps with the corresponding parking space line on the parking space; determining the current state of the parking space based on the second parking space frame information and the grounding point position information of the obstacle, the current state including occupied or unoccupied. The present application solves the problem of low accuracy of existing parking space detection.

Description

Technical Field

[0001] The present application relates to the field of automobile technology, and in particular to a parking space detection method, device, electronic device, and readable storage medium. Background Art

[0002] At present, smart parking functions are widely used in smart driving cars, and parking space detection technology, one of the key technologies for realizing smart parking, has also attracted widespread attention.

[0003] Existing parking space detection methods are usually based on semantic segmentation. However, the performance of parking space detection methods based on semantic segmentation is limited in dealing with complex backgrounds and occlusions, resulting in low parking space detection accuracy.

[0004] Therefore, there is an urgent need for a parking space detection method with high accuracy. Summary of the Invention

[0005] In view of this, embodiments of the present application provide a parking space detection method, device, electronic device, and readable storage medium to solve the problem of low parking space detection accuracy in the prior art.

[0006] A first aspect of an embodiment of the present application provides a parking space detection method, comprising:

[0007] Acquire an AVM image of a target vehicle within a first preset range;

[0008] Obtaining, based on the AVM image, a target detection result within a first preset range by using a pre-selected and trained parking space detection model, wherein the target detection result includes first parking space frame information of the parking spaces within the first preset range and grounding point position information of obstacles;

[0009] For each parking space, convert the first parking space frame information corresponding to the parking space to obtain second parking space frame information, wherein at least one side of the frame line corresponding to the converted second parking space frame information overlaps with the parking space line corresponding to the parking space;

[0010] The current state of the parking space is determined according to the second parking space frame information and the grounding point position information of the obstacle, where the current state includes an occupied state or an unoccupied state.

[0011] According to a second aspect of the embodiments of the present application, a parking space detection device is provided, comprising:

[0012] An acquisition module is used to acquire an AVM image of a target vehicle within a first preset range;

[0013] A prediction module is configured to obtain, based on the AVM image and a pre-trained parking space detection model, a target detection result within a first preset range, wherein the target detection result includes first parking space frame information of parking spaces within the first preset range and grounding point position information of obstacles;

[0014] a conversion module, configured to convert, for each parking space, first parking space frame information corresponding to the parking space to obtain second parking space frame information, wherein at least one side of a frame line corresponding to the converted second parking space frame information overlaps with a parking space line corresponding to the parking space;

[0015] The determination module is used to determine the current state of the parking space according to the second parking space frame information and the grounding point position information of the obstacle, where the current state includes an occupied state or an unoccupied state.

[0016] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0017] According to a fourth aspect of an embodiment of the present application, a readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above method are implemented.

[0018] Compared with the prior art, the embodiments of the present application have the following beneficial effects: by obtaining the panoramic surround view monitoring AVM image within the first preset range of the target vehicle, data support is provided for parking space detection; by obtaining the target detection result within the first preset range through the parking space detection model obtained by pre-selection training based on the AVM image, accurate parking space detection information and obstacle grounding point information are determined, and data support is provided for subsequent target vehicles to judge the parking space status, select the parking space for parking, and avoid obstacles during parking; by converting the first parking space frame information corresponding to each parking space, at least one of the detection frame border lines is obtained. The second parking space frame information whose side frame lines overlap with the parking space lines of the corresponding parking space improves the overlap between the parking space frame information and the parking space in the actual scene, provides accurate reference data for subsequent judgment of the current status of the parking space, reduces the possibility of erroneous detection results, and improves the accuracy of parking space detection; by determining the current status of the parking space based on the second parking space frame information and the grounding point position information of the obstacle, it can provide the target vehicle with the availability of parking spaces within the first preset range, provide reference data for the target vehicle to select a parking space, improve the accuracy of parking space detection, increase the reliability of subsequent vehicle parking, and solve the problem of low accuracy of current parking space detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 This is a flow chart of a parking space detection method provided in an embodiment of the present application;

[0021] Figure 2 This is a schematic diagram of an application scenario of a parking space detection method provided in an embodiment of the present application;

[0022] Figure 3 This is a schematic diagram of the training process of a parking space detection model provided in an embodiment of the present application;

[0023] Figure 4 This is a schematic diagram of a parking space detection result provided by an embodiment of the present application;

[0024] Figure 5 This is a schematic diagram of another parking space detection result provided in an embodiment of the present application;

[0025] Figure 6 This is a schematic diagram of the structure of a parking space detection model provided in an embodiment of the present application;

[0026] Figure 7 Schematic diagram of the structure of a parking space detection device provided in an embodiment of the present application;

[0027] Figure 8 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0029] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein. Furthermore, the objects distinguished by "first," "second," and the like are generally of a class, and do not limit the number of objects. For example, the first object may be one or more.

[0030] In addition, it should be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, the elements defined by the phrase "comprises..." do not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the elements.

[0031] A parking space detection method and device according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0032] Figure 1 It is a flow chart of a parking space detection method provided in an embodiment of the present application. Figure 1 The parking space detection method can be executed by the terminal device. Figure 1 As shown, the parking space detection method includes:

[0033] S101, acquiring an Around View Monitor (AVM) image within a first preset range of a target vehicle.

[0034] Specifically, the first preset range may be a circular range with a radius of 4 meters, 5 meters, or 6 meters. Figure 2 As shown in the application scenario diagram, the application scenario may include target vehicle 1, parking space 2, parking space 3, parking space 4, parking space 5, and obstacles 6 and 7. Assuming that the first preset range is a circular range with a radius of 5 meters, and the distance between parking space 5 and target vehicle 1 is 6 meters, parking space 5 is not in the current AVM image of the target vehicle and is not the parking space detection target of the target vehicle.

[0035] AVM images can connect the environments in different directions of the target vehicle, thereby clearly reflecting the two-dimensional or three-dimensional images of the target vehicle with a 360-degree perspective, expanding the vehicle's field of view, avoiding the problem of inaccurate parking space detection results caused by blind spots in monitoring, and providing accurate data support for parking space detection.

[0036] S102, obtaining a target detection result within a first preset range through a pre-selected and trained parking space detection model based on the AVM image, wherein the target detection result includes first parking space frame information of parking spaces within the first preset range and grounding point position information of obstacles.

[0037] Specifically, the training process of the parking space detection model is as follows: Figure 3 As shown, a large number of AVM images can be collected in advance, and the parking space frames and grounding points of obstacles in the AVM images can be marked using annotation tools. The AVM images and annotated data are preprocessed. The preprocessing can include data screening and format conversion. The screening data can be used to eliminate data that does not conform to practical applications and data that is too different from other data. The format conversion requires converting the format of the annotated data into the input format required by the parking space detection model, such as TXT format, Excel format, etc. The training data of the model is constructed based on the preprocessed AVM images and annotated data, as well as the AVM image conversion data of the pre-acquired public dataset. The training data is divided according to a preset division ratio to obtain the training set and validation set of the model. The constructed parking space detection model is trained and tested based on the training set data of the model. The output results of the training data are post-processed to obtain standard parking space detection results. After the model deployment and verification steps, the model loss function is calculated based on the post-processing results and the annotated data corresponding to the training data. In the process of multiple iterative optimizations, the model loss is reduced and the accuracy of the model prediction is improved.

[0038] The addition of public data sets can expand the application scenarios of parking space detection, increase test data, and thus improve the performance and generalization ability of the parking space detection model; the validation set can test the predictive ability of the parking space detection model. The preset division ratio can be 60% for the training set and 40% for the validation set, or 80% for the training set and 20% for the validation set. Cross-validation method can also be used for division. The specific division method can be set according to the actual application situation, and this application does not specifically limit this.

[0039] Specifically, the first parking space frame information may include the coordinate data of each corner point of the parking space detection frame, and the length of the parking space's entry line and side lines. The obstacle's ground contact point location information may include the coordinate data of each obstacle's ground contact point or the obstacle's ground contact area data. Due to the limited field of view of the target vehicle's camera, it may not be possible to obtain the coordinate data of all obstacle's ground contact points in the presence of obstructions. However, the obstacle's ground contact area data can be estimated based on the obstacle's outline, side length, and other factors, providing data support for the target vehicle's parking space detection results.

[0040] The types of obstacles may include vehicles, pedestrians, buildings and road facilities. Vehicles include two-wheeled vehicles, three-wheeled vehicles, four-wheeled vehicles and multi-wheeled vehicles, that is, motor vehicles and non-motor vehicles; road facilities may include cones, water barriers, guardrails, isolation belts, warning signs, etc. The types of obstacles can be added according to the actual traffic scenarios. This application does not make specific limitations here.

[0041] For example, continue to use the above example to illustrate the target detection results, assuming Figure 2 is the current scene of the target vehicle, Figure 4 The dotted box in the figure is the target detection result within the first preset range of the target vehicle, including the information corresponding to the first parking space box 401 of parking space 2, the information corresponding to the first parking space box 402 of parking space 3, the information corresponding to the first parking space box 403 of parking space 4, and the information corresponding to the touchdown point position 404 of the obstacle within the first preset range. Since the distance between target vehicle 1 and parking space 5 is greater than the radius of the first preset range, there is no first parking space box information corresponding to parking space 5 in the target detection result.

[0042] By using the parking space detection model obtained through pre-selected training based on the AVM image, the target detection results within the first preset range are obtained, and accurate parking space detection information and obstacle grounding point information can be obtained, which provides accurate data support for subsequent target vehicles to judge the parking space status, select parking spaces for parking, and avoid obstacles during parking, thereby increasing the safety and reliability of subsequent vehicle parking.

[0043] S103 , for each parking space, convert the first parking space frame information corresponding to the parking space to obtain second parking space frame information, wherein at least one side of the frame line corresponding to the converted second parking space frame information overlaps with the parking space line corresponding to the parking space.

[0044] Specifically, the second parking space frame information includes the same data content as the first parking space frame information, except that the positional relationship between the detection frame and the parking space has changed. At least one side of the border line in the parking space frame information overlaps with the parking space line corresponding to the parking space, which can improve the overlap between the parking space frame information and the parking space in the actual scene. It can not only provide people with a better visual experience after visualization, but also provide accurate reference data for subsequent judgment of the current status of the parking space, avoiding the situation where the detection result is erroneous due to the low overlap between the parking space detection frame and the parking space.

[0045] For example, continuing with the above example, Figure 4 The detection frame in is the first parking space frame information within the first preset range of the target vehicle and the grounding point position information of the obstacle. Figure 5The second parking space frame information in the first preset range includes the information corresponding to the second parking space frame 501 of parking space 2, the information corresponding to the second parking space frame 502 of parking space 3, and the information corresponding to the second parking space frame 503 of parking space 4. It can be seen that Figure 5 The parking space frame in Figure 4 The parking space frame in the image has a higher degree of overlap with the corresponding parking space, and the relationship with other detection frames and obstacles is clearer.

[0046] By converting the first parking space frame information corresponding to each parking space, a second parking space frame information is obtained in which at least one side of the detection frame line overlaps with the corresponding parking space line on the parking space. This improves the overlap between the parking space frame information and the parking space in the actual scene, can provide people with a better visual experience after visualization, provide accurate reference data for subsequent judgment of the current status of the parking space, reduce the possibility of error in the detection result, and improve the accuracy of parking space detection.

[0047] S104: Determine a current state of the parking space according to the second parking space frame information and the grounding point position information of the obstacle. The current state includes an occupied state or an unoccupied state.

[0048] Specifically, since the parking space detection frame in the second parking space frame information has a higher degree of overlap with the corresponding parking space, the entry line, side line and coordinate data of each corner point of the parking space detection frame in the second parking space frame information have a higher degree of match with the corresponding parking space. The side lines of the detection frame in the second parking space frame information and the grounding point position information of the obstacle can be used to determine whether the obstacle is located in the parking space or blocks the use of the parking space, which is used as the current status of the corresponding parking space.

[0049] like Figure 5 As shown, there is an obstacle in the parking space frame 502, and the grounding point position information of the obstacle is located within the entry line of the parking space frame 502, then the current state of the parking space corresponding to the parking space frame 502 is determined to be occupied; due to the vehicle's perspective, there is also an obstacle in the parking space frame 501, but the grounding point position of the obstacle is located outside the entry line of the parking space frame 501, then the current state of the parking space corresponding to the parking space frame 501 is determined to be unoccupied.

[0050] By determining the current status of the parking space based on the second parking space frame information and the grounding point position information of the obstacle, the detection accuracy of the current status of the parking space is improved, and the availability of parking spaces within the first preset range can be provided to the target vehicle, providing reference data for the target vehicle to select a parking space, and providing data support for avoiding obstacles during the parking process of the target vehicle, thereby improving parking safety.

[0051] According to the technical solution provided by the embodiment of the present application, data support is provided for parking space detection by obtaining a panoramic surround view monitoring AVM image within a first preset range of the target vehicle; by obtaining a target detection result within the first preset range through a parking space detection model obtained through pre-selection training based on the AVM image, accurate parking space detection information and obstacle grounding point information are determined, and data support is provided for subsequent target vehicles to judge the parking space status, select a parking space for parking, and avoid obstacles during parking; by converting the first parking space frame information corresponding to each parking space, the detection frame border line is obtained. The second parking space frame information, in which at least one side frame line overlaps with the parking space line of the corresponding parking space, improves the overlap between the parking space frame information and the parking space in the actual scene, provides accurate reference data for subsequent judgment of the current status of the parking space, reduces the possibility of erroneous detection results, and improves the accuracy of parking space detection; by determining the current status of the parking space based on the second parking space frame information and the grounding point position information of the obstacle, it can provide the target vehicle with the availability of parking spaces within the first preset range, provide reference data for the target vehicle to select a parking space, increase the reliability of subsequent vehicle parking, and solve the problem of low accuracy of current parking space detection.

[0052] In some embodiments, the parking space detection model is a fifth-generation You Only Look Once (YOLO5) model; wherein, a Squeeze-and-Excitation (SE) unit is provided between the last convolution (Conv) unit and the Spatial Pyramid Pooling (SPP) unit in the backbone network module of the YOLO5 model; the feature fusion network layer in the fusion network module of the YOLO5 model is a bidirectional feature pyramid network (BiFPN) layer; the object detection module of the YOLO5 model includes at least two object detection units of different scales;

[0053] The SE unit is used to receive the first feature vector output by the Conv unit and increase the preset weight value corresponding to the weights of the color features of the three channels of red (Red, R), green (Green, G), and blue (Blue, B) in the first feature vector;

[0054] BiFPN is used to perform bidirectional fusion of the received second eigenvector and third eigenvector;

[0055] The target detection unit is used to detect targets of different scales and output target detection results.

[0056] Specifically, see Figure 6Schematic diagram of the parking space detection model structure. The input of the parking space detection model is image data, which enters the backbone network module after data enhancement and adaptive enhancement. The backbone network module extracts features from the input image and converts the image into a multi-layer feature map. The feature maps of different scales and depths are then input into the fusion network module. The fusion network module fuses the feature maps of different levels and depths obtained after upsampling and downsampling operations, and inputs the fused feature maps of different scales into the target detection module for prediction. Finally, the parking space detection result that matches the size of the target parking space is output.

[0057] Specifically, data enhancement can adopt data enhancement operations such as Mosaic data enhancement method, HSV (Hue, Saturation, Value) transformation method, random rotation and translation method, perspective transformation method, and noise addition method to increase data diversity and tap the potential of limited data. Adaptive enhancement can obtain the absolute coordinates of the anchor frame through adaptive anchor frame calculation, and adjust the original AVM image to the preset image size through adaptive image scaling, and avoid the problem of image distortion, thereby improving the accuracy of parking space detection results.

[0058] After the backbone network module slices, splices and convolves the image through the Focus unit, it obtains a feature map by re-sampling the original image with double intervals in both the horizontal and vertical directions, which improves the receptive field while reducing the number of parameters and computational complexity. Then, each Conv unit extracts and organizes features to obtain feature maps of different scales. The Bottleneck CSP unit obtained by combining the Bottleneck network layer and the cross-stage partial connection (CSP) structure reduces the scale of the feature image, enhances the learning ability of the convolutional neural network (CNN), reduces memory consumption and computational bottlenecks, and then uses the SE unit to enhance the weight of the color features of the RGB channels in the feature map, making the model more accurate in identifying parking spaces. Finally, the SPP unit uses three different sizes of pooling kernels to perform maximum pooling on each feature map to obtain the preset feature map sizes. Finally, all feature maps are expanded into feature vectors and fused to improve the recognition ability of the model.

[0059] Specifically, due to the conscious attention operation of humans when observing things, they can quickly lock the target among many objects of different colors, shapes, and sizes, but machine learning lacks this ability. After adding the SE unit, the colors in the feature map are more vivid. Combined with the recognition ability of the model, the accuracy of parking space recognition by the parking space detection model can be improved.

[0060] Converged network modules such as Figure 6As shown in the figure, the path from the BottleneckCSP unit to the BiFPN_Concat2 unit on the left side of the fusion network module is a top-down bidirectional fusion process of deep and shallow layer features, and the path from the BottleneckCSP unit to the last BottleneckCSP unit on the right side is a bottom-up bidirectional fusion process of deep and shallow layer features. Since the Conv unit will extract features through convolution, and the BottleneckCSP unit in the backbone network module scales the image, the three feature maps input from the backbone network module to the fusion network module have different levels from shallow to deep layers and different scales from large to small from top to bottom. The path in the fusion network module also extracts features and scales the image. At the same time, the path on the left side of the fusion network module is upsampled by the UpSample unit to obtain a high-resolution feature map. Therefore, each fusion of the BiFPN layer is a process of fusing feature maps of the same scale and different feature depths.

[0061] Specifically, the second eigenvector and the third eigenvector are two eigenvectors of the same scale and different depths input into the same BiFPN layer. The second eigenvector can be set as a deep eigenvector and the third eigenvector can be set as a shallow eigenvector. The second eigenvector can also be set as a shallow eigenvector and the third eigenvector can be set as a deep eigenvector. This application does not specifically limit this. It should be noted that the second eigenvector and the third eigenvector are different for different BiFPN layers.

[0062] For example, as an example, Figure 6 In the first BiFPN layer from bottom to top on the left path in the fusion network module, the two eigenvectors input to the BiFPN layer are the second eigenvector output by the Conv unit in the backbone network module and the third eigenvector output by the UpSample unit. The third eigenvector is the second eigenvector obtained by scaling the second eigenvector with multiple BottleneckCSP units, convolution of multiple Conv units and SPP units, and upsampling of the UpSample unit. The scales of the second and third eigenvectors are the same, but compared with the second eigenvector, the third eigenvector has deep semantic features, which is equivalent to the features that are closest to human understanding that can be expressed by machine learning. However, shallow graphic features may be lost in the convolution process, such as color, contour, texture, shape, etc., which are only shallow and easy to understand features in the graphics. Therefore, after the second and third eigenvectors are fused, a feature vector with prominent deep and shallow features can be obtained, thereby improving the accuracy of target detection.

[0063] The target detection module performs target detection on the fused feature vector output by the fusion network module. It includes multiple convolutional layers, pooling layers, and fully connected layers. It can perform multi-scale target detection and output target detection results at one or more scales.

[0064] Since the target detection module may output target detection results of multiple scales, but there may be only one target detection result matching the target parking space, a filtering unit needs to be set after the target detection module. The filtering unit can use a deformation of non-maximum suppression (NMS) called efficient region intersection over union-NMS (Efficient-Intersection Over Union-NMS, EIoU-NMS) to project the distance between detection frames into the embedding space, and then calculate the distance in the embedding space instead of the traditional IOU calculation, so as to better handle the overlap between detection frames and improve the accuracy of target detection.

[0065] According to the technical solution provided in the embodiment of the present application, by setting the SE unit between the last convolution Conv unit and the SPP unit in the backbone network module of the YOLO5 model, the model can more accurately identify parking spaces. By setting the feature fusion network layer in the fusion network module to BiFPN, feature vectors with different depth features can be fully fused to obtain feature vectors with prominent deep features and shallow features, thereby improving the accuracy of parking space detection.

[0066] In some embodiments, for each parking space, converting the first parking space frame information corresponding to the parking space to obtain the second parking space frame information includes:

[0067] Establishing a first coordinate system with the center point of the parking space frame corresponding to the first parking space frame information as the center, and establishing a second coordinate system with the actual center position of the parking space as the center;

[0068] According to the mapping relationship between the first coordinate system and the second coordinate system, the coordinates of the corner points in the first parking space frame information are converted to the second coordinate system to obtain the second parking space frame information.

[0069] Specifically, the first coordinate system and the second coordinate system are coordinate systems of the same dimension.

[0070] To obtain the mapping relationship between the first coordinate system and the second coordinate system, after determining the first coordinate system and the second coordinate system, the distances between the corner points of the parking space frame and the corner points of the actual parking space, as well as the first vectors between the center point of the parking space frame and the first corner points of the parking space frame and the second vectors between the center position of the actual parking space and the second corner points of the actual parking space are calculated. The mapping relationship between the first coordinate system and the second coordinate system is determined based on the first vectors and the second vectors corresponding to the first corner points and the second corner points in the parking space frame and the actual parking space that are closest to each other.

[0071] After the first corner point and the second corner point are overlapped according to the mapping relationship, the parking space frame and the parking space line of the actual parking space are also overlapped. The obtained second parking space frame information can provide the driver with a better visual reference and accurate parking space detection information.

[0072] According to the technical solution provided in the embodiments of the present application, a first coordinate system is established with the center point of the parking space frame corresponding to the first parking space frame information as the center, and a second coordinate system is established with the actual center position of the parking space as the center; then, based on the mapping relationship between the first coordinate system and the second coordinate system, the coordinates of the corner points in the parking space frame information are converted to the second coordinate system to obtain the second parking space frame information. This can increase the overlap between the second parking space frame information and the corresponding actual parking space, making the second parking space frame information more accurate and reliable, and improving the user experience.

[0073] In some embodiments, determining the current state of the parking space based on the second parking space frame information and the grounding point location information of the obstacle includes:

[0074] Detecting whether the obstacle's grounding point is within the parking space based on the second parking space frame information and the obstacle's grounding point location information;

[0075] When the grounding point of the obstacle is within the parking space, determining that the current state of the parking space is occupied;

[0076] When the grounding point of the obstacle is not within the parking space, it is determined that the current state of the parking space is an unoccupied state.

[0077] Specifically, if Figure 5 As shown, there is an obstacle grounding point in the parking space corresponding to the parking space frame 502, and the current state of the parking space is determined to be occupied.

[0078] It should be noted that if the parking space frame information and the obstacle's grounding point location information indicate that the obstacle's grounding point is outside the parking space but on one side of the corresponding parking space's entry line, the parking space is also determined to be occupied. The entry line of the parking space is the first parking line that the vehicle crosses during parking.

[0079] For example, Figure 2As shown, parking line n1 is the entry line for parking space 4 in the parking frame. If an obstacle is outside the parking frame but on one side of parking line n1, the vehicle will still encounter an obstacle when entering the parking space, so the parking space is determined to be occupied. However, there is no obstacle contact point inside parking space 2 in the parking frame. There is an obstacle contact point outside the parking frame, but it is on the side line of the parking frame corresponding to parking space 2, not on the entry line. Therefore, the current state of the parking space is determined to be unoccupied.

[0080] In addition, the orientation and size of the corresponding parking space can be determined based on the entry line, side line, and the angle between the entry line and side line of the parking space frame, thereby providing suitable parking space selection and reference direction for vehicle parking.

[0081] According to the technical solution provided in the embodiment of the present application, by detecting whether the grounding point of the obstacle is within the parking space based on the second parking space frame information and the grounding point position information of the obstacle; by determining whether the grounding point of the obstacle is within the parking space, it is determined whether the parking space is currently occupied, thereby achieving effective utilization of resources, providing data reference for users to park, and improving the user's parking experience.

[0082] In some embodiments, for each parking space, before converting the first parking space frame information corresponding to the parking space to obtain the second parking space frame information, the method further includes:

[0083] According to the first parking space frame information, a first parking space frame corner point of a first border line corresponding to a parking space entry line in the first parking space frame is obtained, and a second parking space frame corner point opposite to the first parking space frame corner point is obtained;

[0084] If the clarity of the first parking space frame corner point is greater than a preset value, and the clarity of the second parking space frame corner point is less than a preset value, construct a second frame line adjacent to the first frame line according to the preset parking space length, and construct virtual second parking space frame corner points according to the second frame line;

[0085] The second border line and the virtual second parking space frame corner points are fused with the parking space frame information of the parking space detected in real time to obtain updated parking space frame information.

[0086] Specifically, the first border line corresponds to the entrance line of the corresponding parking space, the first parking space frame corner points are the two corner points in the parking space frame connected to the first border line, and the second parking space frame corner points are the corner points in the parking space frame other than the first parking space frame corner points. Figure 2 As shown, the entry line of parking space 4 corresponds to n1, the side line is n4, the first parking space frame corner point corresponds to p1, and the second parking space frame corner point corresponds to p2.

[0087] The preset value may be 80% or 70%. When the clarity of the corner points of a parking space is less than the preset value, parking in the parking space may result in collisions, scratches, and other accidents due to the lack of corner point reference.

[0088] The preset parking space length can refer to the entry line length of the parking space detection frame, or be set to the sum of the target vehicle body length and the preset length. The preset length can be 1 meter, 1.2 meters, etc., leaving sufficient space for the target vehicle to park in and out of the parking space to avoid collision and scratch accidents.

[0089] The second border lines may be side lines on one side or both sides of the parking space entrance line, and the number of the second border lines needs to be determined according to the number of second parking space frame corner points whose clarity is less than a preset value.

[0090] In addition, the reason why the corner points of the second parking space frame are unclear may be that they are blocked by obstacles. Therefore, even after the virtual corner points of the second parking space frame are constructed, the corresponding parking space may still be unusable. It is necessary to combine the parking space frame information obtained by real-time detection to update the constructed second border line and the virtual corner points of the second parking space frame to improve the accuracy of parking space detection and the user's parking experience.

[0091] According to the technical solution provided in the embodiment of the present application, a first parking space frame corner point of a first border line corresponding to a parking space entry line in the first parking space frame is obtained according to the first parking space frame information, and a second parking space frame corner point opposite to the first parking space frame corner point is obtained to determine whether the parking space corner point corresponding to the parking space frame is clear and available; if the clarity of the first parking space frame corner point is greater than a preset value, and the clarity of the second parking space frame corner point is less than a preset value, a second border line adjacent to the first border line is constructed according to the preset parking space length, and a virtual second parking space frame corner point is constructed according to the second border line, so as to predict whether the parking space can be parked; the second border line and the virtual second parking space frame corner point are integrated with the parking space frame information of the parking space detected in real time to obtain updated parking space frame information, thereby providing users with real-time and accurate parking space information, providing users with data reference for parking, and improving users' parking experience.

[0092] In some embodiments, obtaining an AVM image within a first preset range of a target vehicle includes:

[0093] The fisheye camera installed on the target vehicle collects original images in different directions;

[0094] Correct the original image according to the camera parameters of the fisheye camera and the preset dedistortion function;

[0095] The rectified image is subjected to inverse perspective transformation by the Direct Linear Transform (DLT) algorithm to obtain an inverse perspective mapping (IPM) image.

[0096] According to the preset region of interest (ROI) division rules, the IPM images are spliced ​​and fused to generate the AVM image.

[0097] Specifically, there can be multiple fisheye cameras. For example, when there are four fisheye cameras, they are installed in the front, rear, left and right directions of the target vehicle, respectively, so as to obtain a 360° field of view image around the target vehicle.

[0098] The camera parameters of a fisheye camera include intrinsic parameters, extrinsic parameters, and distortion coefficients. The intrinsic parameters and distortion coefficients remain unchanged and can be obtained through camera calibration. Calibration methods can use Zhang Zhengyou's calibration method or other calibration methods. In addition, when shooting with a camera, the captured image may be distorted or deformed due to lens distortion, camera calibration, and other factors. To solve this problem, we need to use image dedistortion technology. Using the camera's intrinsic parameters, we convert the pixel coordinates to the camera coordinate system, calculate the distortion amount and distortion location, and then convert the coordinates in the camera coordinate system to the pixel coordinate system to restore the image to its true plane.

[0099] In addition, due to the existence of perspective effect, actually parallel objects may intersect in the image captured by the camera. The IPM image calculated based on the IPM formula, camera parameters, coordinates in the image coordinate system, and coordinates of the vehicle center can eliminate this perspective effect and provide accurate input data for the parking space detection model.

[0100] Specifically, the preset ROI division rule may be a pre-set image ROI area, such as setting the image ROI area as an image boundary area, and the image boundary area can be quickly locked according to the image ROI area, thereby achieving the effect of splicing IPM images.

[0101] According to the technical solution provided in the embodiment of the present application, a fisheye camera installed on the target vehicle is used to collect original images in different orientations to expand the vehicle's field of view; the original image is corrected according to the camera parameters of the fisheye camera and a preset dedistortion function to obtain a true plane image; the corrected image is subjected to an inverse perspective transformation through the DLT algorithm to eliminate the perspective effect of the original image and obtain an IPM image; and then, according to the preset ROI division rule, the IPM images are spliced ​​and fused to generate an AVM image, which is used to obtain the input of the parking space detection model and provide data support for parking space detection.

[0102] In some embodiments, after determining the current state of the parking space based on the second parking space frame information and the grounding point location information of the obstacle, the method further includes:

[0103] When the obstacle ratio in the parking space is less than a preset ratio, the obstacle is identified and the movable index of the obstacle is determined;

[0104] When the movable index of the obstacle is greater than the preset index, the second parking space frame information and the obstacle information corresponding to the parking space are visualized, and the visualized image is uploaded to the cloud or sent to vehicles within the second preset range.

[0105] Specifically, the preset proportion can be 20%, 30%, etc., the preset index can be 80%, 90%, etc., and the second preset range can be a circular range with a radius of 50 meters or 80 meters. This application does not make any specific limitations on this.

[0106] like Figure 5 As shown, assuming a preset percentage of 20%, the obstacle percentage within the parking space corresponding to parking space frame 502 is 10% of the parking space. This indicates that the obstacle within the parking space is relatively small, and it is possible to manually move or clear the obstacle within the parking space, thereby enabling parking in the parking space. Therefore, by identifying the size or type of the obstacle, the mobility of the obstacle can be determined. Specifically, this embodiment can pre-set a correspondence between different obstacle types and mobility indices, and then determine whether the obstacle's mobility index is greater than a preset index based on this correspondence. Assuming that the obstacle is a water barrier or a warning sign, moving the obstacle may require a lot of manpower or affect traffic safety, so the obstacle's mobility index is determined to be greater than the preset index. Assuming that the obstacle is a pedestrian, garbage, or bicycle, it can be moved through communication or a small amount of manpower, so the obstacle's mobility index is determined to be less than the preset index.

[0107] It should be noted that the target vehicle can set whether to enable visualization of the second parking space frame information and obstacle information corresponding to the parking space when the obstacle's movable index is greater than the preset index, and upload the visualized image to the cloud or send it to vehicles within the second preset range, as well as the permission to receive images sent by other vehicles.

[0108] When the movable index of the obstacle is greater than the preset index, it means that the parking space is highly available. The second parking space frame information and obstacle information corresponding to the parking space are visualized, and the visualized image is uploaded to the cloud or sent to vehicles within the second preset range. This can provide analysis data for the cloud, provide data support for subsequent optimization model functions, or provide parking options for other surrounding vehicles. Accordingly, the target vehicle can also park according to the visualized images sent by other vehicles, thereby improving the user's parking experience.

[0109] According to the technical solution provided in the embodiment of the present application, when the obstacle ratio in the parking space is less than a preset ratio, obstacles are identified and the obstacle's movable index is determined, thereby determining whether the obstacle can be cleared and parking can be performed in the parking space. When the obstacle's movable index is greater than the preset index, the second parking space frame information and obstacle information corresponding to the parking space are visualized, and the visualized image is uploaded to the cloud or sent to vehicles within the second preset range, thereby sharing the highly available parking space, increasing interaction during parking, and improving the user's parking experience.

[0110] All of the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described in detail here.

[0111] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0112] Figure 7 Schematic diagram of a parking space detection device provided in an embodiment of the present application. Figure 7 As shown, the parking space detection device includes:

[0113] An acquisition module 701 is configured to acquire an AVM image of a target vehicle within a first preset range.

[0114] A prediction module 702 is configured to obtain, based on the AVM image and a pre-trained parking space detection model, a target detection result within a first preset range, wherein the target detection result includes first parking space frame information of parking spaces within the first preset range and grounding point position information of obstacles;

[0115] The conversion module 703 is configured to convert the first parking space frame information corresponding to each parking space to obtain second parking space frame information, wherein at least one side of the frame line corresponding to the converted second parking space frame information overlaps with the parking space line corresponding to the parking space;

[0116] The determination module 704 is configured to determine a current state of the parking space according to the second parking space frame information and the grounding point position information of the obstacle, where the current state includes an occupied state or an unoccupied state.

[0117] According to the technical solution provided in the embodiment of the present application, the acquisition module 701 provides data support for parking space detection by acquiring the panoramic surround view monitoring AVM image within the first preset range of the target vehicle; the prediction module 702 obtains the target detection result within the first preset range by pre-selecting the parking space detection model obtained through training based on the AVM image, determines the accurate parking space detection information and obstacle grounding point information, and provides data support for the subsequent target vehicle to judge the parking space status, select the parking space for parking, and avoid obstacles during parking; the conversion module 703 converts the first parking space frame information corresponding to the parking space for each parking space, and obtains at least one of the detection frame border lines. The second parking space frame information, whose border line on one side overlaps with the parking space line of the corresponding parking space, improves the overlap between the parking space frame information and the parking space in the actual scene, can provide a better visual experience for the driver of the target vehicle, and provide accurate reference data for subsequent judgment of the current status of the parking space, thereby reducing the possibility of erroneous detection results and improving the accuracy of parking space detection; the determination module 704 determines the current status of the parking space based on the second parking space frame information and the grounding point position information of the obstacle, and can provide the target vehicle with the availability of parking spaces within the first preset range, providing reference data for the target vehicle to select a parking space, thereby increasing the reliability of subsequent vehicle parking and solving the problem of low accuracy in current parking space detection.

[0118] In some embodiments, the parking space detection model is a YOLO5 model; wherein, an attention SE unit is arranged between the last convolution Conv unit and the spatial pyramid pooling SPP unit in the backbone network module of the YOLO5 model; the feature fusion network layer in the fusion network module of the YOLO5 model is a bidirectional feature pyramid network layer BiFPN; the target detection module of the YOLO5 model includes at least two target detection units of different scales; wherein, the SE unit is used to receive the first feature vector output by the Conv unit, and increase the preset weight value corresponding to the weights of the color features of the red, green and blue channels in the first feature vector; BiFPN is used to perform bidirectional fusion of the received second feature vector and the third feature vector; the target detection unit is used to detect targets of different scales and output target detection results.

[0119] In some embodiments, the conversion module 703 is specifically used to: establish a first coordinate system with the center point of the parking space frame corresponding to the first parking space frame information as the center, and establish a second coordinate system with the actual center position of the parking space as the center; according to the mapping relationship between the first coordinate system and the second coordinate system, convert the corner point coordinates in the first parking space frame information to the second coordinate system to obtain the second parking space frame information.

[0120] In some embodiments, the determination module 704 is specifically used to: detect whether the grounding point position of the obstacle is within the parking space based on the second parking space frame information and the grounding point position information of the obstacle; when the grounding point position of the obstacle is within the parking space, determine that the current state of the parking space is occupied; when the grounding point position of the obstacle is not within the parking space, determine that the current state of the parking space is unoccupied.

[0121] In some embodiments, the conversion module 703 is further used to: obtain a first parking space frame corner point of a first border line corresponding to a parking space entry line in the first parking space frame according to the first parking space frame information, and obtain a second parking space frame corner point opposite to the first parking space frame corner point; if the clarity of the first parking space frame corner point is greater than a preset value, and the clarity of the second parking space frame corner point is less than a preset value, construct a second border line adjacent to the first border line according to a preset parking space length, and construct a virtual second parking space frame corner point according to the second border line; merge the second border line and the virtual second parking space frame corner point with the parking space frame information of the parking space detected in real time to obtain updated parking space frame information.

[0122] In some embodiments, the acquisition module 701 is specifically used to: collect original images of different orientations through a fisheye camera installed on the target vehicle; correct the original image according to the camera parameters of the fisheye camera and a preset dedistortion function; perform an inverse perspective transformation on the corrected image through a direct linear transformation (DLT) algorithm to obtain an inverse perspective mapping (IPM) image; and splice and fuse the IPM images according to a preset region of interest (ROI) division rule to generate an AVM image.

[0123] In some embodiments, the determination module 704 is also used to: when the obstacle ratio of the parking space is less than a preset ratio, identify the obstacle and determine the movable index of the obstacle; when the obstacle movable index is greater than the preset index, visualize the second parking space frame information and obstacle information corresponding to the parking space, and upload the visualized image to the cloud or send it to vehicles within the second preset range.

[0124] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0125] Figure 8 Schematic diagram of the electronic device 8 provided in the embodiment of the present application. Figure 8As shown, the electronic device 8 of this embodiment includes: a processor 801, a memory 802, and a computer program 803 stored in the memory 802 and executable by the processor 801. When the processor 801 executes the computer program 803, the steps of the above-described method embodiments are implemented. Alternatively, when the processor 801 executes the computer program 803, the functions of the modules / units in the above-described device embodiments are implemented.

[0126] The electronic device 8 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 8 may include but is not limited to a processor 801 and a memory 802. Those skilled in the art will appreciate that Figure 8 This is merely an example of the electronic device 8 and does not limit the electronic device 8 . The electronic device 8 may include more or fewer components than shown in the figure, or different components.

[0127] The processor 801 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0128] The memory 802 can be an internal storage unit of the electronic device 8, such as a hard disk or memory of the electronic device 8. The memory 802 can also be an external storage device of the electronic device 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc. equipped on the electronic device 8. The memory 802 can also include both an internal storage unit of the electronic device 8 and an external storage device. The memory 802 is used to store computer programs and other programs and data required by the electronic device.

[0129] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0130] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium (such as a computer-readable storage medium). Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. Computer-readable storage media may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0131] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A parking space detection method, characterized in that: include: Acquire an AVM image of a target vehicle within a first preset range; Obtaining, based on the AVM image, a target detection result within the first preset range by using a pre-selected and trained parking space detection model, wherein the target detection result includes first parking space frame information of parking spaces within the first preset range and grounding point position information of obstacles; According to the first parking space frame information, a first parking space frame corner point of a first border line corresponding to a parking space entry line in the first parking space frame is obtained, and a second parking space frame corner point opposite to the first parking space frame corner point is obtained; if the clarity of the first parking space frame corner point is greater than a preset value, and the clarity of the second parking space frame corner point is less than a preset value, a second border line adjacent to the first border line is constructed according to a preset parking space length, and virtual second parking space frame corner points are constructed according to the second border line; the second border line and the virtual second parking space frame corner points are merged with the parking space frame information of the parking space detected in real time to obtain updated parking space frame information; For each parking space, converting the first parking space frame information corresponding to the parking space to obtain second parking space frame information, wherein at least one side of the frame line corresponding to the converted second parking space frame information overlaps with the parking space line corresponding to the parking space; The current state of the parking space is determined according to the second parking space frame information and the grounding point position information of the obstacle, where the current state includes an occupied state or an unoccupied state.

2. The method according to claim 1, characterized in that The parking space detection model is a YOLO5 model; wherein an attention SE unit is provided between the last convolution Conv unit and the spatial pyramid pooling SPP unit in the backbone network module of the YOLO5 model; the feature fusion network layer in the fusion network module of the YOLO5 model is a bidirectional feature pyramid network layer BiFPN; the object detection module of the YOLO5 model includes at least two object detection units of different scales; The SE unit is configured to receive the first feature vector output by the Conv unit and to increase the preset weight value corresponding to the weights of the color features of the red, green, and blue channels in the first feature vector; The BiFPN is used to perform bidirectional fusion of the received second eigenvector and the third eigenvector; The target detection unit is used to detect targets of different scales and output the target detection results.

3. The method according to claim 1, characterized in that For each parking space, converting the first parking space frame information corresponding to the parking space to obtain the second parking space frame information includes: Establishing a first coordinate system with the center point of the parking space frame corresponding to the first parking space frame information as the center, and establishing a second coordinate system with the actual center position of the parking space as the center; According to the mapping relationship between the first coordinate system and the second coordinate system, the coordinates of the corner points in the first parking space frame information are converted to the second coordinate system to obtain the second parking space frame information.

4. The method according to claim 1, wherein The determining the current state of the parking space according to the second parking space frame information and the grounding point position information of the obstacle includes: detecting, based on the second parking space frame information and the grounding point position information of the obstacle, whether the grounding point position of the obstacle is within the parking space; When the grounding point of the obstacle is within the parking space, determining that the current state of the parking space is the occupied state; When the grounding point of the obstacle is not within the parking space, it is determined that the current state of the parking space is the unoccupied state.

5. The method according to claim 1, wherein The method of obtaining an AVM image of a target vehicle within a first preset range includes: Collecting original images in different directions by using a fisheye camera installed on the target vehicle; Correcting the original image according to camera parameters of the fisheye camera and a preset dedistortion function; The inverse perspective transformation is performed on the corrected image by the direct linear transformation DLT algorithm to obtain the inverse perspective mapping IPM image; According to the preset region of interest (ROI) division rule, the IPM images are spliced ​​and fused to generate the AVM image.

6. The method according to claim 1, wherein After determining the current state of the parking space according to the second parking space frame information and the grounding point position information of the obstacle, the method further includes: When the obstacle ratio in the parking space is less than a preset ratio, identifying the obstacle and determining a movable index of the obstacle; When the movable index of the obstacle is greater than the preset index, the second parking space frame information and the obstacle information corresponding to the parking space are visualized, and the visualized image is uploaded to the cloud or sent to vehicles within the second preset range.

7. A parking space detection device, characterized in that: include: An acquisition module is used to acquire an AVM image of a target vehicle within a first preset range; A prediction module is configured to obtain, based on the AVM image and a pre-trained parking space detection model, a target detection result within the first preset range, wherein the target detection result includes first parking space frame information of parking spaces within the first preset range and grounding point position information of obstacles; a conversion module, configured to obtain, based on the first parking space frame information, a first parking space frame corner point of a first border line corresponding to a parking space entry line in the first parking space frame, and obtain a second parking space frame corner point opposite to the first parking space frame corner point; if the clarity of the first parking space frame corner point is greater than a preset value, and the clarity of the second parking space frame corner point is less than a preset value, constructing a second border line adjacent to the first border line according to a preset parking space length, and constructing a virtual second parking space frame corner point based on the second border line; fusing the second border line and the virtual second parking space frame corner point with the parking space frame information of the parking space detected in real time to obtain updated parking space frame information; for each parking space, converting the first parking space frame information corresponding to the parking space to obtain second parking space frame information, wherein at least one side of the border line corresponding to the converted second parking space frame information overlaps with the parking space line corresponding to the parking space; A determination module is configured to determine a current state of the parking space according to the second parking space frame information and the grounding point position information of the obstacle, where the current state includes an occupied state or an unoccupied state.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images