Space parking space and lineation parking space fusion detection method and device
Through the integration strategy of space parking space and scribing parking space detection models, combined with additional attributes, the problem of low recall and accuracy of parking space detection in the existing technology is solved, and effective parking space detection in the same scenario is achieved, which improves the accuracy of the detection and recall rate.
Patent Information
- Application Number
- CN202410860693.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-28
- Publication Date
- 2024-11-15
AI Technical Summary
In the prior art, the detection recall and accuracy of overlapping space parking spaces and scribed parking spaces are low, and parking spaces cannot be effectively utilized, and parking spaces in urban spaces are prone to missed inspection of scribed parking spaces during the search stage.
The space parking space detection model and the scribed parking space detection model are used to reason and analyze the multi-view image data, and the parking space overlap is detected through the rear fusion strategy. Combined with the additional properties of the space parking space, non-overlapping space parking spaces and scribed parking spaces are output.
It improves the recall and accuracy of parking space inspection, so that the system can effectively detect lined parking spaces and space parking spaces in the same scenario, avoid overlapping and missing parking spaces, and make full use of parking space resources in urban space.
Smart Images

Figure CN118968460A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of automatic parking, and in particular to a fusion detection method and device for spatial parking spaces and lined parking spaces. Background Art
[0002] In modern cities, as the number of cars increases, the demand for parking spaces is also growing. However, due to the limited urban space, effective parking space resources are relatively scarce. Therefore, how to effectively detect and utilize various types of parking spaces has become an important issue.
[0003] Traditional parking spaces are usually pre-demarcated with clear line markings, which are called lined parking spaces. The advantage of this type of parking space is that the rules are clear and it is easy to manage and use. However, the number of such parking spaces is limited and it cannot fully utilize all available parking spaces. Another type of parking space is space parking spaces, which are usually found on the roadside and have no clear line markings. The existence of space parking spaces makes it possible to make better use of urban space and provide more parking spaces. However, since space parking spaces are not clearly marked, it is relatively complicated to detect and manage space parking spaces.
[0004] Nowadays, the industry's solution for lined parking spaces is generally a pure visual detection solution based on the BEV view (bird's eye view), in which the sensor is a camera that provides the system with necessary data and information. After completing the conversion of the multi-view image to the BEV view image, the visual detection model can complete the detection of lined parking spaces based on the BEV view image. Traditional spatial parking spaces are usually detected by ultrasonic sensors, but in recent years, spatial parking space detection technology based on pure vision has gradually developed to maturity, and spatial parking space detection models based on CNN or Transformer architecture have gradually been completed or implemented, which makes the spatial parking space detection method and the lined parking space detection method gradually consistent, and even the two can use the same backbone network to complete multi-task visual detection tasks.
[0005] In many cases, in the same scenario, there are usually two types of parking spaces: spatial parking spaces and lined parking spaces. Spatial parking spaces are more flexible than lined parking spaces from the perspective of product definition. For example, in a lined parking space scenario, as long as there are other parked vehicles and there is space for parking between or on both sides of the vehicles, a spatial parking space scenario can be formed. Therefore, the spatial parking space detection model based on pure vision will often detect spatial parking spaces in the lined parking space scenario. These spatial parking spaces may exist next to or near the lined parking spaces, but they may also overlap with the lined parking spaces.
[0006] Among the existing solutions to the overlap of spatial parking spaces and marked parking spaces, the first solution is to distinguish between marked parking spaces and spatial parking spaces to solve the problem of overlapping parking spaces; the second solution is to integrate marked parking spaces and spatial parking spaces at the same location. This solution will store these deleted spatial parking spaces in a cache, and use these spatial parking spaces as reference parking spaces by matching the spatial parking spaces in the cache.
[0007] However, the fusion solution of the first solution is too rough and cannot solve many boundary scenarios, which often makes the recall rate of parking space detection of the entire system low; and the second solution deletes all spatial parking spaces near the marked parking spaces, which will make the recall rate of parking space detection of the entire system low, making it impossible for users to effectively use all parking spaces within this space range, and it cannot solve the problem of missed detection of marked parking spaces during the parking space search stage.
[0008] Therefore, the prior art needs to be improved. Summary of the invention
[0009] The technical problem to be solved by the present invention is that, in view of the defects of the prior art, the present invention provides a method and device for fusion detection of spatial parking spaces and marked parking spaces, so as to solve the problem of low recall rate and low precision of parking space detection in the existing solution of overlapping spatial parking spaces and marked parking spaces.
[0010] The technical solution adopted by the present invention to solve the technical problem is as follows:
[0011] In a first aspect, the present invention provides a fusion detection method for spatial parking spaces and lined parking spaces, comprising:
[0012] Acquire multi-view image data;
[0013] Performing reasoning analysis on the multi-view image data based on a spatial parking space detection model and a lined parking space detection model to obtain a spatial parking space and a lined parking space; wherein the spatial parking space includes a first additional attribute;
[0014] Based on the post-fusion strategy, a parking space overlap detection is performed on the obtained spatial parking space and the marked parking space, and the first additional attribute of the spatial parking space is detected, and the detected spatial parking space and the marked parking space are output.
[0015] In one implementation, the spatial parking space detection model includes: a backbone network, a transformer encoder, a classification head and a regression head; wherein the classification head includes: a parking space classification layer and a scene classification layer.
[0016] In one implementation, the multi-view image data is inferred and analyzed based on the spatial parking space detection model and the lined parking space detection model to obtain the spatial parking space and the lined parking space; wherein the spatial parking space contains a first additional attribute, including:
[0017] Performing reasoning analysis on the multi-view image data based on the spatial parking space detection model to determine the spatial parking space and corresponding position information, posture information, first additional attribute, and second additional attribute;
[0018] Based on the marked parking space detection model, the multi-view image data is inferred and analyzed to determine the marked parking space and the corresponding position information, posture information and third additional attribute.
[0019] In one implementation, the reasoning analysis of the multi-view image data based on the spatial parking space detection model to determine the spatial parking space and the corresponding position information, posture information, first additional attribute and second additional attribute includes:
[0020] Extracting image features from the multi-view image data based on the backbone network of the spatial parking space detection model;
[0021] A transformer encoder based on the spatial parking space detection model converts multi-view image features into BEV view features;
[0022] Based on the classification head of the spatial parking space detection model, the classification head performs parking space classification and scene classification on the BEV viewing angle feature to obtain the spatial parking space and the corresponding first additional attribute and second additional attribute;
[0023] Based on the regression head of the spatial parking space detection model, regression processing is performed on the position and orientation related attributes of the spatial parking space to obtain the spatial parking space and the corresponding position information and posture information.
[0024] In one implementation, the inference analysis of the multi-view image data based on the spatial parking space detection model and the lined parking space detection model to obtain the spatial parking space and the lined parking space includes:
[0025] For each frame of multi-view image data, determining whether the corresponding spatial parking space is in the spatial parking space scene in units of parking spaces, and obtaining a first additional attribute of the corresponding spatial parking space;
[0026] Based on the obtained first additional attribute of the spatial parking space, labeling the corresponding spatial parking space data;
[0027] The spatial parking space detection model is obtained by training based on all the labeled data.
[0028] In one implementation, performing parking space overlap detection on the obtained spatial parking spaces and the marked parking spaces based on the post-fusion strategy, detecting the first additional attribute of the spatial parking spaces, and outputting the detected spatial parking spaces and the marked parking spaces include:
[0029] Traversing all the spatial parking spaces, and determining whether each spatial parking space overlaps with the marked parking space;
[0030] If there is an overlap with the marked parking space, the corresponding space parking space is deleted;
[0031] If there is no overlap with the marked parking space, determining whether the first additional attribute of the corresponding spatial parking space scene is true;
[0032] If the first additional attribute of the spatial parking space scene is true, the corresponding spatial parking space is reserved;
[0033] If the first additional attribute of the spatial parking space scene is not true, deleting the corresponding spatial parking space;
[0034] Output the spatial parking spaces and marked parking spaces after attribute detection.
[0035] In one implementation, the method of performing parking space overlap detection on the obtained spatial parking spaces and the marked parking spaces based on the post-fusion strategy, detecting the first additional attribute of the spatial parking spaces, and outputting the detected spatial parking spaces and the marked parking spaces further includes:
[0036] All spatial parking spaces that do not overlap with the lined parking spaces are retained, and spatial parking spaces that overlap with the lined parking spaces and whose intersection-over-union ratio of the overlapping area is greater than a set threshold and whose additional attribute is that they do not belong to the spatial parking space scene are deleted, and the spatial parking spaces and lined parking spaces after attribute detection are output.
[0037] In a second aspect, the present invention provides a fusion detection device for spatial parking spaces and marked parking spaces, comprising:
[0038] An acquisition module, used for acquiring multi-view image data;
[0039] An analysis module, configured to perform reasoning analysis on the multi-view image data based on a spatial parking space detection model and a lined parking space detection model to obtain a spatial parking space and a lined parking space; wherein the spatial parking space includes a first additional attribute;
[0040] The detection module is used to perform parking space overlap detection on the obtained spatial parking spaces and the marked parking spaces based on a post-fusion strategy, detect the first additional attribute of the spatial parking spaces, and output the detected spatial parking spaces and the marked parking spaces.
[0041] In a third aspect, the present invention provides a terminal, comprising: a processor and a memory, wherein the memory stores a fusion detection program for spatial parking spaces and marked parking spaces, and when the fusion detection program for spatial parking spaces and marked parking spaces is executed by the processor, it is used to implement the operation of the fusion detection method for spatial parking spaces and marked parking spaces as described in the first aspect.
[0042] In a fourth aspect, the present invention further provides a medium, which is a computer-readable storage medium, and which stores a fusion detection program for spatial parking spaces and marked parking spaces. When the fusion detection program for spatial parking spaces and marked parking spaces is executed by a processor, it is used to implement the operation of the fusion detection method for spatial parking spaces and marked parking spaces as described in the first aspect.
[0043] The present invention adopts the above technical solution to achieve the following effects:
[0044] The present invention performs reasoning analysis on multi-view image data based on a spatial parking space detection model and a lined parking space detection model, and performs parking space overlap detection on the obtained spatial parking spaces and lined parking spaces based on a post-fusion strategy, and detects the first additional attribute of the spatial parking spaces, and can output non-overlapping spatial parking spaces and lined parking spaces after detection. The present invention enables the system to complete both lined parking space detection and spatial parking space detection in the same scene by adding attributes to the spatial parking spaces, and enables the system to have both detection recall and accuracy through a post-fusion strategy. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.
[0046] Figure 1 It is a flow chart of a method for fusion detection of spatial parking spaces and marked parking spaces in one implementation of the present invention.
[0047] Figure 2 It is a structural schematic diagram of a spatial parking space detection model in one implementation of the present invention.
[0048] Figure 3 It is a flow chart of a post-fusion strategy in one implementation of the present invention.
[0049] Figure 4 It is a flow chart of parking space overlap and additional attribute detection in one implementation of the present invention.
[0050] Figure 5It is a functional principle diagram of a terminal in one implementation of the present invention.
[0051] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION
[0052] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0053] Exemplary Methods
[0054] Among the existing solutions for overlapping spatial parking spaces and lined parking spaces, the first solution is to distinguish lined parking spaces from spatial parking spaces to solve the problem of overlapping parking spaces; the second solution is to fuse lined parking spaces and spatial parking spaces at the same location; this solution will store these deleted spatial parking spaces in a cache, and use these spatial parking spaces as reference parking spaces by matching the spatial parking spaces in the cache. However, the fusion solution of the first solution is too rough and cannot solve many boundary scenarios, which often leads to the problem that the posture of the vehicle after parking does not meet expectations; and the second solution cannot effectively use all parking spaces within this spatial range, nor can it solve the problem of missed detection of lined parking spaces during the parking space search stage.
[0055] In response to the above technical problems, an embodiment of the present invention provides a fusion detection method for spatial parking spaces and lined parking spaces. The method performs reasoning analysis on multi-view image data based on a spatial parking space detection model and a lined parking space detection model, and performs parking space overlap detection on the obtained spatial parking spaces and lined parking spaces based on a post-fusion strategy, and detects the first additional attribute of the spatial parking spaces, and can output non-overlapping spatial parking spaces and lined parking spaces after detection. The embodiment of the present invention enables the system to complete both lined parking space detection and spatial parking space detection in the same scenario by adding attributes to the spatial parking spaces, and enables the system to have both detection recall and accuracy through a post-fusion strategy.
[0056] like Figure 1 As shown, an embodiment of the present invention provides a fusion detection method for spatial parking spaces and marked parking spaces, comprising the following steps:
[0057] Step S100: acquiring multi-view image data.
[0058] In this embodiment, for the situation where there are two types of parking spaces, spatial parking spaces and lined parking spaces, in the same scene, a fusion strategy is used so that the system can complete both lined parking space detection and spatial parking space detection in the same scene; at the same time, by using the fusion strategy, the entire parking space detection system can have both detection recall and accuracy, so that each detected parking space can appear where they should appear, without unreasonable overlap between parking spaces, or unexpected parking spaces, or missing parking spaces that should be detected.
[0059] For the usage scenario where both spatial parking spaces and marked parking spaces exist at the same time, the present embodiment firstly uses multiple cameras configured on the vehicle to capture multi-perspective image data of the target area (i.e., the area to be parked); wherein the target area may be the side area of the vehicle, or the front or rear area; the type of the marked parking spaces in the target area may be parallel parking spaces, vertical parking spaces, diagonal parking spaces, etc.; the multi-perspective image data includes at least: image data of front, rear, left, right, etc. perspectives.
[0060] In this embodiment, by acquiring multi-perspective image data of the area to be parked, the spatial parking space detection model and the lined parking space detection model can be used simultaneously to convert the multi-perspective image data into a BEV (bird's eye view) perspective image, and through a corresponding analysis process, the spatial parking spaces and lined parking spaces, as well as the additional attribute information of these two types of parking spaces, are extracted.
[0061] like Figure 1 As shown, in an implementation of the embodiment of the present invention, the fusion detection method of the spatial parking space and the marked parking space further includes the following steps:
[0062] Step S200: performing inference analysis on the multi-view image data based on a spatial parking space detection model and a lined parking space detection model to obtain spatial parking spaces and lined parking spaces.
[0063] In this embodiment, based on the spatial parking space detection model and the line parking space detection model, inference analysis is simultaneously performed on multi-perspective image data to obtain the spatial parking spaces and line parking spaces in the area to be parked, and obtain additional attribute information of these two parking spaces; wherein the additional attributes corresponding to the spatial parking spaces include: a first additional attribute and a second additional attribute; the first additional attribute is the attribute information of belonging to / not belonging to the spatial parking space scene obtained by the spatial parking space detection model detecting and analyzing the spatial parking spaces in the area to be parked; the second additional attribute is other additional attribute information obtained by the spatial parking space detection model detecting and analyzing the spatial parking spaces in the area to be parked; the additional attributes corresponding to the line parking spaces include: a third additional attribute; the third additional attribute is other additional attribute information obtained by the line parking space detection model analyzing the line parking spaces in the area to be parked.
[0064] For the spatial parking space detection model, a pure visual detection solution is used, so that it can be basically aligned with the marked parking space from the detection angle. Secondly, the parking space fusion strategy is advanced. Unlike the previous fusion strategy, the fusion is performed after both the marked parking space and the spatial parking space are output. Instead, in the design of the spatial parking space detection model, a new channel is added to the final output. The new channel is used to determine whether the spatial parking space belongs to the spatial parking space scene, and a binary classification judgment is made to determine the first additional attribute corresponding to the spatial parking space, that is, to determine whether the spatial parking space belongs to / does not belong to the spatial parking space scene. Additional attribute information; thereby, in the post-processing stage of the spatial parking space model, by parsing the first additional attribute, if it is determined to belong to the spatial parking space scene and the spatial parking space does not overlap with any line parking space, the spatial parking space is directly retained; if it is determined not to belong to the spatial parking space scene and there is a line parking space overlapping with it at this position, the spatial parking space is chosen not to be output.
[0065] like Figure 2 As shown, the spatial parking space detection model includes: a backbone network, a transformer encoder, a classification head and a regression head; wherein the classification head includes: a parking space classification layer and a scene classification layer, and the regression head includes: a parking space position and a posture regression layer.
[0066] Before the spatial parking space detection model is analyzed, the spatial parking space detection model needs to be trained with labeled data so as to be able to analyze the spatial parking spaces and the corresponding first additional attributes and second additional attributes.
[0067] In one implementation of the present embodiment, the inference analysis of the multi-perspective image data based on the spatial parking space detection model and the line parking space detection model includes: for each frame of multi-perspective image data, determining whether the corresponding spatial parking space is in a spatial parking space scene in units of parking spaces, and obtaining a first additional attribute of the corresponding spatial parking space; based on the obtained first additional attribute of the spatial parking space, annotating the corresponding spatial parking space data; and obtaining the spatial parking space detection model through training based on all annotated data.
[0068] In this embodiment, since the spatial parking space detection model finally needs to learn a binary classification task of whether it is a spatial parking space, it is necessary to obtain some historical spatial parking space data as a training set, and complete the labeling of the additional attributes of this parking space in the spatial parking space data labeling stage (that is, label whether this spatial parking space belongs to the spatial parking space scene); specifically, during labeling, the standard for judging whether a spatial parking space is in the spatial parking space scene is:
[0069] 1) Frame level, using parking spaces as units; wherein, using parking spaces as units means that in the parking area, the entire parking area is divided into a number of virtual parking spaces, and these virtual parking spaces are used as units to determine whether a space parking space is in the space parking space scene in turn.
[0070] 2) If the space parking space is obviously located in the marked parking space position, that is, if the parking space markings can be clearly seen on the ground, it is marked as not belonging to the space parking space scene;
[0071] 3) All other situations are considered to belong to spatial parking space scenarios, such as when there are no markings on the ground, or the markings are no-parking lines, traffic direction guide lines, and other markings.
[0072] In this embodiment, based on the spatial parking space data after the additional attributes of the parking spaces are marked, the spatial parking space detection model is trained, and the trained spatial parking space detection model can be used to perform reasoning analysis on the multi-view image data to obtain the spatial parking spaces and their additional attribute information.
[0073] Specifically, in an implementation of this embodiment, step S200 includes the following steps:
[0074] Step S201, performing reasoning analysis on the multi-view image data based on the spatial parking space detection model to determine the spatial parking space and corresponding position information, posture information, additional attributes of whether it belongs to a spatial parking space scene, and other additional attribute information;
[0075] In one implementation of this embodiment, step S201 includes the following steps:
[0076] Step S201a, extracting image features from the multi-view image data based on the backbone network of the spatial parking space detection model;
[0077] Step S201b, converting the multi-view image features into BEV view features based on the transformer encoder of the spatial parking space detection model;
[0078] Step S201c, performing parking space classification and scene classification on the BEV viewing angle feature based on the classification head of the spatial parking space detection model to obtain the spatial parking space and the corresponding first additional attribute and second additional attribute;
[0079] Step S201d: performing regression processing on the position and orientation related attributes of the spatial parking space based on the regression head of the spatial parking space detection model to obtain the spatial parking space and the corresponding position information and posture information.
[0080] In this embodiment, a pure vision-based spatial parking space model is used for the spatial parking space detection model. At the same time, the model structure adopts the BEV paradigm commonly used in the current autonomous driving industry. Therefore, the input of the model is multi-perspective image data.
[0081] The backbone network of the spatial parking space detection model is mainly used to extract rich image features. Therefore, common visual image backbone networks such as ResNet network or ViT network can be selected as the backbone network of the model; based on the backbone network, image features in the multi-perspective image data can be extracted.
[0082] The function of the transformer encoder of the spatial parking space detection model is to convert multi-view image features into BEV view features, so as to realize the panoramic image function, thereby facilitating subsequent processing by the classification head and regression head; for the image feature conversion process of the transformer encoder, you can refer to the processing process of the existing transformer encoder, which is relatively mature in the industry as a whole.
[0083] The function of the classification head of the spatial parking space detection model is to perform parking space classification and scene classification to obtain the spatial parking space and the corresponding first additional attribute and second additional attribute information; the structure of the classification head can refer to the classification head design method of achor-free (target detection algorithm) networks such as FCOS network or CenterNet network. It should be noted that the classification head here is that, in addition to the classification channel of the parking space itself, a new channel is added in this embodiment. The new channel is the first additional attribute channel, which is used to complete the binary classification of whether it is a spatial parking space scene, thereby adding an attribute of belonging to / not belonging to the spatial parking space scene to the corresponding spatial parking space.
[0084] The regression head of the spatial parking space detection model is used to complete the regression of the spatial parking space position and orientation related attributes, that is, the position and orientation related attributes of the spatial parking space are obtained through the regression head.
[0085] In this embodiment, through the analysis and processing process of the above spatial parking space detection model, the spatial parking space in the parking area and its corresponding position information, posture information, additional attributes of whether it belongs to the spatial parking space scene and other additional attribute information can be determined.
[0086] In this embodiment, for the area to be parked, in addition to obtaining the spatial parking spaces and corresponding attributes through the above classification by the spatial parking space detection model, the multi-view image data can also be inferred and analyzed based on the line parking space detection model.
[0087] Specifically, in an implementation of this embodiment, step S200 further includes the following steps:
[0088] Step S202: performing inference analysis on the multi-view image data based on the marked parking space detection model to determine the marked parking space and the corresponding position information, posture information and a third additional attribute.
[0089] In this embodiment, for the marked parking space detection model, a pure vision-based marked parking space model can be selected. Of course, the structure of the marked parking space detection model is similar to that of the spatial parking space detection model, that is, the marked parking space detection model includes: a backbone network, a transformer encoder, a classification head and a regression head; wherein the classification head includes: a parking space classification layer, and the regression head includes: a parking space position and posture regression layer.
[0090] Specifically, based on the marked parking space detection model, image features in the multi-perspective image data can be extracted, and the multi-perspective image features can be converted into BEV perspective features, and then parking spaces are classified according to the BEV perspective features to obtain marked parking spaces and other additional attribute information; and, the regression head is used to complete the regression of the position and orientation related attributes of the marked parking spaces.
[0091] In this embodiment, the multi-perspective image data is inferred and analyzed based on the spatial parking space detection model and the line parking space detection model to achieve the task of binary classification of parking spaces. Binary classification judgment is a relatively simple task for deep learning models. In the model output stage, the accuracy of spatial parking spaces can be well guaranteed, that is, each spatial parking space output has an accessory attribute of whether it is a spatial parking space scene, which can provide a strong priori information for the subsequent fusion strategy, and at the same time, there is post-fusion as a backup. If the spatial parking space and the line parking space overlap, then the spatial parking space is deleted. If there is no overlap and the accessory attribute is a spatial parking space scene, it is directly retained. In this way, the post-fusion logic is greatly simplified, making the entire system easy to maintain, expand and transplant, and can better balance the system's recall rate and accuracy.
[0092] like Figure 1 As shown, in an implementation of the embodiment of the present invention, the fusion detection method of the spatial parking space and the marked parking space further includes the following steps:
[0093] Step S300: Perform parking space overlap detection on the obtained spatial parking spaces and the marked parking spaces based on a post-fusion strategy, detect the first additional attribute of the spatial parking spaces, and output the detected spatial parking spaces and the marked parking spaces.
[0094] In this embodiment, if Figure 3As shown, when the online system of the fusion detection method of spatial parking spaces and lined parking spaces is running, through model reasoning and corresponding post-processing, several spatial parking spaces and lined parking spaces can be parsed. Generally speaking, these parking spaces will have information such as center point position, corner point position, orientation angle and other additional attributes. Compared with lined parking spaces, spatial parking spaces will have an additional attribute of whether it is a spatial parking space scene. For these several spatial parking spaces and lined parking spaces obtained by parsing, the present embodiment performs parking space overlap and additional attribute detection based on the post-fusion strategy, so as to accurately divide spatial parking spaces and lined parking spaces while satisfying the parking space recall rate.
[0095] Specifically, in an implementation of this embodiment, step S300 includes the following steps:
[0096] Step S301, traverse all the space parking spaces, and determine whether each space parking space overlaps with the marked parking space;
[0097] Step S302, if there is an overlap with the marked parking space, the corresponding space parking space is deleted;
[0098] Step S303, if there is no overlap with the marked parking space, then determining whether the first additional attribute of the corresponding spatial parking space scene is true; that is, determining whether the first additional attribute of the corresponding spatial parking space scene is: belonging to the spatial parking space scene;
[0099] Step S304: if the first additional attribute of the spatial parking space scene is true, retain the corresponding spatial parking space;
[0100] Step S305, if the first additional attribute of the spatial parking space scene is not true, deleting the corresponding spatial parking space;
[0101] Step S306, retaining all the spatial parking spaces that do not overlap with the lined parking spaces, and deleting the spatial parking spaces that overlap with the lined parking spaces and whose intersection-over-union ratio of the overlapping areas is greater than a set threshold and whose additional attribute is that they do not belong to the spatial parking space scene;
[0102] Step S307, outputting the spatial parking spaces and the marked parking spaces after attribute detection.
[0103] In this embodiment, the detection process of the post-fusion strategy mainly includes:
[0104] 1. If a spatial parking space overlaps with a lined parking space, it will be deleted directly regardless of the result of the additional attribute of whether it is a spatial parking space scene. If you want to increase the recall rate, you can set a threshold appropriately (the threshold is set according to specific business and product requirements). For example, if the IoU (intersection over union) of the overlapping area is greater than a certain threshold, the spatial parking space will be deleted directly.
[0105] 2. If the spatial parking space does not overlap with the lined parking space, the additional attribute of whether it is a spatial parking space scene needs to be used. If the additional attribute is true, the spatial parking space is directly retained. If it is not true, the spatial parking space is not retained. If you want to increase the recall rate, you can set a threshold and regard the overlapping area with an IoU less than the threshold as non-overlapping.
[0106] 3. If you want to make the most of all parking spaces in the scene, especially when the marked parking spaces are missed, you can adjust the post-fusion strategy appropriately. Here is a solution: all spatial parking spaces that do not overlap with the marked parking spaces are retained, and only the spatial parking spaces that overlap with the marked parking spaces and whose IoU of the overlapping area is greater than the set threshold and whose additional attribute is that they do not belong to the spatial parking space scene are deleted. However, this requires weighing the problem of false detection of spatial parking spaces.
[0107] like Figure 4 As shown, in the actual application scenario of this embodiment, the process of parking space overlap and additional attribute detection includes the following steps:
[0108] S11, traverse the parking spaces;
[0109] S12, determining whether all parking spaces have been merged; if yes, executing step S19; if no, executing step S13;
[0110] S13, traversing the marked parking spaces;
[0111] S14, determining whether the spatial parking space overlaps with the marked parking space; if yes, executing step S18; if no, executing step S15;
[0112] S15, determining whether the additional attribute (whether it belongs to the space parking scene) is true; if so, executing step S17; if not, executing step S16;
[0113] S16, the parking space is not reserved;
[0114] S17, reserve the parking space;
[0115] S18, the parking space is not reserved;
[0116] S19, output space parking space.
[0117] In the above steps S16, S17 and S18, for each step, the process returns to step S12 to determine whether all the space parking spaces have been merged, until all the space parking spaces have been merged.
[0118] In another implementation of this embodiment, the marked parking space and spatial parking space models can be completely unified, and a more general parking space detection paradigm can be adopted. In theory, the model output result plus post-processing can only output one parking space at the same position. In theory, the fusion strategy can be removed entirely, which not only saves computing overhead, but also makes the effect meet product requirements.
[0119] This embodiment achieves the following technical effects through the above technical solution:
[0120] This embodiment performs reasoning analysis on multi-view image data based on the spatial parking space detection model and the lined parking space detection model, and performs parking space overlap detection on the obtained spatial parking spaces and lined parking spaces based on the post-fusion strategy, and detects the first additional attribute of the spatial parking spaces, and can output non-overlapping spatial parking spaces and lined parking spaces after detection. This embodiment enables the system to complete both lined parking space detection and spatial parking space detection in the same scene by adding attributes to the spatial parking spaces, and enables the system to have both detection recall and accuracy through the post-fusion strategy.
[0121] Exemplary Devices
[0122] Based on the above embodiments, the present invention further provides a fusion detection device for spatial parking spaces and marked parking spaces, comprising:
[0123] An acquisition module, used for acquiring multi-view image data;
[0124] An analysis module, configured to perform reasoning analysis on the multi-view image data based on a spatial parking space detection model and a lined parking space detection model to obtain a spatial parking space and a lined parking space; wherein the spatial parking space includes a first additional attribute;
[0125] The detection module is used to perform parking space overlap detection on the obtained spatial parking spaces and the marked parking spaces based on a post-fusion strategy, detect the first additional attribute of the spatial parking spaces, and output the detected spatial parking spaces and the marked parking spaces.
[0126] This embodiment achieves the following technical effects through the above technical solution:
[0127] This embodiment performs reasoning analysis on multi-view image data based on the spatial parking space detection model and the lined parking space detection model, and performs parking space overlap detection on the obtained spatial parking spaces and lined parking spaces based on the post-fusion strategy, and detects the first additional attribute of the spatial parking spaces, and can output non-overlapping spatial parking spaces and lined parking spaces after detection. This embodiment enables the system to complete both lined parking space detection and spatial parking space detection in the same scene by adding attributes to the spatial parking spaces, and enables the system to have both detection recall and accuracy through the post-fusion strategy.
[0128] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be as follows: Figure 5 shown.
[0129] The terminal includes: a processor, a memory, an interface, a display screen and a communication module connected through a system bus; wherein the processor of the terminal is used to provide computing and control capabilities; the memory of the terminal includes a storage medium and an internal memory; the storage medium stores an operating system and a computer program; the internal memory provides an environment for the operation of the operating system and the computer program in the storage medium; the interface is used to connect to external devices; the display screen is used to display corresponding information; and the communication module is used to communicate with a cloud server or other devices.
[0130] When the computer program is executed by a processor, it is used to implement the operation of the fusion detection method of spatial parking spaces and marked parking spaces.
[0131] It can be understood by those skilled in the art that Figure 5 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0132] In one embodiment, a terminal is provided, which includes: a processor and a memory, the memory storing a fusion detection program for spatial parking spaces and marked parking spaces, and the fusion detection program for spatial parking spaces and marked parking spaces is used to implement the operation of the above-mentioned fusion detection method for spatial parking spaces and marked parking spaces when executed by the processor.
[0133] In one embodiment, a storage medium is provided, wherein the storage medium stores a fusion detection program for spatial parking spaces and marked parking spaces, and when the fusion detection program for spatial parking spaces and marked parking spaces is executed by a processor, it is used to implement the operation of the above-mentioned fusion detection method for spatial parking spaces and marked parking spaces.
[0134] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and volatile memory.
[0135] In summary, the present invention provides a fusion detection method and device for spatial parking spaces and lined parking spaces, the method comprising: acquiring multi-perspective image data; performing inference analysis on the multi-perspective image data based on a spatial parking space detection model and a lined parking space detection model to obtain spatial parking spaces and lined parking spaces; wherein the spatial parking spaces include a first additional attribute; performing parking space overlap detection on the obtained spatial parking spaces and lined parking spaces based on a post-fusion strategy, and detecting the first additional attribute of the spatial parking spaces, and outputting the detected spatial parking spaces and lined parking spaces. The present invention enables the system to complete both lined parking space detection and spatial parking space detection in the same scenario by adding attributes to spatial parking spaces, and enables the system to have both detection recall and accuracy through a post-fusion strategy.
[0136] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.
Claims
1. A fusion detection method for spatial parking spaces and lined parking spaces, characterized in that: include: Acquire multi-view image data; Performing reasoning analysis on the multi-view image data based on a spatial parking space detection model and a lined parking space detection model to obtain a spatial parking space and a lined parking space; wherein the spatial parking space includes a first additional attribute; Based on the post-fusion strategy, a parking space overlap detection is performed on the obtained spatial parking space and the marked parking space, and the first additional attribute of the spatial parking space is detected, and the detected spatial parking space and the marked parking space are output.
2. The fusion detection method of spatial parking spaces and lined parking spaces according to claim 1 is characterized in that: The spatial parking space detection model includes: a backbone network, a transformer encoder, a classification head and a regression head; wherein the classification head includes: a parking space classification layer and a scene classification layer.
3. The fusion detection method of spatial parking spaces and marked parking spaces according to claim 1, characterized in that: The multi-view image data is inferred and analyzed based on the spatial parking space detection model and the lined parking space detection model to obtain spatial parking spaces and lined parking spaces; wherein the spatial parking spaces contain a first additional attribute, including: Performing reasoning analysis on the multi-view image data based on the spatial parking space detection model to determine the spatial parking space and corresponding position information, posture information, first additional attribute, and second additional attribute; Based on the marked parking space detection model, the multi-view image data is inferred and analyzed to determine the marked parking space and the corresponding position information, posture information and third additional attribute.
4. The fusion detection method of spatial parking spaces and lined parking spaces according to claim 3 is characterized in that: The reasoning analysis of the multi-view image data based on the spatial parking space detection model to determine the spatial parking space and the corresponding position information, posture information, first additional attribute and second additional attribute includes: Extracting image features from the multi-view image data based on the backbone network of the spatial parking space detection model; A transformer encoder based on the spatial parking space detection model converts multi-view image features into BEV view features; Based on the classification head of the spatial parking space detection model, the classification head performs parking space classification and scene classification on the BEV viewing angle feature to obtain the spatial parking space and the corresponding first additional attribute and second additional attribute; Based on the regression head of the spatial parking space detection model, regression processing is performed on the position and orientation related attributes of the spatial parking space to obtain the spatial parking space and the corresponding position information and posture information.
5. The fusion detection method of spatial parking spaces and lined parking spaces according to claim 1, characterized in that: The method of performing reasoning and analysis on the multi-view image data based on the spatial parking space detection model and the lined parking space detection model to obtain the spatial parking space and the lined parking space comprises: For each frame of multi-view image data, determining whether the corresponding spatial parking space is in the spatial parking space scene in units of parking spaces, and obtaining a first additional attribute of the corresponding spatial parking space; Based on the obtained first additional attribute of the spatial parking space, labeling the corresponding spatial parking space data; The spatial parking space detection model is obtained by training based on all the labeled data.
6. The fusion detection method of spatial parking spaces and lined parking spaces according to claim 1, characterized in that: The method of performing parking space overlap detection on the obtained spatial parking spaces and the marked parking spaces based on the post-fusion strategy, detecting the first additional attribute of the spatial parking spaces, and outputting the detected spatial parking spaces and the marked parking spaces includes: Traversing all the spatial parking spaces, and determining whether each spatial parking space overlaps with the marked parking space; If there is an overlap with the marked parking space, the corresponding space parking space is deleted; If there is no overlap with the marked parking space, determining whether the first additional attribute of the corresponding spatial parking space scene is true; If the first additional attribute of the spatial parking space scene is true, the corresponding spatial parking space is reserved; If the first additional attribute of the spatial parking space scene is not true, deleting the corresponding spatial parking space; Output the spatial parking spaces and marked parking spaces after attribute detection.
7. The fusion detection method of spatial parking spaces and lined parking spaces according to claim 1, characterized in that: The method of performing parking space overlap detection on the obtained spatial parking spaces and the marked parking spaces based on the post-fusion strategy, detecting the first additional attribute of the spatial parking spaces, and outputting the detected spatial parking spaces and the marked parking spaces, further includes: All spatial parking spaces that do not overlap with the lined parking spaces are retained, and spatial parking spaces that overlap with the lined parking spaces and whose intersection-over-union ratio of the overlapping area is greater than a set threshold and whose additional attribute is that they do not belong to the spatial parking space scene are deleted, and the spatial parking spaces and lined parking spaces after attribute detection are output.
8. A fusion detection device for space parking spaces and lined parking spaces, characterized in that: include: An acquisition module, used for acquiring multi-view image data; An analysis module, configured to perform reasoning analysis on the multi-view image data based on a spatial parking space detection model and a lined parking space detection model to obtain a spatial parking space and a lined parking space; wherein the spatial parking space includes a first additional attribute; The detection module is used to perform parking space overlap detection on the obtained spatial parking spaces and the marked parking spaces based on a post-fusion strategy, detect the first additional attribute of the spatial parking spaces, and output the detected spatial parking spaces and the marked parking spaces.
9. A terminal, characterized in that: include: A processor and a memory, wherein the memory stores a fusion detection program for spatial parking spaces and marked parking spaces, and when the fusion detection program for spatial parking spaces and marked parking spaces is executed by the processor, it is used to implement the operation of the fusion detection method for spatial parking spaces and marked parking spaces as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a fusion detection program for spatial parking spaces and marked parking spaces. When the fusion detection program for spatial parking spaces and marked parking spaces is executed by a processor, it is used to implement the operation of the fusion detection method for spatial parking spaces and marked parking spaces as described in any one of claims 1-7.
Citation Information
Patent Citations
Automatic parking method and system based on optical flow method
CN113753029A
Automatic parking system, method and device based on multi-sensor front fusion
CN114379544A
Parking space detection method and device
CN116453372A
Spatial parking space detection method and device, vehicle and computer readable storage medium
CN117541888A
Parking space detection method and device, vehicle and storage medium
CN118172762A
