Idle parking space detection method based on maximum matching and parking space segmentation

Abstract

The invention relates to the technical field of video image recognition, in particular to an idle parking space detection method based on maximum matching and parking space segmentation, which comprises the following steps of: S1, calibrating all parking spaces in a picture under the view angle of a camera to form a parking space calibration pixel map and a JSON dictionary configuration file; s2, performing semantic segmentation on an input video stream by using a segmentation model, and performing BIFPNEMA bidirectional cross-scale connection and a weighted feature fusion model by using a picture to obtain a detection result graph of a parking space occupancy condition; and associating the parking space attribute of the calibration pixel map to the corresponding parking space of the detection result map through a mapping relationship, and generating a detection result map with the attribute. And S3, filtering out misrecognized parking spaces and irregularly occupied parking spaces by using a multi-priority matching algorithm on the parking space calibration pixel map obtained by calibration and the detection result map to obtain idle parking spaces and occupied parking spaces. S4, the detected parking spaces are output in a serialized mode, and the attribute information and the state information of the parking spaces are combined and output to a service platform.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video image recognition, and particularly relates to an idle parking space detection method based on maximum matching and parking space segmentation. BACKGROUND

[0002] The number of parking spaces in high-speed service areas and scenic areas is limited. When the traffic volume is large, if the parking spaces are not fully utilized, it may cause serious congestion at the entrance and exit, thereby causing safety accidents. In many high-speed service areas, there are phenomena of illegal parking and one vehicle occupying multiple spaces, which may cause illegal occupation of parking spaces if not timely controlled by management personnel. At the same time, the management personnel of the service area also need to know the number of idle parking spaces in time, so as to improve the management ability of the reception service of the scenic area and the high-speed service area.

[0003] Traditional parking spaces use micro radar detection, infrared detection, ground magnetic induction coil detection and radio frequency identification technology. This kind of method needs to install special sensing equipment for each parking space in the parking lot, which has large engineering cost, is difficult to maintain in later period, and needs to invest a lot of manpower and material resources.

[0004] There are also methods of using camera video stream to recognize the state of parking spaces. For example, a service area square parking space detection system and method based on high point video image technology, with publication number CN118379901B. This patent uses a vehicle type recognition model to judge whether the parked vehicle is an illegally parked vehicle based on the segmentation result, and generates the number of remaining parking spaces of each type of parking space according to the occupancy of each type of parking space. The core of this method is to use two models of segmentation and detection to detect the parked vehicle. The vehicle types are numerous, and the algorithm cannot guarantee that the segmentation and detection algorithms can be completely recognized, so the precision is not high. At the same time, the two models of segmentation and detection are executed in sequence, and the requirement for computing power is also large, so the detection speed is not fast enough. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, the present application provides an idle parking space detection method based on maximum matching and parking space segmentation, so as to solve the problems in the above background.

[0006] The specific method steps of the present application include the following:

[0007] S1: Label the information of all points needing parking space detection in the picture under the camera view, form a parking space calibration pixel graph and a JSON dictionary configuration file.

[0008] S2: semantic segmentation is performed on the input video stream in the parking state using a segmentation model to obtain three consecutive images, and a BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network model is used to obtain a detection result image of the parking space occupancy.

[0009] By calibrating the spatial mapping relationship between the pixel map and the detection result image and matching and fusing the parking spaces two by two, the parking space attribute in the calibration pixel map is associated with the parking space region corresponding to the pixel coordinates in the detection result image, and an attribute information detection result image is generated.

[0010] S3: further fuse the parking space state information output by combining the detection result image with the parking space calibration pixel map, and filter out the misrecognized parking spaces and illegally occupied parking spaces to obtain all the idle parking spaces and occupied parking spaces.

[0011] S4: serializing the detected parking spaces: combining the attribute information and state information of the parking spaces and outputting to the service platform.

[0012] Preferably, the calibration process in S1 comprises:

[0013] S11. Collect the image of the parking space area to be detected, which is obtained by a fixed angle camera deployed at the parking space site. S12. Provide the image to the labeling personnel, and label all the parking spaces in the image by the labeling personnel. The labeling content includes: for the unoccupied parking space, directly identify the parking space area; for the occupied parking space, based on the parking space boundary feature, restore the complete parking space contour and perform pixel identification. S13. Map the semantic segmentation labeling result to the original image through a projection transformation algorithm to generate a parking space calibration pixel map.

[0014] Further, the semantic segmentation labeling method comprises:

[0015] (a) assign a unique index value to each parking space in the calibration order.

[0016] (b) set the three-channel value of the pixel points belonging to the parking space to the index value of the current parking space. (c) set the three-channel value of the pixel points in the non-parking space area to a preset background value.

[0017] Wherein, the index value is an integer in the range of 1 to 254, and the index values of different parking spaces are different, and the preset background value is 255.

[0018] Further, the index value assignment order is consistent with the parking space number calibration order, and the index value of the kth calibrated parking space is k, where k=1, 2,..., park_count, park_count is the total number of parking spaces and park_count≤254.

[0019] Preferably, a corresponding JSON dictionary configuration file is established based on the parking space calibration pixel map; the JSON dictionary configuration file includes:

[0020] a total parking space record item;

[0021] a parking space attribute set, wherein each parking space attribute includes a parking space number identifier, a parking space pixel area value, an associated video stream identifier, a physical space coordinate value, and a parking space classification attribute.

[0022] Preferably, the parking space pairwise matching and fusion specific process includes:

[0023] 1) Parking space coordinate system unification: affine transformation is performed on the parking space polygons in the detection result map and the calibration pixel map to unify the coordinate system and record parameters, the polygon centroids are translated to the origin and then normalized and scaled according to the maximum Euclidean distance from the vertices to the origin, the cross product method is used to unify the vertex order to the clockwise direction and take the x+y minimum vertex as the starting point, wherein x and y represent pixel coordinates.

[0024] 2) Parking space vertex feature extraction: the histogram of vertex distance, the mean and variance of adjacent edge angles, and the convex hull vertex ratio are extracted by Delaunay triangulation as vertex features, the polygon is converted into a binary image to calculate the logarithmic normalized values of 7 Hu moments and the first 10 coefficients of the Fourier transform of the contour turning function as contour features, the two types of features are Min-Max normalized respectively, then fused according to a 6:4 weight, and finally reduced by PCA to retain 95% variance.

[0025] 3) Parking space vertex similarity calculation: the vertex distribution difference is calculated based on the intersection and union ratio of the Thiessen polygon, the Hausdorff distance, and the Earth Mover's Distance, the comprehensive similarity score is generated by combining the Hu moment weighted Euclidean distance and the L2 norm distance of the contour turning function, and the shape complexity factor is introduced for correction.

[0026] 4) Parking space candidate matching generation: an m×n similarity matrix is constructed and the Top-3 candidates are retained with a 0.7 threshold filter, a bidirectional verification strategy is used to match the parking spaces, the highest score matching is retained for the conflict items, the dynamic time warping relaxation algorithm is used to process the unmatched items, and finally an one-to-one m×n parking space matching matrix is generated, wherein m represents the number of parking spaces in the calibration pixel map and n represents the number of parking spaces in the detection result map.

[0027] Preferably, the step S3 includes:

[0028] S31. Obtain a set of parking space segmentation results P_seg output by the BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network, and perform a sorting operation on P_seg to generate an ordered queue Queue_seg, including:

[0029] Primary sorting: arrange in descending order according to the confidence value of the segmentation result from high to low.

[0030] Secondary sorting: when the confidence value is the same, arrange in descending order according to the parking space pixel area corresponding to the detection box from large to small.

[0031] Calculate the queue length value Queue_seg_length of Queue_seg.

[0032] S32. Based on the sorted parking space segmentation result queue Queue_seg, create a corresponding matching state queue Queue_result, wherein:

[0033] The queue length of Queue_result is the same as Queue_seg.

[0034] Each parking space state value of Queue_result is selected from the set {0, 1, 2, 3}, wherein:

[0035] 0 represents that the corresponding parking space has not started to be traversed and matched;

[0036] 1 represents that the corresponding parking space is partially matched;

[0037] 2 represents that the corresponding parking space is completely matched;

[0038] 3 represents that the corresponding parking space is an invalid detection result.

[0039] i represents the i-th parking space in the Queue_result queue, 1≤i≤Queue_seg_length.

[0040] Initialize all parking space state values of Queue_result to 0.

[0041] S33. Select a parking space with a state value of 0 from the matching state queue Queue_result in order as a seed node .

[0042] Based on the pixel coordinates of the parking space, calculate the intersection over union Ratio_i of the pixel area of the seed node and all parking space pixel areas in the calibrated parking space pixel map.

[0043] When Ratio_i>0.9, then execute: set the seed node the state value in the Queue_result is updated to 2, and step S34 is executed;

[0044] When Ratio_i≤0.9, then execute: update the state value in the Queue_result to 3, and execute step S33.

[0045] wherein:

[0046] (i) 1≤i≤Queue_seg_length;

[0047] (ii) the seed node after the state value is updated to 3 no longer satisfies the selection condition of the state value being 0;

[0048] (iii) the intersection-over-union ratio .

[0049] S34. Based on the seed node parking space determined in step S33, execute the following operations:

[0050] (1) Traverse each parking space in the Queue_result whose state value is 0 or 1, wherein 1≤i≤Queue_seg_length.

[0051] (2) For each traversed parking space , calculate the intersection-over-union ratio Ratio_ij of the seed node parking space pixel area and parking space pixel area.

[0052] (3) Update the state value of the parking space according to the intersection-over-union ratio Ratio_ij:

[0053] When Ratio_ij>0.8, update the state value of the parking space to 3;

[0054] When 0.4<Ratio_ij<0.8, update the state value of the parking space to 1;

[0055] When Ratio_ij≤0.4, keep the state value of the parking space unchanged.

[0056] (4) After the traversal is completed:

[0057] When it is detected that there is a parking space with a state value of 0 in the Queue_result, return to step S33 to execute the new seed node selection process.

[0058] ​​​When no parking space with a state value of 0 is detected in the Queue_result, the parking space matching result is output and the process is terminated.

[0059] wherein:

[0060] (i) the intersection-over-union

[0061] (ii) the state value is updated to 3 to indicate an invalid detection result.

[0062] Preferably, the parking space state information in S4 includes free parking spaces and partially occupied parking spaces.

[0063] The parking space attribute information includes parking space number, parking space physical coordinates, and parking space type.

[0064] Technical effects and advantages of the present application:

[0065] 1. High detection accuracy

[0066] The BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network model processes continuous images, enhancing the accuracy of parking space occupancy detection and reducing false positives and false negatives caused by factors such as changes in lighting and occlusion.

[0067] 2. Efficient and scalable data processing

[0068] The parking space detection map and the parking space calibration pixel map form a matrix through pixel area overlap, and then a multi-priority matching algorithm is used to simplify the training and inference process of the semantic segmentation model, reducing computational complexity. Reducing the requirement for computing power, cost-effective, faster response speed.

[0069] 3. Strong adaptability

[0070] Through the fusion of the calibration pixel map and the real-time detection result, efficient processing of parking information updates is implemented.

[0071] Dynamic threshold design (based on parking space pixel intersection-over-union value adjustment) ensures the consistency of detection of large and small parking spaces, avoiding false positives due to small parking spaces with fewer pixels.

[0072] 4. Convenient system integration

[0073] The output data is serialized (including parking space index, coordinates, category, and state), which can be directly connected to the intelligent parking platform without additional conversion, improving system response speed. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 Flowchart of the free parking space detection method based on maximum matching and parking space segmentation;

[0075] Figure 2The figure shows the image under the camera for detecting the parking space when the parking space is not calibrated.

[0076] Figure 3 The figure shows the pixel calibration image for the parking space.

[0077] Figure 4 The figure shows the FPN structure.

[0078] Figure 5 The figure shows the BIFPN_EMA structure.

[0079] Figure 6 The figure shows the indicators of the original YOLO model on the test set.

[0080] Figure 7 The figure shows the indicators of the optimized YOLO_EMA model on the test set.

[0081] Figure 8 The figure shows the detection result obtained after semantic segmentation of the video image in the parking state.

[0082] Figure 9 The figure shows the specific steps of S3. DETAILED DESCRIPTION

[0083] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0084] In this embodiment, the idle parking space detection algorithm is deployed on the camera side or the background server side. If deployed on the camera side, a chip with an algorithm power of 4T or more is required for camera detection and operation, the camera resolution is 2K or more, the camera installation height is 10 meters, and the camera angle is in a top-down view. If deployed on the server, a graphics card is required, and the camera video stream is pushed to the server side to run the detection based on the TensorRT acceleration environment.

[0085] This embodiment mainly includes four steps:

[0086] S1: In the camera view, the information of all points needing parking space detection in the picture is calibrated to form a parking space calibration pixel map P_all and a JSON dictionary configuration file.

[0087] The position of all parking spaces in the picture, the total number of parking spaces, the actual spatial position of the parking spaces and the pixel area of each parking space need to be calibrated under the camera view of the high-speed service area, wherein the position of the parking spaces in the picture is stored in the PNG picture pixel mode, and the pixel area of each parking space, the actual spatial position of the parking spaces and the total number of parking spaces are stored in the JSON dictionary format configuration file.

[0088] First, the picture of the parking space point to be detected under the camera view is collected, and the picture is given to the labeling personnel to identify all the parking spaces in the picture by using semantic segmentation (including the occupied parking spaces, which need to be manually drawn), and the identification result of the parking space segmentation is processed, and the identification result is projected into the picture.

[0089] For an RGB image, set the pixel point P(x, y) as a parking space, modify the three channels of the pixel point as B(x, y) = G(x, y) = R(x, y) = index_of_park (where 1 index_of_park park_count), wherein x and y are pixel coordinates, BGR represents three channels of the picture, index_of_park represents the index of the current parking space, the index value ranges from 1 to the total number of parking spaces park_count, and the total number of parking spaces is maximally 254, and the order of the parking space index is calibrated by the labeling personnel in sequence, and for the pixel point P(x, y) which is not a parking space, the three channels of the pixel point are modified as B(x, y) = G(x, y) = R(x, y) = 255, and the parking space calibration pixel graph is obtained.

[0090] The comparison before and after the calibration of the parking space is shown in Figure 2 and Figure 3 .

[0091] The corresponding JSON dictionary configuration file is established for the labeled parking space calibration pixel graph, and the data structure includes:

[0092] First-level field:

[0093] park_count: records the total number n of parking spaces in the calibration area; park_values: stores a dictionary set of parking space attributes.

[0094] The second-level field under park_values: parking number: a unique identifier corresponding to index_of_park; squares: records the pixel area of the corresponding parking space; resp_id: associated video stream identifier; position_x, position_y: records the coordinate position of the parking space in the actual physical space; park_cls: records the classification attribute information of the parking space.

[0095] These basic attribute information is loaded as a configuration file at algorithm initialization.

[0096] S2: using a segmentation model to perform semantic segmentation on the input video stream in the parking state to obtain three continuous images, and using the three continuous images to perform BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network model to obtain a detection result image P_seg of the parking space occupancy.

[0097] By calibrating the spatial mapping relationship of the pixel map P_all and the detection result image P_seg and the two-by-two matching fusion of the parking spaces, the parking space attribute in the calibration pixel map P_all is associated with the parking space region corresponding to the parking space pixel coordinates in the detection result image P_seg, and an attribute information detection result image is generated.

[0098] Due to the large inclination and small size of the parking space in the distance under the high point camera view, the current model is modified on the YOLO algorithm architecture to obtain a YOLO_EMA model, as shown in Figure 4 and Figure 5 The original YOLO model has a core structure feature pyramid network (FPN) that is a top-down approach to combine multi-scale features, which is not sufficient for feature fusion of small parking spaces and inclined parking spaces in the distance. In the YOLO_EMA of the present embodiment, a BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network model is adopted, which has higher accuracy and efficiency. Ablation comparison experiments are performed between the original YOLO model and the YOLO_EMA network model after adopting the BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion, and the experimental results prove that compared with the original YOLO model, the average precision Map50 of YOLO_EMA is improved by 6.2%, and the inference speed and YOLO model structure are comparable. The EMA module proposed in the present embodiment focuses on reducing the calculation overhead while retaining important information in each channel. This module reshapes part of the input channels into batch dimensions and groups the channel dimensions into multiple sub-features. The purpose of this is to enable spatial semantic features to be evenly distributed within each feature group, thereby better capturing local features of the data. Specifically, in addition to encoding global information in each parallel branch to recalibrate the weights in the channel dimension, the output features of the two parallel branches are further aggregated through cross-dimensional interaction to capture the relationship between pixel-level pairs.

[0099] As shown in Figure 6 and Figure 7As shown, the ablation contrast experiment model training set and test set are from the high point camera pictures of the service area, and the current training set is 2.2w, and the test set is 4000. From the test indicators of the training set and the test set in the ablation experiment, it can be seen that: 1. Model precision: the original model has an average precision of 86.7% in Map50, and the optimized YOLO_EMA model has an average precision of 92.9% in Map50, which increases by 6.2% in precision. 2. The optimized YOLO_EMA model has faster Loss convergence speed and is more advantageous in iterative optimization. This model is directly deployed in the background service area or the camera side with computing power.

[0100] The effect after semantic segmentation is as shown in Figure 8 As can be seen from the segmented pixel results of the parking space, there are still sawtooth patterns and irregular characteristics. Purely relying on the segmented results cannot obtain the attribute information of each parking space and is accompanied by problems such as redundant detection of part of the parking space and possible misidentification. It is still necessary to combine the pixel calibration map in step S1 and the configuration file to further match and output the attribute information of each parking space.

[0101] Due to the same camera height and viewing angle, through the spatial mapping relationship between the calibration pixel map P_all and the detection result map P_seg and the two-by-two matching fusion of the parking spaces, the parking space attribute in the calibration pixel map P_all is associated with the parking space area corresponding to the pixel coordinates in the detection result map P_seg. Then, through the three-channel pixel value index_of_park on the matched calibration pixel map P_all<x, y> (i.e., the corresponding first index in the Json file), the attribute information fusion of the detection result map P_seg parking space can be realized.

[0102] The specific process of the two-by-two matching fusion of the parking spaces includes:

[0103] 1) Parking coordinate system unification: check and unify the coordinate systems of the two groups of polygon parking spaces, perform affine transformation conversion, and record the conversion parameters for subsequent verification.

[0104] Normalization processing: calculate the centroid of each polygon and translate it to the origin, and normalize and scale according to the maximum distance from the vertex to the origin.

[0105] Vertex ordering standardization: use the cross product method to determine the vertex order, uniformly adjust to the clockwise direction, and select the vertex with the smallest x+y value as the starting point, where x and y represent pixel coordinates.

[0106] 2) Parking vertex feature extraction: generate a Voronoi polygon through Delaunay triangulation, calculate the histogram (10-bin) of the vertex distance, the statistical features (mean, variance) of the adjacent edge angle, and the proportion of the convex hull vertex.

[0107] Contour feature extraction: Convert the polygon to a binary image and calculate 7 Hu moments (normalized after taking the logarithm), extract the contour turning function and perform Fourier transform on 100 sampling points to get the first 10 coefficients.

[0108] Feature fusion: Min-Max normalization is performed on the two types of features respectively, then combined with a weight of 6:4, and PCA is used for dimensionality reduction to retain 95% of the variance information.

[0109] 3) Parking space vertex similarity calculation: The intersection-over-union (IoU) and Hausdorff distance of the Thiessen polygon are used to measure the difference in vertex distribution, combined with the Earth Mover's Distance to measure the overall distribution.

[0110] Contour similarity calculation: Compare the weighted Euclidean distance of 7 Hu moments, and calculate the L2 norm distance after aligning the turning function.

[0111] Comprehensive score: Weight the vertex and contour similarity with a weight of 0.4:0.6, and introduce a shape complexity factor (log(N_edges) / 10) for correction.

[0112] 4) Parking space candidate matching generation: Construct an m x n similarity matrix (m represents the number of parking spaces in the calibration pixel map, n represents the number of parking spaces in the detection result map), and keep Top-3 candidates for each polygon and set an absolute threshold of 0.7 for filtering.

[0113] Bidirectional verification: First, select the best match from A→B, then verify from B→A, if there is a conflict, keep the highest score match and perform a relaxed matching for the unmatched items, finally generate a one-to-one m x n parking space matching matrix.

[0114] S3: Further fuse the parking space state information output for the detection result map P_seg and the parking space calibration pixel map P_all, use the multi-priority matching algorithm on the calibration parking space calibration pixel map P_all and the detection result map P_seg to filter out misidentified parking spaces and illegally occupied parking spaces to obtain all free parking spaces and occupied parking spaces.

[0115] As shown in Figure 9 , the specific steps of S3 include:

[0116] S31. Sort all parking spaces obtained by the BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network P_seg according to confidence from high to low and parking space area from large to small to obtain Queue_seg, and calculate the queue length Queue_seg_length of Queue_seg.

[0117] First, sort the parking spaces with high confidence (m represents the mth segmentation result, 1≤m≤254, considering the weather and actual camera height and angle, generally not more than 254 parking spaces under the management of a camera vision, also consistent with the previous calibration pixel map limited upper limit) in front of Queue_seg, when the same result is placed in front of the detection box parking pixel area, it is convenient for subsequent processing of high precision, short distance parking, and finally get the sorted Queue_seg.

[0118] S32. According to the sorted Queue_seg, create a corresponding matching state queue Queue_result, There are four states, 0, 1, 2, 3. Among them, 0 represents that the parking space has not started to be traversed, 1 represents that the parking space has been partially matched, 2 represents that the parking space has been completely matched, and 3 represents that the parking space is an invalid detection result (invalid detection result includes misrecognized parking space, illegal occupation, etc.). The default value is 0 (the starting state is that all parking spaces have not started to be traversed and matched).

[0119] S33. First, select a from the head of the Queue_result queue. And =0 (that is, select the segmentation result with high confidence and not traversed and matched, 0 ), the selected as a seed node, based on the pixel coordinates of the parking space The seed node parking pixel area and all parking pixel areas IOU intersection ratio Ratio_i in the calibration pixel map P_all, when there is a Ratio_i greater than 0.9, it is considered as an effective idle parking space (selected as a seed node parking space), and Set to 2, and proceed to S34. When there is no Ratio_i greater than 0.9, set To 3, representing that the parking space is an invalid detection result. Start again Step S33 to select a new seed node parking space from the Queue_result queue (the current invalid detection result of this seed node parking space Is changed to 3, which does not meet the condition of =0, so it will not be selected as the new seed node parking space in the next round).

[0120] Among them, the pixel area IOU intersection ratio Ratio_i is calculated as follows:

[0121] .

[0122] S34. To the seed node parking space As the initial calculation result, calculate The pixel overlap area ratio Ratio_ij of (where it needs to satisfy =0 or 1, representing that the jth parking space is not traversed and matched or partially matched. i+1≤j≤Queue_seg_length, when i=Queue_seg_length exists in S33, i+1=Queue_seg_length+1, at this time, the lower limit of j exceeds the upper limit, and naturally forms an empty traversal.), when the Ratio_ij is greater than 0.8, it is considered that the parking space is an invalid detection result, representing misrecognition or redundant detection, and directly setting =3, when the Ratio_ij is greater than 0.4 and less than 0.8, it is considered that the parking space is partially occupied, and directly setting =1, when the Ratio_ij is less than 0.4, it is skipped. The purpose of this step is to use the determined seed node parking space to eliminate the redundant detection results in the segmentation results and the misrecognized parking spaces after vehicle occlusion.

[0123] Wherein, the pixel area IOU intersection ratio Ratio_ij is calculated as follows: .

[0124] Finally, when there are still =0 repeated S33 after traversing, a new seed parking space node is selected, otherwise it is ended.

[0125] From the multi-priority matching algorithm flow and traversal structure, it can be calculated that the worst time complexity of the algorithm is O(n) and the space complexity is O(3n). Compared with the traditional Hungarian matching algorithm, the worst time complexity is O( ) and the space complexity is O( ). It can be seen that in the matching scene of the pixel map that has been calibrated in advance, the algorithm has great advantages in memory occupation and algorithm calculation speed compared with the commonly used Hungarian matching algorithm.

[0126] After the multi-priority matching algorithm is executed, stores the state of each parking space after traversal, including 1 representing that the parking space is partially matched (i.e., partially occupied), 2 representing that the parking space is completely matched (i.e., an idle parking space), and 3 representing that the parking space is an invalid detection result.

[0127] S4: Serializing and outputting the detected parking spaces: combining the attribute information and state information of the parking spaces, and outputting to the service platform.

[0128] The parking space states =2 and ​=1 corresponding P_seg parking space fusion attribute information and state information is reported to the service area management platform.

[0129] The detection result image P_seg parking space of the S2 step is fused with the parking space JSON dictionary configuration file to obtain parking space attribute information, and the detection result image P_seg parking space is sorted with a parking space index in the S31 to obtain Queue_seg parking space. The Queue_seg parking space is obtained in the S33 and the S34 Each parking space state information.

[0130] The parking space state information includes an idle parking space and a partially occupied parking space.

[0131] The parking space attribute information includes a parking space number, a parking space physical coordinate and a parking space type.

[0132] Finally, the above only describes the preferred embodiments of the present application and is not used to limit the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for detecting vacant parking spaces based on maximum matching and parking space segmentation, characterized in that: The steps include: S1: Calibrate the information of all points that need parking space detection in the image from the camera's perspective to generate a parking space calibration pixel map and a JSON dictionary configuration file; S2: Use the segmentation model to perform semantic segmentation on the input video stream of the parking state, obtain three consecutive pictures, and use the three consecutive pictures to perform BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network model to obtain the detection result map of parking space occupancy; By spatially mapping the calibration pixel map and the detection result map and matching the parking spaces one by one, the parking space attributes in the calibration pixel map are associated with the parking space areas corresponding to the parking space pixel coordinates in the detection result map, generating a detection result map with attribute information. S3: The detection result image is combined with the parking space calibration pixel image to further integrate the parking space status information output. The calibrated parking space calibration pixel image and the detection result image are matched using a multi-priority matching algorithm to filter out misidentified parking spaces and illegally occupied parking spaces to obtain all available parking spaces and occupied parking spaces. S4: Serialize and output the detected parking spaces: Combine the attribute information and status information of the parking spaces and output them to the service platform.

2. The vacant parking space detection method based on maximum matching and parking space segmentation according to claim 1 is characterized in that: The calibration process in S1 includes: S11. Collect an image of the parking area to be inspected, and the image is obtained by a fixed-view camera deployed at the parking space; S12. Provide the image to annotators, who perform semantic segmentation and annotation on all parking spaces in the image. The annotation content includes: for unoccupied parking spaces, directly mark the parking space area; for occupied parking spaces, restore the complete parking space outline based on the parking space boundary features and perform pixel identification; S13. Map the semantic segmentation and annotation results to the original image through a projection transformation algorithm to generate a parking space calibration pixel map.

3. The vacant parking space detection method based on maximum matching and parking space segmentation according to claim 2 is characterized in that: The semantic segmentation and annotation method comprises: (a) Assign a unique index value to each parking space in the order of calibration; (b) Set the three-channel value of the pixel belonging to the parking space as the index value of the current parking space; (c) setting the three-channel values ​​of the pixels in the non-parking area to the preset background values; The index value is an integer in the range of 1 to 254, and the index values ​​of different parking spaces are different from each other. The preset background value is 255.

4. The method for detecting vacant parking spaces based on maximum matching and parking space segmentation according to claim 3, characterized in that: The order of assigning the index values ​​is consistent with the order of calibrating the parking space numbers. The index value of the kth calibrated parking space is k, where k=1, 2, ..., park_count, park_count is the total number of parking spaces and park_count≤254.

5. The method for detecting vacant parking spaces based on maximum matching and parking space segmentation according to claim 2, characterized in that: Create a corresponding JSON dictionary configuration file based on the parking space calibration pixel map; The JSON dictionary configuration text includes: Total number of parking spaces recorded; A parking space attribute set, where each parking space attribute includes: parking space number identifier, parking space pixel area value, associated video stream identifier, physical space coordinate value, and parking space classification attribute.

6. The method for detecting vacant parking spaces based on maximum matching and parking space segmentation according to claim 1, characterized in that: The specific process of pairwise matching and fusion of parking spaces includes: 1) Parking space coordinate system normalization: Perform an affine transformation on the parking space polygons in the detection result image and the calibration pixel image to unify the coordinate system and record the parameters. After translating the centroid of each polygon to the origin, the polygons are normalized and scaled according to the maximum Euclidean distance from the vertex to the origin. The cross product method is used to normalize the vertex order to a clockwise direction and use the vertex with the minimum x+y as the starting point, where x and y represent pixel coordinates. 2) Parking space vertex feature extraction: Delaunay triangulation is used to extract vertex spacing histograms, the mean and variance of adjacent edge angles, and the ratio of convex hull vertices as vertex features. The polygon is converted into a binary image and the seven logarithmically normalized Hu moments and the first 10 coefficients of the Fourier transform of the contour steering function are calculated as contour features. The two types of features are Min-Max normalized and then fused with a 6:4 weighting. PCA dimensionality reduction is performed to retain 95% variance. 3) Parking space vertex similarity calculation: Vertex distribution differences are calculated based on the intersection-over-union ratio of Thiessen polygons, Hausdorff distance, and Earth Mover's Distance. A comprehensive similarity score is generated by combining the Hu moment weighted Euclidean distance and the L2 norm distance of the contour turning function. A shape complexity factor is introduced for correction. 4) Parking space candidate matching generation: Construct an m×n similarity matrix and retain the top-3 candidates, setting a 0.7 threshold for filtering. Use a bidirectional verification strategy to match parking spaces. For conflicting items, retain the highest-scoring match. Unmatched items are processed using a dynamic time warping relaxation algorithm. Finally, a one-to-one m×n parking space matching matrix is ​​generated, where m represents the number of parking spaces in the calibration pixel image and n represents the number of parking spaces in the detection result image.

7. The method for detecting vacant parking spaces based on maximum matching and parking space segmentation according to claim 1, characterized in that: The step S3 comprises: S31. Obtain the parking space segmentation result set P_seg output by the BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network, and perform a sorting operation on P_seg to generate an ordered queue Queue_seg, including: Main sorting: sort the segmentation results in descending order from high to low confidence values; Secondary sorting: When the confidence values ​​are the same, the parking spaces corresponding to the detection boxes are sorted in descending order from large to small according to their pixel areas; Calculate the queue length value Queue_seg_length of the Queue_seg; S32. Based on the sorted parking space segmentation result queue Queue_seg, create a corresponding matching status queue Queue_result, where: The queue length of Queue_result is the same as Queue_seg; Each parking space status value of the Queue_result Selected from the set {0,1,2,3}, where: 0 means that the corresponding parking space has not started traversal matching; 1 means the corresponding parking space is partially matched; 2 means the corresponding parking space is fully matched; 3 means the corresponding parking space is an invalid detection result; i represents the i-th parking space in the Queue_result queue, 1≤i≤Queue_seg_length; Initialize all parking space status values ​​of the Queue_result to 0; S33. Select parking spaces with a status value of 0 from the matching status queue Queue_result in order as seed nodes ; Based on the pixel coordinates of the parking space, calculate the seed node The intersection-and-union ratio (Ratio_i) of the parking space pixel area and the areas of all parking space pixels in the calibrated parking space pixel map; When Ratio_i>0.9, execute: set the seed node The status value in is updated to 2, and step S34 is executed; When Ratio_i≤0.9, execute: set the seed node The status value in is updated to 3, and step S33 is executed; in: (i)1≤i≤Queue_seg_length; (ii) After the state value is updated to 3, the seed node no longer meets the selection condition of the state value being 0; (iii) the intersection-over-intersection ratio ； S34. Based on the seed node parking space determined in step S33 , do the following: (1) Traversal Each parking space with a corresponding status queue Queue_result status value of 0 or 1 , where i+1≤j≤Queue_seg_length; (2) For each traversed parking space , calculate the seed node Parking space pixel area and Parking space pixel area intersection-to-union ratio Ratio_ij; (3) Update the parking space according to the intersection-combination ratio Ratio_ij Status value: When Ratio_ij>0.8, The status value of the parking space is updated to 3; When 0.4 < Ratio_ij < 0.8, update the status value of the parking space to = 1; When Ratio_ij≤0.4, keep The status value of the parking space remains unchanged; (4) After completing the traversal: When it is detected that there is a parking space with a status value of 0 in Queue_result, return to step S33 to execute the new seed node selection process; When no parking space with a status value of 0 in Queue_result is detected, the parking space matching result is output and the process is terminated; in: (i) The intersection-over-intersection ratio ； (ii) The status value is updated to 3 to indicate an invalid detection result.

8. The method for detecting vacant parking spaces based on maximum matching and parking space segmentation according to claim 1, characterized in that: The parking space status information in S4 includes vacant parking spaces and partially occupied parking spaces; The parking space attribute information includes parking space number, parking space physical coordinates, and parking space type.

Images