An idle parking space detection method based on maximum matching and parking space segmentation

Abstract

The present application relates to the technical field of video image recognition, and particularly relates to an idle parking space detection method based on maximum matching and parking space segmentation, comprising the following steps: S1: calibrating all parking spaces in a picture under the camera visual angle to form a parking space calibration pixel graph and a JSON dictionary configuration file; S2: performing semantic segmentation on the input video stream using a segmentation model, obtaining a detection result graph of the parking space occupancy situation by using a BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion model, and associating the parking space attribute of the calibration pixel graph to the corresponding parking space of the detection result graph through a mapping relationship to generate a detection result graph with attributes; S3: using a multiple priority matching algorithm on the parking space calibration pixel graph and the detection result graph obtained by calibration to filter out misrecognized parking spaces and illegally occupied parking spaces to obtain idle parking spaces and occupied parking spaces; and S4: serializing the detected parking spaces: combining the attribute information and state information of the parking spaces and outputting to a service platform.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video image recognition, and particularly relates to an idle parking space detection method based on maximum matching and parking space segmentation. BACKGROUND

[0002] The number of parking spaces in high-speed service areas and scenic areas is limited. When the traffic volume is large, if the parking spaces are not fully utilized, it may cause serious congestion at the entrance and exit, thereby causing safety accidents. In many high-speed service areas, there are phenomena of illegal parking and one vehicle occupying multiple spaces, which may cause illegal occupation of parking spaces if not timely controlled by management personnel. At the same time, the management personnel of the service area also need to know the number of idle parking spaces in time, so as to improve the management ability of the reception service of the scenic area and the high-speed service area.

[0003] Traditional parking spaces use micro radar detection, infrared detection, ground magnetic induction coil detection and radio frequency identification technology. This kind of method needs to install special sensing equipment for each parking space in the parking lot, which has large engineering cost, is difficult to maintain in later period, and needs to invest a lot of manpower and material resources.

[0004] There are also methods of using camera video stream to recognize the state of parking spaces. For example, a service area square parking space detection system and method based on high point video image technology, with publication number CN118379901B. This patent uses a vehicle type recognition model to judge whether the parked vehicle is an illegally parked vehicle based on the segmentation result, and generates the number of remaining parking spaces of each type of parking space according to the occupancy of each type of parking space. The core of this method is to use two models of segmentation and detection to detect the parked vehicle. The vehicle types are numerous, and the algorithm cannot guarantee that the segmentation and detection algorithms can be completely recognized, so the precision is not high. At the same time, the two models of segmentation and detection are executed in sequence, and the requirement for computing power is also large, so the detection speed is not fast enough. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, the present application provides an idle parking space detection method based on maximum matching and parking space segmentation, so as to solve the problems in the above background.

[0006] The specific method steps of the present application include the following:

[0007] S1: Label the information of all points needing parking space detection in the picture under the camera view, form a parking space calibration pixel graph and a JSON dictionary configuration file.

[0008] S2: semantic segmentation is performed on the input video stream in a parking state using a segmentation model to obtain three continuous images, and a BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network model is used to obtain a detection result image of the parking space occupancy.

[0009] By calibrating the spatial mapping relationship of the pixel map and the detection result image and matching and fusing the parking spaces two by two, the parking space attribute in the calibration pixel map is associated with the parking space region corresponding to the parking space pixel coordinates in the detection result image, and an attribute information detection result image is generated.

[0010] S3: further fuse the parking space state information output by combining the detection result image with the parking space calibration pixel map, filter out the misrecognized parking spaces and illegally occupied parking spaces by using a multi-priority matching algorithm on the calibration parking space pixel map and the detection result image to obtain all the idle parking spaces and occupied parking spaces.

[0011] The multi-priority matching algorithm comprises:

[0012] a) obtaining a parking space segmentation result set and sorting it according to the confidence and the parking space pixel area to generate an ordered queue, and initializing a corresponding state queue.

[0013] b) selecting an unmatched seed parking space from the state queue in order, calculating its intersection-over-union with the calibration pixel parking space, and updating its state according to a preset threshold for determining complete matching: when the intersection-over-union is greater than the threshold, the state is updated to complete matching. Otherwise, the state is updated to invalid.

[0014] c) based on the seed parking space whose state is determined to be completely matched, traversing other unmatched or partially matched parking spaces in the state queue, calculating the intersection-over-union of the seed parking space and these parking spaces, and updating the states of these parking spaces to invalid, partially matched or unchanged according to a preset threshold for determining invalid and a preset threshold for determining partially matched.

[0015] d) iteratively execute the above b and c processes until there is no unmatched parking space in the state queue, and output the final matching result.

[0016] S4: serializing the detected parking spaces: combining the attribute information and the state information of the parking spaces, and outputting to the service platform.

[0017] Preferably, the calibration process in S1 comprises:

[0018] S11. Collecting images of the parking area to be detected, the images being acquired by fixed-view cameras deployed at the parking spots. S12. Providing the images to a labeling personnel, who performs semantic segmentation labeling on all parking spaces in the images, the labeling content including: for the parking spaces not occupied, directly identifying the parking space area; for the parking spaces occupied, based on the parking boundary features, restoring the complete parking contour by imagination, and performing pixel identification. S13. Mapping the semantic segmentation labeling results to the original images by a projection transformation algorithm to generate a parking calibration pixel map.

[0019] Further, the semantic segmentation labeling method comprises:

[0020] (a) Assigning a unique index value to each parking space in the calibration order.

[0021] (b) Setting the three-channel value of the pixel points belonging to the parking space to the index value of the current parking space. (c) Setting the three-channel value of the pixel points in the non-parking space area to a preset background value.

[0022] Wherein, the index value is an integer in the range of 1 to 254, and the index values of different parking spaces are different, and the preset background value is 255.

[0023] Further, the index value assignment order is consistent with the parking number calibration order, and the index value of the kth calibrated parking space is k, where k = 1, 2,..., park_count, park_count is the total number of parking spaces and park_count ≤ 254.

[0024] Preferably, a corresponding JSON dictionary configuration file is established based on the parking calibration pixel map; the JSON dictionary configuration file comprises:

[0025] a parking space total quantity record item;

[0026] a parking space attribute set, wherein each parking space attribute includes: a parking space number identifier, a parking space pixel area value, an associated video stream identifier, a physical space coordinate value, and a parking space classification attribute.

[0027] Preferably, the parking space pairwise matching and fusion specific process comprises:

[0028] 1) Parking coordinate system unification: performing affine transformation on the parking polygons in the detection result map and the calibration pixel map to unify the coordinate system and record the parameters, translating the centroids of the polygons to the origin, then normalizing and scaling according to the maximum Euclidean distance from the vertices to the origin, and using the cross product method to unify the vertex order to the clockwise direction and taking the vertex with the minimum x+y as the starting point, wherein x and y represent pixel coordinates.

[0029] 2) Parking space vertex feature extraction: Extract the vertex distance histogram, the mean and variance of the adjacent edge angle, and the convex hull vertex ratio as vertex features through Delaunay triangulation, convert the polygon to a binary image to calculate the logarithmic normalized value of 7 Hu moments and the first 10 coefficients of the Fourier transform of the contour turning function as contour features, and then Min-Max normalize the two types of features, fuse them according to the weight of 6:4, and reduce the dimension by PCA to retain 95% of the variance.

[0030] 3) Parking space vertex similarity calculation: Calculate the vertex distribution difference based on the intersection and union ratio of the Thiessen polygon, the Hausdorff distance, and the Earth Mover's Distance, combine the Hu moment weighted Euclidean distance and the L2 norm distance of the contour turning function to generate a comprehensive similarity score, and introduce a shape complexity factor for correction.

[0031] 4) Parking space candidate matching generation: Construct an m x n similarity matrix and retain Top-3 candidates with a threshold of 0.7, use a bidirectional verification strategy to match the parking spaces, retain the highest score for conflicting items, and use a dynamic time warping relaxation algorithm to process unmatched items, finally generate an m x n parking space matching matrix, where m represents the number of parking spaces in the calibration pixel graph, and n represents the number of parking spaces in the detection result graph.

[0032] Preferably, the multiple priority matching algorithm in step S3 further comprises:

[0033] S31. Obtain the parking space segmentation result set P_seg output by the BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network, perform an ordering operation on P_seg to generate an ordered queue Queue_seg, including:

[0034] Primary sorting: Sort the segmentation results in descending order from high to low according to the confidence values.

[0035] Secondary sorting: When the confidence values are the same, sort the bounding boxes in descending order from large to small according to the corresponding parking space pixel area.

[0036] Calculate the queue length value Queue_seg_length of Queue_seg.

[0037] S32. Based on the ordered parking space segmentation result queue Queue_seg, create a corresponding matching state queue Queue_result, wherein:

[0038] The queue length of Queue_result is the same as Queue_seg.

[0039] Each parking space state value of Queue_result is selected from the set {0, 1, 2, 3}, wherein:

[0040] 0 represents that the corresponding parking space is not started to be matched;

[0041] 1 represents that the corresponding parking space is partially matched;

[0042] 2 represents that the corresponding parking space is completely matched;

[0043] 3 represents that the corresponding parking space is an invalid detection result;

[0044] i represents the i-th parking space in the Queue_result queue, and 1≤i≤Queue_seg_length.

[0045] Initialize all parking space state values of the Queue_result to 0.

[0046] S33. Select a parking space with a state value of 0 from the matching state queue Queue_result as a seed node in order .

[0047] Based on the pixel coordinates of the parking space, calculate the intersection-over-union Ratio_i of the pixel area of the seed node parking space and all parking space pixel areas in the calibrated parking space pixel map.

[0048] When Ratio_i>0.9, then execute: update the state value in the seed node to 2, and execute step S34;

[0049] When Ratio_i≤0.9, then execute: update the state value in the seed node to 3, and execute step S33.

[0050] wherein:

[0051] (i) 1≤i≤Queue_seg_length;

[0052] (ii) the seed node after the state value is updated to 3 no longer meets the selection condition of the state value being 0;

[0053] (iii) the intersection-over-union .

[0054] S34. Based on the seed node parking space determined in step S33 , execute the following operations:

[0055] (1) Traverse each parking space corresponding to the state queue Queue_result with a state value of 0 or 1 , where \(i + 1\leq j\leq Queue\_seg\_length\).

[0056] (2) For each traversed parking space , calculate the seed node the intersection - over - union ratio \(Ratio_{ij}\) of the pixel area of the parking space and the pixel area of the parking space.

[0057] (3) Update the status value of the parking space according to the intersection - over - union ratio \(Ratio_{ij}\): When \(Ratio_{ij}>0.8\), update the status value of the parking space to 3;

[0058] When \(0.4 < Ratio_{ij}<0.8\), update the status value of the parking space to 1; When \(Ratio_{ij}\leq0.4\), keep the status value of the parking space unchanged.

[0059] (4) After the traversal is completed: When it is detected that there is a parking space with a status value of 0 in \(Queue\_result\), return to step S33 to execute the new seed node selection process.

[0060] When it is not detected that there is a parking space with a status value of 0 in \(Queue\_result\), output the parking space matching result and terminate the process.

[0061] Among them:

[0062] (i) The intersection - over - union ratio

[0063]

[0064]

[0065] (ii) Updating the status value to 3 indicates an invalid detection result.

[0066]

[0067] Preferably, the parking space status information in S4 includes free parking spaces and partially occupied parking spaces.

[0068] The parking space attribute information includes parking space number, parking space physical coordinates, and parking space type.

[0069]

[0070]

[0071] The technical effects and advantages of the present invention:

[0072] 1. High detection accuracy

[0073] By processing consecutive images through the BIFPN_EMA bidirectional cross - scale connection and weighted feature fusion network model, the accuracy of parking space occupancy detection is enhanced, and false detections and missed detections caused by factors such as light changes and occlusions are reduced.

[0072] 2. Data processing is efficient and scalable

[0073] The parking space detection map and the parking space calibration pixel map are overlapped by pixel area to form a matrix, and then a multi-priority matching algorithm is used to simplify the training and inference process of the semantic segmentation model and reduce the computational complexity. The algorithm reduces the requirement for computing power, is cost-effective, and has a faster response speed.

[0074] 3. Strong adaptability

[0075] By fusing the calibration pixel map and the real-time detection result, efficient processing of parking information update is implemented.

[0076] The dynamic threshold design (based on the pixel intersection-over-union value adjustment) ensures the consistency of the detection of large and small parking spaces, and avoids the misjudgment of small parking spaces due to the lack of pixels.

[0077] 4. Convenient system integration

[0078] The output data is serialized (including parking space index, coordinates, category, and state), which can be directly connected to the intelligent parking platform without additional conversion, thereby improving the system response speed. BRIEF DESCRIPTION OF DRAWINGS

[0079] Figure 1 FIG. 1 is a flowchart of an idle parking space detection method based on maximum matching and parking space segmentation;

[0080] Figure 2 FIG. 3 is a schematic diagram of an image without parking space calibration under a camera that needs to detect a parking space;

[0081] Figure 3 FIG. 4 is a schematic diagram after parking space pixel calibration;

[0082] Figure 4 FIG. 5 is a schematic diagram of an FPN structure;

[0083] Figure 5 FIG. 6 is a schematic diagram of a BIFPN_EMA structure;

[0084] Figure 6 FIG. 7 is a schematic diagram of the indicators of a YOLO original model on a test set;

[0085] Figure 7 FIG. 8 is a schematic diagram of the indicators of an optimized YOLO_EMA model on a test set;

[0086] Figure 8 FIG. 9 is a schematic diagram of the detection result obtained after semantic segmentation of a video image in a parking state;

[0087] Figure 9 FIG. 10 is a schematic diagram of the specific steps of S3. DETAILED DESCRIPTION

[0088] With reference to the accompanying drawings: the technical solutions in the embodiments of the present application will be apparently and completely described, obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should belong to the scope of protection of the present application.

[0089] In this embodiment, the idle parking space detection algorithm is deployed on the camera side or the background server side. Deployment on the camera side requires a chip with an algorithm power of 4T or more to detect and run, the camera resolution is 2K or more, the camera installation height is 10 meters, and the camera angle is in a top-down view. Deployment on the server requires a graphics card, and the camera video stream is pushed to the server side to run detection based on the TensorRT acceleration environment.

[0090] This embodiment mainly includes four steps:

[0091] S1: In the camera view, the information of all points needing parking space detection in the picture is calibrated to form a parking space calibration pixel graph P_all and a JSON dictionary configuration file.

[0092] In the camera view of the high-speed service area, the positions of all parking spaces in the picture, the total number of parking spaces, the actual spatial positions of the parking spaces, and the pixel areas of each parking space are calibrated, wherein the positions of the parking spaces in the picture are stored in the form of PNG picture pixels, and the pixel areas of each parking space, the actual spatial positions of the parking spaces, and the total number of parking spaces are stored in the form of a JSON dictionary configuration file.

[0093] First, the picture of the point needing to detect the parking space in the camera view is collected, and the picture is given to the labeling personnel to identify all the parking spaces in the picture by using semantic segmentation (including the occupied parking spaces, which need to be manually drawn), and the identification result is projected onto the picture.

[0094] For an RGB image, set the pixel point P(x, y) as a parking space, modify the three channels of the pixel point to B(x, y)=G(x, y)=R(x, y)=index_of_park (where 1 index_of_park park_count), where x, y are pixel coordinates, BGR represents three channels of the picture, index_of_park represents the index of the current parking space, the index value ranges from 1 to the total number of parking spaces park_count, the total number of parking spaces is at most 254, and the order of the parking space index is determined by the calibration personnel in sequence. For non-parking pixel points P(x, y), the three channels of the pixel points are modified to B(x, y) = G(x, y) = R(x, y) = 255. This step obtains a parking calibration pixel graph.

[0095] The comparison before and after parking calibration is shown in Figure 2 and Figure 3 .

[0096] A corresponding JSON dictionary configuration file is established for the labeled parking calibration pixel graph, and the data structure includes:

[0097] The first-level field:

[0098] park_count: records the total number of parking spaces n in the calibration area; park_values: stores a dictionary set of parking space attributes.

[0099] The second-level field under park_values: parking space number: a unique identifier corresponding to index_of_park; squares: records the pixel area of the corresponding parking space; resp_id: associated video stream identifier; position_x, position_y: record the coordinate position of the parking space in the actual physical space; park_cls: record the classification attribute information of the parking space.

[0100] These basic attribute information is loaded as a configuration file at algorithm initialization.

[0101] S2: using a segmentation model to perform semantic segmentation on the input video stream in the parking state to obtain three consecutive pictures, and using the three consecutive pictures to perform BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network model to obtain a detection result graph P_seg of the parking space occupancy.

[0102] Through the spatial mapping relationship between the calibration pixel graph P_all and the detection result graph P_seg and the two-by-two matching fusion of the parking spaces, the parking space attributes in the calibration pixel graph P_all are associated to the parking space area corresponding to the parking space pixel coordinates in the detection result graph P_seg, to generate a detection result graph with attribute information.

[0103] Due to the large inclination and small size of the parking spaces far away from the high point camera, the current model is modified on the YOLO algorithm architecture to obtain a YOLO_EMA model, as shown in Figure 4 and Figure 5As shown, since the original YOLO model, its core structural feature pyramid network (FPN) is a top-down approach to combine multi-scale features for small parking spaces and inclined parking spaces. In the embodiment YOLO_EMA, the BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network model is adopted, which has higher accuracy and efficiency. The ablation comparison experiment is carried out between the original YOLO model and the YOLO_EMA network model after adopting the BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion. The experimental results prove that compared with the original YOLO model, the average precision Map50 of YOLO_EMA increases by 6.2%, and the inference speed and YOLO model structure are equivalent. The EMA module proposed in the embodiment focuses on reducing the calculation overhead while retaining important information in each channel. The module reshapes the input partial channels to batch dimensions and groups the channel dimensions into multiple sub-features. The purpose of this is to make the spatial semantic features evenly distributed within each feature group, so as to better capture the local features of the data. Specifically, in addition to encoding global information in each parallel branch to recalibrate the weights in the channel dimension, the output features of the two parallel branches are further aggregated through cross-dimensional interaction to capture the relationship between pixel-level pairs.

[0104] As shown in Figure 6 and Figure 7 , the ablation comparison experiment model training set and test set come from the service area high point camera pictures. At present, the training set is 2.2w, and the test set is 4000. From the test indicators of the ablation experiment in the training set and the test set, it can be seen that: 1. Model precision: the original model has an average precision of 86.7% in Map50, and the optimized YOLO_EMA model has an average precision of 92.9% in Map50, which increases by 6.2% in precision. 2. The optimized YOLO_EMA model has faster Loss convergence speed and is more advantageous in iterative optimization. This model is directly deployed in the background service area or the camera side with computing power.

[0105] The effect of semantic segmentation is shown in Figure 8 , it can be seen that the segmented parking pixel result still has the characteristics of sawtooth and irregularity, and pure reliance on the segmented result cannot obtain the attribute information of each parking space and is accompanied by the problems of partial parking redundancy detection and possible misidentification. It is still necessary to combine the pixel calibration chart in step S1 and the configuration file to further match and output the attribute information of each parking space.

[0106] Due to the same camera height and viewing angle, the spatial mapping relationship between the calibration pixel map P_all and the detection result map P_seg is matched and fused, and the parking space attribute in the calibration pixel map P_all is associated with the parking space region corresponding to the parking space pixel coordinates in the detection result map P_seg. Then, through the three-channel pixel value index_of_park on the matched calibration pixel map P_all <x, y> (i.e., the corresponding first index in the Json file), the detection result map P_seg parking space attribute information fusion can be realized.

[0107] The two-by-two matching and fusion of the parking spaces includes the following specific processes:

[0108] 1) Parking space coordinate system unification: check and unify the coordinate systems of the two groups of polygon parking spaces, perform affine transformation conversion, and record the conversion parameters for subsequent verification.

[0109] Normalization processing: calculate the centroid of each polygon and translate it to the origin, and normalize and scale according to the maximum distance from the vertex to the origin.

[0110] Vertex ordering standardization: use the cross product method to determine the vertex order, uniformly adjust to the clockwise direction, and select the vertex with the smallest x+y value as the starting point, where x and y represent pixel coordinates.

[0111] 2) Parking space vertex feature extraction: generate a Voronoi polygon through Delaunay triangulation, calculate the vertex distance histogram (10-bin), adjacent edge angle statistical features (mean, variance), and convex hull vertex ratio.

[0112] Contour feature extraction: convert the polygon to a binary image to calculate 7 Hu moments (normalized after taking the logarithm), extract the contour turning function, and perform Fourier transform on 100 sampling points to obtain the first 10 coefficients.

[0113] Feature fusion: after Min-Max normalization of the two types of features, combine them according to a 6:4 weight, and use PCA dimensionality reduction to retain 95% of the variance information.

[0114] 3) Parking space vertex similarity calculation: measure the vertex distribution difference through the intersection-over-union (IoU) and Hausdorff distance of the Voronoi polygon, and combine the Earth Mover's Distance to measure the overall distribution.

[0115] Contour similarity calculation: compare the weighted Euclidean distance of 7 Hu moments, and calculate the L2 norm distance after aligning the turning function.

[0116] Comprehensive scoring: weight the vertex and contour similarity according to a 0.4:0.6 ratio, and introduce a shape complexity factor (log(N_edges) / 10) for correction.

[0117] 4) Slot candidate matching generation: Construct an mxn similarity matrix (m represents the number of slot calibration pixel maps, and n represents the number of slot detection result maps), reserve Top-3 candidates for each polygon and set an absolute threshold of 0.7 for filtering.

[0118] Bidirectional verification: First, select the best matching A→B, then verify B→A, when there is a conflict, keep the highest score matching and perform a relaxed matching on the unmatched items, and finally generate an m×n slot matching matrix.

[0119] S3: Further fuse slot state information output for the detection result map P_seg and the slot calibration pixel map P_all, and use a multi-priority matching algorithm to filter out misrecognized slots, illegally occupied slots, and obtain all free slots and occupied slots.

[0120] As shown in Figure 9 , the specific steps of the multi-priority matching algorithm in S3 include:

[0121] S31. Sort all slot sets P_seg obtained by the BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network according to confidence from high to low and slot area from large to small to obtain Queue_seg, and calculate the queue length Queue_seg_length of Queue_seg.

[0122] First, the slot with high confidence (m represents the mth segmentation result, 1≤m≤254, considering weather and actual camera height and angle, generally no more than 254 slots are managed under one camera vision, which is consistent with the upper limit of the number of slots in the previous calibration pixel map) is placed in front of Queue_seg, and when the confidence is the same, the slot with larger detection frame slot pixel area is placed in front, which facilitates subsequent processing of slots with higher precision and shorter distance, and finally Queue_seg is obtained.

[0123] S32. According to the sorted Queue_seg, create a corresponding matching state queue Queue_result, There are four states, 0, 1, 2, and 3. Among them, 0 represents that the slot has not started to be traversed and matched, 1 represents that the slot is partially matched, 2 represents that the slot is completely matched, and 3 represents that the slot is an invalid detection result (invalid detection results include misrecognized slots, illegal occupation, etc.). The default value is 0 (the starting state is that all slots have not started to be traversed and matched).

[0124] S33. First select a seed node from the head of the Queue_result queue And Corresponding = 0 (i.e. select the segmentation result with high confidence and not traversed and matched, 0 ), the selected As a seed node, calculate the seed node parking pixel area IOU intersection ratio Ratio_i with all parking pixel areas in the calibration pixel map P_all. When there is a Ratio_i greater than 0.9, it is considered to be an effective free parking space (selected as a seed node parking space), set To 2, and proceed to S34. When there is no Ratio_i greater than 0.9, set To 3, representing that the parking space is an invalid detection result. Start again from step S33 to select a new seed node parking space from the Queue_result queue (the seed node parking space of the current invalid detection result Is changed to 3, which does not meet the condition of = 0, so it will not be selected as the new seed node parking space in the next round).

[0125] Among them, the pixel area IOU intersection ratio Ratio_i is calculated as follows:

[0126] .

[0127] S34. Calculate the pixel overlap area ratio Ratio_ij between As the initial calculation result, and (Where, need to meet = 0 or 1, representing the jth parking space is not traversed and matched or partially matched. i+1≤j≤Queue_seg_length, when i=Queue_seg_length exists in S33, i+1=Queue_seg_length+1, at this time the lower limit of j exceeds the upper limit, naturally forming an empty traversal.), when Ratio_ij is greater than 0.8, it is considered that the parking space is an invalid detection result, representing misidentification or redundant detection, directly set To 3, when Ratio_ij is greater than 0.4 and less than 0.8, it is considered that the parking space is partially occupied, directly set To 1, when Ratio_ij is less than 0.4, skip. The purpose of this step is to use the determined seed node parking space to eliminate redundant detection results and misidentified parking spaces after vehicle occlusion in the segmentation result.

[0128] ​​Wherein, the pixel area IOU intersection over union Ratio_ij is calculated as follows: .

[0129] Finally, the After traversing, if there still exists =0, a new seed parking node is selected in S33, otherwise the process is ended.

[0130] From the multi-priority matching algorithm flow and traversal structure, it can be calculated that the worst time complexity of the algorithm is O(n) and the space complexity is O(3n). Compared with the traditional Hungarian matching algorithm, the worst time complexity is O( ) and the space complexity is O( ). It can be seen that in the matching scene of the pixel map marked in advance, the algorithm has great advantages in memory occupation and algorithm calculation speed compared with the commonly used Hungarian matching algorithm.

[0131] After the multi-priority matching algorithm is executed, The state of each parking space after traversal is stored, including 1 representing that the parking space is partially matched (i.e., partially occupied), 2 representing that the parking space is completely matched (i.e., an idle parking space), and 3 representing that the parking space is an invalid detection result.

[0132] S4: Serializing and outputting the detected parking spaces: combining the attribute information and state information of the parking spaces, and outputting to the service platform.

[0133] The parking space state =2 and =1 obtained in S3 are fused with the attribute information and state information of the P_seg parking spaces, and then reported to the service area management platform.

[0134] The detection result image P_seg parking space in S2 has fused the parking space attribute information obtained by the parking space JSON dictionary configuration file, and the Queue_seg parking space is obtained after the detection result image P_seg parking space in S31 is sorted by the parking space index. Each parking space state information.

[0135] The parking space state information includes an idle parking space and a partially occupied parking space.

[0136] The parking space attribute information includes a parking space number, a parking space physical coordinate, and a parking space type.

[0137] Finally, the above only describes the preferred embodiments of the present application and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting vacant parking spaces based on maximum matching and parking space segmentation, characterized in that, Includes the following steps: S1: Under the camera's view, the information of all points that need to be detected for parking spaces in the image is calibrated to form a parking space calibration pixel map and a JSON dictionary configuration file; S2: Use a segmentation model to perform semantic segmentation on the input video stream in the parking state, obtain three consecutive images, and use the three consecutive images to obtain the detection result image of parking space occupancy through the BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network model. By calibrating the spatial mapping relationship between the pixel image and the detection result image and matching and fusing parking spaces pairwise, the parking space attributes in the calibrated pixel image are associated with the parking space area corresponding to the parking space pixel coordinates in the detection result image, thus generating a detection result image with attribute information. S3: Further integrate the detection result image with the parking space calibration pixel image to output parking space status information. Use a multi-priority matching algorithm to filter out misidentified parking spaces and illegally occupied parking spaces to obtain all available parking spaces and occupied parking spaces. The multi-priority matching algorithm includes: a) Obtain the parking space segmentation result set, sort it according to confidence level and parking space pixel area to generate an ordered queue, and initialize the corresponding state queue; b) Select unmatched seed parking spaces sequentially from the state queue, calculate their intersection-over-union ratio (IoU) with the calibrated pixel parking spaces, and judge and update their state according to a preset threshold for determining a complete match: if the IoU is greater than the threshold, update their state to a complete match; otherwise, update their state to invalid. c) Based on the seed parking space whose state has been determined to be a complete match, traverse the other unmatched or partially matched parking spaces in the state queue, calculate the intersection-union ratio of the seed parking space with these parking spaces, and update the state of these parking spaces to invalid, partially matched or unchanged according to a preset threshold for determining invalidity and a preset threshold for determining partial match. d) Iterate through steps b and c above until there are no unmatched parking spaces in the state queue, and output the final matching result; S4: Serialize the detected parking spaces: Combine the attribute and status information of the parking spaces and output them to the service platform.

2. The vacant parking space detection method based on maximum matching and parking space segmentation according to claim 1, characterized in that, The calibration process in S1 includes: S11. Acquire an image of the parking space area to be detected, the image being acquired by a fixed-view camera deployed at the parking space location; S12. Provide the image to the annotator, who performs semantic segmentation and annotation on all parking spaces in the image. The annotation includes: for unoccupied parking spaces, directly identifying the parking space area; for occupied parking spaces, reconstructing the complete parking space outline based on the parking space boundary features and marking it with pixels; S13. Map the semantic segmentation and annotation results to the original image using a projection transformation algorithm to generate a parking space calibration pixel map.

3. The vacant parking space detection method based on maximum matching and parking space segmentation according to claim 2, characterized in that, The semantic segmentation and annotation method includes: (a) Assign a unique index value to each parking space in the specified order; (b) Set the three-channel values ​​of the pixels belonging to the parking space to the index value of the current parking space; (c) Set the three-channel values ​​of pixels in non-parking areas to the preset background values; The index value is an integer in the range of 1 to 254, and the index values ​​of different parking spaces are different from each other. The preset background value is 255.

4. The vacant parking space detection method based on maximum matching and parking space segmentation according to claim 3, characterized in that: The index values ​​are assigned in the same order as the parking space numbering. The index value of the kth marked parking space is k, where k = 1, 2, ..., park_count, and park_count is the total number of parking spaces and park_count ≤ 254.

5. The vacant parking space detection method based on maximum matching and parking space segmentation according to claim 2, characterized in that, Establish a corresponding JSON dictionary configuration file based on the pixel map of the parking space calibration; The JSON dictionary configuration file includes: Total number of parking spaces recorded; A set of parking space attributes, where each parking space attribute includes: parking space number identifier, parking space pixel area value, associated video stream identifier, physical space coordinate value, and parking space category attribute.

6. The method for detecting vacant parking spaces based on maximum matching and parking space segmentation according to claim 1, characterized in that, The specific process of matching and merging parking spaces in pairs includes: 1) Unification of parking space coordinate system: Perform affine transformation on the parking space polygons in the detection result image and calibration pixel image to unify the coordinate system and record the parameters. After translating the centroid of each polygon to the origin, normalize and scale according to the maximum Euclidean distance from the vertex to the origin. Use the cross product method to unify the vertex order to a clockwise direction and take the vertex with the minimum x+y as the starting point, where x and y represent pixel coordinates. 2) Parking space vertex feature extraction: The vertex spacing histogram, the mean and variance of the angle between adjacent sides, and the ratio of convex hull vertices are extracted by Delaunay triangulation as vertex features. The polygon is converted into a binary image and the log-normalized values ​​of 7 Hu moments and the first 10 coefficients of the Fourier transform of the contour turning function are calculated as contour features. The two types of features are min-max normalized respectively and then fused with a weight of 6:

4. Dimensionality reduction is performed by PCA to retain 95% of the variance. 3) Parking space vertex similarity calculation: The vertex distribution difference is calculated based on the intersection-union ratio of Thiessen polygons, Hausdorff distance and Earth Mover's Distance. The comprehensive similarity score is generated by combining Hu moment weighted Euclidean distance and L2 norm distance of the contour turning function. The shape complexity factor is introduced for correction. 4) Parking space candidate matching generation: Construct an m×n similarity matrix and retain the top-3 candidates with a 0.7 threshold for filtering. Use a two-way verification strategy to match parking spaces. For conflicting items, retain the highest score match. Unmatched items are processed by a dynamic time warping relaxation algorithm. Finally, generate a one-to-one m×n parking space matching matrix, where m represents the number of parking spaces in the calibrated pixel image and n represents the number of parking spaces in the detection result image.

7. The method for detecting vacant parking spaces based on maximum matching and parking space segmentation according to claim 1, characterized in that, The multi-priority matching algorithm in step S3 further includes: S31. Obtain the parking space segmentation result set P_seg output by the BIFPN_EMA bidirectional cross-scale connection and weighted feature fusion network, and perform a sorting operation on P_seg to generate an ordered queue Queue_seg, including: Main sorting: Sort the segmentation results in descending order of confidence scores from highest to lowest; Secondary sorting: When the confidence values ​​are the same, sort them in descending order according to the pixel area of ​​the parking space corresponding to the detection box; Calculate the queue length value of Queue_seg, Queue_seg_length; S32. Based on the sorted parking space segmentation result queue Queue_seg, create a corresponding matching status queue Queue_result, where: The queue length of Queue_result is the same as that of Queue_seg; The status value of each parking space in the Queue_result Selected from the set {0,1,2,3}, where: 0 indicates that the corresponding parking space has not yet been traversed and matched; 1 indicates that the corresponding parking space was partially matched; 2 indicates that the corresponding parking space is fully matched; 3 indicates that the corresponding parking space is an invalid detection result; i represents the i-th parking space in the Queue_result queue, 1≤i≤Queue_seg_length; Initialize the state value of all parking spaces in the Queue_result to 0; S33. Select parking spaces with a status value of 0 as seed nodes from the matching status queue Queue_result in order. ; The seed node is calculated based on the pixel coordinates of the parking space. The intersection-union ratio (Ratio_i) of the parking space pixel area with the areas of all parking space pixels in the calibrated parking space pixel map; When Ratio_i > 0.9, then execute: change the seed node. The state value in the data is updated to 2, and step S34 is executed; When Ratio_i ≤ 0.9, then execute: seed node The state value in the data is updated to 3, and step S33 is executed. in: (i)1≤i≤Queue_seg_length; (ii) The seed node whose state value is updated to 3 no longer meets the selection condition of a state value of 0; (iii) The crossover ratio ； S34. Parking spaces based on the seed node determined in step S33 Perform the following operations: (1) Traversal Each parking space corresponds to a state queue (Queue_result) with a state value of 0 or 1. Where i+1≤j≤Queue_seg_length; (2) For each parking space that is traversed Calculate seed node Parking space pixel area and Parking space pixel area intersection-union ratio Ratio_ij; (3) Update the parking spaces according to the intersection-union ratio Ratio_ij. State values: When Ratio_ij > 0.8, The parking space's status value has been updated to 3; When 0.4 < Ratio_ij < 0.8, update the status value of the parking space to 1; When Ratio_ij ≤ 0.4, maintain The parking space's status value remains unchanged; (4) After completing the traversal: When a parking space with a status value of 0 is detected in Queue_result, return to step S33 to execute the new seed node selection process; If no parking space with a status value of 0 is detected in Queue_result, output the parking space matching result and terminate the process; in: (i) The crossover ratio ； (ii) The status value is updated to 3 to indicate an invalid detection result.

8. The vacant parking space detection method based on maximum matching and parking space segmentation according to claim 1, characterized in that, The parking space status information mentioned in S4 includes vacant parking spaces and partially occupied parking spaces; The parking space attribute information includes the parking space number, the parking space physical coordinates, and the parking space type.

Images