A method for accurately monitoring vehicle parking violations and littering

Abstract

The present application relates to the field of video monitoring, and relates to a method for accurately monitoring vehicle illegal parking and disorderly parking, comprising the following steps: 1) setting a standard parking direction range, an illegal parking time threshold value, and a plurality of parking areas, and establishing a vehicle illegal parking database and a vehicle disorderly parking database; 2) obtaining a vehicle information data set at a previous time; 3) obtaining a vehicle information data set at a current time; 4) using a Hungarian algorithm to obtain a maximum matching set of the vehicle information data sets at adjacent two times; 5) determining the position coordinates of the vehicle at the current time according to the maximum matching set; 6) judging whether each vehicle at the current time is illegally parked or disorderly parked; and 7) repeating steps 2) to 6). On the basis of similarity judgment of traditional IOU and REID features, more feature data are introduced to calculate the similarity of vehicles, so that the same vehicle can be accurately identified in various complex environments, and the accuracy of judging vehicle illegal parking and disorderly parking is improved.

Description

Technical Field

[0001] This invention relates to the field of video surveillance technology, specifically to a method for accurately monitoring illegally parked or improperly placed vehicles. Background Technology

[0002] With the development of society and the economy, the number of vehicles in China has been increasing year by year, and illegal parking and disorderly parking of motor vehicles are frequent occurrences. Although ordinary streets and public parking lots are equipped with ordinary video surveillance, because the cameras do not have the corresponding AI functions, enforcement can only be carried out by manual on-site evidence collection. This results in information lag, easy omissions, and a lot of manpower and resources being wasted, but without achieving good regulatory results.

[0003] To provide timely warnings about illegal parking of motor vehicles, reduce the burden on on-duty personnel, and improve the appearance of the city, the patent application CN202111153392.1, entitled "A Method for Identifying Illegal Parking of Vehicles in Complex Application Scenarios," although it achieves the integration of cloud computing and deep learning through a combination of CNN+OpenCV architecture recognition, deep detection models, and IOU tracking algorithms, and can judge the illegal parking status of vehicles, often suffers from a high false alarm rate due to the complex environment of the vehicles, the large monitoring range, and the fact that most vehicles have similar shapes. Using only the Intersection-over-Union (IoU) method in object detection for similarity judgment makes it difficult to distinguish whether vehicles at the same location at the current moment and the previous moment are the same vehicle.

[0004] While the "A Method and System for Detecting Illegal Parking of Vehicles" with application number CN202111513371.6 can identify dynamic vehicles based on the contour recognition results of road condition images, perform feature recognition on dynamic vehicles, randomly acquire first-line images, and generate illegal parking information, its ability to represent targets using contour features is insufficient, resulting in poor model adaptability to the environment and an increase in missed detections and false detections.

[0005] In summary, the existing vehicle similarity measurement standards are relatively simple. If multiple vehicles with high similarity need to be judged and matched at the same time, it is easy to judge different vehicles in complex environments as the same vehicle, or the same vehicle at different times as different vehicles, which will lead to false alarms or false alarms. There is an urgent need for a method that can accurately identify the same vehicle in various complex environments in order to improve the accuracy of judging illegally parked vehicles. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by providing a method for accurately monitoring illegally parked and haphazardly parked vehicles. Based on the traditional similarity judgment using IOU and REID features, this invention introduces more feature data to calculate vehicle similarity, enabling accurate identification of the same vehicle in various complex environments, thereby improving the accuracy of judging illegally parked and haphazardly parked vehicles.

[0007] The objective of this invention is achieved through the following solution: a method for accurately monitoring illegally parked or improperly placed vehicles, comprising the following steps:

[0008] 1) Set standard parking direction range, illegal parking time threshold, and several parking areas, and establish a database of illegally parked vehicles and a database of improperly parked vehicles;

[0009] 2) Number all vehicles in the previous time step, and extract at least three corresponding feature data for each vehicle through a neural network model. Combine the corresponding position coordinates of each vehicle to obtain the vehicle information corresponding to each vehicle in the previous time step, forming the vehicle information dataset of the previous time step.

[0010] 3) Number all vehicles at the current moment, and extract at least three corresponding feature data for each vehicle through a neural network model. Combine the corresponding position coordinates of each vehicle to obtain the vehicle information corresponding to each vehicle at the current moment, forming the vehicle information dataset for the current moment.

[0011] 4) Using the Hungarian algorithm, the vehicle information of all vehicles at two adjacent time points in steps 2) and 3) is paired one by one according to the similarity of the feature data to obtain the maximum matching set of the vehicle information datasets at two adjacent time points.

[0012] 5) Based on the maximum matching set, the current position coordinates of each vehicle with the corresponding number at the previous time are determined by the number at the current time.

[0013] 6) Based on the current location coordinates of each vehicle, as well as the preset parking area and illegal parking time threshold, determine whether each vehicle is illegally parked at the current moment:

[0014] ① If a vehicle is illegally parked, an illegal parking number is generated for the vehicle and stored in the vehicle illegal parking database along with the vehicle information;

[0015] ② If the vehicle is not illegally parked, determine whether the vehicle is parked haphazardly based on the standard parking direction range. If the vehicle is parked haphazardly, generate a haphazard parking number for the vehicle and store it together with the vehicle information in the vehicle haphazard parking database.

[0016] 7) After judging all vehicles at the current moment, complete one loop and repeat steps 2) to 6) to loop.

[0017] Preferably, the specific method for determining whether each vehicle is illegally parked at the current moment is as follows:

[0018] ① When the location coordinates of any vehicle are outside the preset parking area, record the current time as the first suspected illegal parking time of the vehicle, until the location coordinates of the vehicle are within the parking area, record the current time as the last suspected illegal parking time of the vehicle;

[0019] ②If the difference between the final suspected illegal parking time and the initial suspected illegal parking time of the vehicle is greater than the preset illegal parking time threshold, then the vehicle is an illegally parked vehicle.

[0020] Preferably, the feature data includes directional features. If the vehicle is not illegally parked, the specific steps for determining whether the vehicle is parked haphazardly based on the standard parking direction range are as follows:

[0021] ① Determine the parking direction of any vehicle based on its current directional characteristics;

[0022] ②If the vehicle is parked in a direction outside the standard parking direction range, then the vehicle is considered to be parked haphazardly.

[0023] ③ If the vehicle is parked in the direction of the standard parking direction, then the vehicle is not parked haphazardly.

[0024] Preferably, each element in the maximum matching set includes the ID corresponding to the same vehicle at the previous moment, as well as the ID and location coordinates corresponding to the current moment.

[0025] Preferably, the steps of the Hungarian algorithm in step 4) are as follows:

[0026] ① Set a confidence classification threshold and calculate the confidence level of each vehicle at the current time. Classify the vehicle information of each vehicle at the current time according to the confidence classification threshold to obtain a set of high-confidence vehicles and a set of low-confidence vehicles.

[0027] ② Calculate the vehicle information similarity matrix between the high-confidence vehicle set and the vehicle information dataset of the previous time step. By recursively finding augmenting paths, pair the vehicle information with similar feature data in the high-confidence vehicle set with the vehicle information dataset of the previous time step one by one to obtain the high-confidence matching set. The vehicle information in the vehicle information dataset of the previous time step that did not participate in the matching is used as the second round of matching set.

[0028] ③ Calculate the vehicle information similarity matrix between the second-round matching set and the low-confidence vehicle set. By recursively finding augmenting paths, pair the vehicle information with similar feature data in the second-round matching set and the low-confidence vehicle set one by one to obtain the low-confidence matching set.

[0029] ④ Merge the high-confidence matching set with the low-confidence matching set to obtain the maximum matching set.

[0030] Preferably, the vehicle information similarity matrix is ​​calculated as follows:

[0031] ① Calculate the cosine similarity between the feature data of the two sets using the following formulas:

[0032]

[0033] In the formula, cosθ represents the cosine similarity. For the i-th feature data in one of the sets, For the j-th feature data in another set;

[0034] ② Use the cosine similarities obtained in step ① to form a feature similarity matrix that reflects the degree of similarity of feature data in two sets;

[0035] ③ Obtain the bounding box of each vehicle based on its position coordinates, and calculate the intersection-union ratio of the bounding boxes of each vehicle in the two sets according to the following formula:

[0036]

[0037] In the formula, S iou S is the intersection-union ratio of the rectangles. C S is the intersection of the rectangular bounding boxes of the two vehicles. D The union of the rectangular bounding boxes of the two vehicles;

[0038] ④ Using the intersection-union ratio of several rectangles obtained in step ③, a position similarity matrix is ​​formed to reflect the degree of similarity between the vehicle positions in the two sets;

[0039] ⑤ Add the product of the feature similarity matrix and the weights of the feature similarity to the product of the position similarity matrix and the weights of the position similarity to obtain the vehicle information similarity matrix of the two sets.

[0040] Preferably, the intersection of the rectangular frames of the two vehicles is calculated according to the following formula:

[0041]

[0042] In the formula, S C The intersection of the rectangular bounding boxes of the two vehicles. and These are the coordinates of two non-adjacent vertices within the rectangle containing one of the vehicles. and Let each be a non-adjacent vertex within the rectangle containing the other car, and let the point be... point Points are vertices in the same direction as the rectangle. point These are the vertices of the rectangle in the same direction;

[0043] The union of the rectangular frames of the two vehicles is calculated according to the following formula:

[0044]

[0045] In the formula, S D Let the rectangular bounding boxes of the two vehicles be the union. and These are the coordinates of two non-adjacent vertices within the rectangle containing one of the vehicles. and Let each be a non-adjacent vertex within the rectangle containing the other car, and let the point be... point Points are vertices in the same direction as the rectangle. point These are the vertices of the rectangle in the same direction.

[0046] Preferably, the vehicle information includes the vehicle's feature data and location coordinates.

[0047] Preferably, the confidence level is obtained by a video acquisition unit using the YOLOv5 algorithm, and the video acquisition unit using the YOLOv5 algorithm can also obtain the vehicle's position coordinates.

[0048] Preferably, the feature data includes the vehicle's Re-ID features, color features, direction features, and vehicle type features;

[0049] Preferably, the Re-ID feature is obtained by the Re-ID feature recognition branch module, the color feature is obtained by the color feature recognition branch module, the direction feature is obtained by the direction feature recognition branch module, and the vehicle type feature is obtained by the vehicle type feature recognition branch module. The Re-ID feature recognition branch module, the color feature recognition branch module, the direction feature recognition branch module, and the vehicle type feature recognition branch module, together with the backbone network, form a neural network model for extracting feature data.

[0050] The advantages of this invention are as follows:

[0051] ① Based on the traditional IOU and REID features for similarity judgment, more feature data such as color features, direction features, and vehicle type features are introduced to calculate vehicle similarity. This method can accurately identify the same vehicle in various complex environments, thereby improving the accuracy of judging illegally parked vehicles.

[0052] ② After initial screening of vehicle data using confidence level, Hungarian matching is then performed to avoid the low quality of the original vehicle information affecting the final judgment result, which greatly improves the accuracy of monitoring illegal parking and disorderly parking. Attached Figure Description

[0053] Figure 1 This is a flowchart of the present invention;

[0054] Figure 2 This is a schematic diagram of the video acquisition unit processing the monitoring screen in this invention. Detailed Implementation

[0055] like Figure 1 As shown, a method for accurately monitoring illegally parked or improperly parked vehicles includes the following steps:

[0056] 1) Set standard parking direction range, illegal parking time threshold, and several parking areas, and establish a database of illegally parked vehicles and a database of improperly parked vehicles;

[0057] 2) Number all vehicles in the previous time step, and extract at least three corresponding feature data for each vehicle through a neural network model. Combine the corresponding position coordinates of each vehicle to obtain the vehicle information corresponding to each vehicle in the previous time step, forming the vehicle information dataset of the previous time step.

[0058] 3) Number all vehicles at the current moment, and extract at least three corresponding feature data for each vehicle through a neural network model. Combine this with the location coordinates of each vehicle to obtain the vehicle information for each vehicle at the current moment, forming the vehicle information dataset for the current moment. That is, the vehicle information includes the vehicle's feature data and location coordinates. The feature data includes the vehicle's Re-ID feature, color feature, direction feature, and vehicle type feature. The Re-ID feature is obtained by the Re-ID feature recognition branch module, the color feature by the color feature recognition branch module, the direction feature by the direction feature recognition branch module, and the vehicle type feature by the vehicle type feature recognition branch module. The Re-ID feature recognition branch module, the color feature recognition branch module, the direction feature recognition branch module, and the vehicle type feature recognition branch module, together with the backbone network ResNet50-IBN-a, form a neural network model for extracting feature data. That is, the neural network model is formed by adding the color feature recognition branch module, the direction feature recognition branch module, and the vehicle type feature recognition branch module to the BoT-BS structure vehicle REID network, together with the original Re-ID feature recognition branch module. The BoT-BS structure is the BOT-BS bag of tricks-baseline.

[0059] The last layer of the backbone network ResNet50-IBN-a has a convolution stride parameter of 1. The output head of the Re-ID feature recognition branch module consists of an adaptive average pooling layer, a one-dimensional batch normalization layer, and a ReID fully connected layer. The ReID fully connected layer has 2048 input channels and the number of output channels is the number of REID categories. The Re-ID feature before the one-dimensional batch normalization layer is defined as f. r That is, the video acquisition unit inputs the RGB image into the Re-ID feature recognition branch module, which can obtain a 2048-dimensional Re-ID feature f. r ;

[0060] The output head of the color feature recognition branch module consists of an adaptive average pooling layer, a first color attribute fully connected layer, and a second color attribute fully connected layer. The first color attribute fully connected layer has 2048 input channels and 512 output channels, while the second color attribute fully connected layer has 512 input channels and 13 output channels. The output feature of the first color attribute fully connected layer is defined as color feature f. c When the video acquisition unit inputs an RGB image into the color feature recognition branch module, it can obtain a 512-dimensional color feature f. c For example, color features f can be set during model training. c The correspondence between the values ​​and colors is as follows:

[0061] {0: "Black", 1: "White", 2: "Gray", 3: "Brown", 4: "Red", 5: "Orange", 6: "Yellow", 7: "Green", 8: "Blue", 9: "Pink", 10: "Purple", 11: "Champagne", 12: "Floral"}

[0062] The output head of the directional feature recognition branch module consists of an adaptive average pooling layer, a first directional attribute fully connected layer, and a second directional attribute fully connected layer. The first directional attribute fully connected layer has 2048 input channels and 512 output channels, while the second directional attribute fully connected layer has 512 input channels and 8 output channels. The output feature of the first directional attribute fully connected layer is defined as the directional feature f. o When the video acquisition unit inputs an RGB image into the orientation feature recognition branch module, it can obtain a 512-dimensional orientation feature f. o For example, during model training, the orientation can be divided at 45° intervals, and the correspondence between the values ​​of the orientation features and the orientation can be set as follows: 0° corresponds to "0", 45° corresponds to "1", 90° corresponds to "2", 135° corresponds to "3", 180° corresponds to "4", 225° corresponds to "5", 270° corresponds to "6", and 315° corresponds to "7".

[0063] The output head of the vehicle type feature recognition branch module consists of an adaptive average pooling layer, a first fully connected vehicle type layer, and a second fully connected vehicle type layer. The first fully connected vehicle type layer has 2048 input channels and 512 output channels, while the second fully connected vehicle type layer has 512 input channels and 6 output channels. The output feature of the first fully connected vehicle type layer is defined as the vehicle type feature f. t When the video acquisition unit inputs the RGB image into the vehicle type feature recognition branch module, it will obtain a 512-dimensional vehicle type feature f. t For example, vehicle type features f can be set during model training. t The correspondence between the values ​​of and vehicle type is as follows:

[0064] {0: "Sedan", 1: "SUV", 2: "Mercedes-Benz", 3: "Sports Car", 4: "Minibus", 5: "Pickup Truck"}

[0065] 4) Using the Hungarian algorithm, the vehicle information of all vehicles at two adjacent time points in steps 2) and 3) is paired one-to-one according to the similarity of the feature data to obtain the maximum matching set of the vehicle information datasets at two adjacent time points:

[0066] 4-1) Set the confidence classification threshold to 0.6 and calculate the confidence level of each vehicle at the current time. Classify the vehicle information of each vehicle at the current time according to the confidence classification threshold. Usually, based on the confidence level of each vehicle at the current time, the vehicle information of each vehicle at the current time can be divided into two or three categories, or even four or five categories. The number of categories is determined through simulation experiments or calibration experiments to meet the required monitoring accuracy.

[0067] In this embodiment, vehicle information corresponding to vehicles with a confidence level ≥ 0.6 is used to form a high-confidence vehicle set, and vehicle information corresponding to vehicles with a confidence level ≥ 0.1 (0.6 > 0.6) is used to form a low-confidence vehicle set. Vehicle information corresponding to vehicles with a confidence level less than 0.1 is of low quality, possibly because the RGB images used to extract vehicle information are blurry, or because key parts of the vehicle in the image are obscured. This type of vehicle information needs to be discarded and not included in the matching calculation to avoid incorrect matching, which would affect the final judgment result. This improves the accuracy of monitoring illegally parked and disorderly parking vehicles. The confidence level classification threshold is usually determined based on specific circumstances through simulation experiments or calibration experiments.

[0068] The confidence level is obtained by a video acquisition unit using the YOLOv5 algorithm. This video acquisition unit, also using the YOLOv5 algorithm, can also obtain the vehicle's position coordinates. Typically, a video surveillance system is used as the video acquisition unit, and a vehicle detection model is built using the YOLOv5 algorithm. Figure 2 As shown, after the video surveillance system captures a monitoring image at a certain moment, it obtains the position coordinates of each vehicle on the monitoring image through a vehicle detection model, and generates an RGB image corresponding to each vehicle for extracting vehicle information.

[0069] 4-2) Calculate the vehicle information similarity matrix between the high-confidence vehicle set and the vehicle information dataset of the previous time step. By recursively finding augmenting paths, pair the vehicle information with similar feature data in the high-confidence vehicle set with the vehicle information dataset of the previous time step one by one to obtain the high-confidence matching set. The vehicle information in the vehicle information dataset of the previous time step that did not participate in the matching is used as the second round of matching set.

[0070] 4-3) Calculate the vehicle information similarity matrix between the second-round matching set and the low-confidence vehicle set. By recursively finding augmenting paths, pair the vehicle information with similar feature data in the second-round matching set and the low-confidence vehicle set one by one to obtain the low-confidence matching set.

[0071] 4-4) Merge the high-confidence matching set with the low-confidence matching set to obtain the maximum matching set.

[0072] In this embodiment, the calculation method for the vehicle information similarity matrix between the high-confidence vehicle set and the vehicle information dataset of the previous time step is the same as the calculation method for the vehicle information similarity matrix between the second-round matching set and the low-confidence vehicle set. The specific steps are as follows:

[0073] ① Calculate the cosine similarity between the same feature data of the two sets according to the following formula:

[0074]

[0075] In the formula, cosθ represents the cosine similarity. For the i-th feature data in one of the sets, For the j-th feature data in another set;

[0076] ② Using the cosine similarities obtained in step ①, a feature similarity matrix is ​​constructed to represent the degree of similarity of feature data in the two sets, namely:

[0077] a) Calculate several cosine similarities based on the Re-ID features in the two sets to obtain the Re-ID feature similarity matrix D. id ;

[0078] b) Calculate several cosine similarities based on the color features in the two sets to obtain the color feature similarity matrix D. color ;

[0079] c) Calculate several cosine similarities based on the directional features in the two sets to obtain the directional feature similarity matrix D. orientation ;

[0080] d) Calculate several cosine similarities based on the vehicle type features in the two sets to obtain the vehicle type feature similarity matrix D. type ;

[0081] ③ Obtain the bounding box of each vehicle based on its position coordinates, and calculate the intersection-union ratio of the bounding boxes of each vehicle in the two sets according to the following formula:

[0082]

[0083] In the formula, S iou S is the intersection-union ratio of the rectangles. C S is the intersection of the rectangular bounding boxes of the two vehicles. D The union of the rectangular bounding boxes of the two vehicles;

[0084] In this embodiment, the coordinates of each vertex of the rectangle corresponding to each vehicle in the monitoring video are determined by the vehicle's position coordinates (x, y, w, h). In the position coordinates, x is the horizontal coordinate of the top-left vertex of the vehicle's rectangle in the monitoring screen, y is the vertical coordinate of the top-left vertex of the vehicle's rectangle in the monitoring screen, w is the width of the vehicle's rectangle, and h is the height of the vehicle's rectangle. The intersection of the rectangles of two vehicles is calculated according to the following formula:

[0085]

[0086] In the formula, S C The intersection of the rectangular bounding boxes of the two vehicles. and These are the coordinates of two non-adjacent vertices within the rectangle containing one of the vehicles. and Let each be a non-adjacent vertex within the rectangle containing the other car, and let the point be... point Points are vertices in the same direction as the rectangle. point These are the vertices of the rectangle in the same direction;

[0087] The union of the rectangular frames of the two vehicles is calculated according to the following formula:

[0088]

[0089] In the formula, S D Let the rectangular bounding boxes of the two vehicles be the union. and These are the coordinates of two non-adjacent vertices within the rectangle containing one of the vehicles. and Let each be a non-adjacent vertex within the rectangle containing the other car, and let the point be... point Points are vertices in the same direction as the rectangle. point These are the vertices of the rectangle in the same direction.

[0090] For example, if the coordinates of the top-left vertex of the rectangle A corresponding to the a-th car are... The coordinates of the bottom right vertex are The coordinates of the top-left vertex of the rectangle B corresponding to car b are: The coordinates of the bottom right vertex are Then the intersection-union ratio S of rectangles A and B is... iou for:

[0091]

[0092] ④ Using the intersection-union ratio of several rectangles obtained in step ③, a position similarity matrix D is constructed to represent the degree of similarity between the vehicle positions in the two sets. iou ;

[0093] ⑤ Add the product of the feature similarity matrix and the weights of the feature similarity to the product of the position similarity matrix and the weights of the position similarity to obtain the vehicle information similarity matrix D for the two sets:

[0094] D=λ*D id +φ*D iou +α*D color +β*D type -γ*D orientation

[0095] In the formula, D id Let D be the Re-ID feature similarity matrix, where λ is the weight of the Re-ID feature similarity, and D is the weight of the Re-ID feature similarity. iou Let D be the positional similarity matrix, where φ is the weight of the positional similarity, and D is the positional similarity matrix. color Let D be the color feature similarity matrix, where α is the weight of the color feature similarity, and D is the color feature similarity matrix. type Let D be the vehicle type feature similarity matrix, where β is the weight of the vehicle type feature similarity, and D is the vehicle type feature similarity matrix. orientation Let be the directional feature similarity matrix, and γ be the weight of the directional feature similarity.

[0096] 5) Based on the maximum matching set, the current position coordinates of each vehicle with the corresponding number at the previous time are determined by the number at the current time.

[0097] In this embodiment, each element in the maximum matching set includes the number corresponding to the previous moment of the same vehicle, as well as the number and position coordinates corresponding to the current moment. That is, based on the feature data of the same vehicle, the two numbers corresponding to the vehicle at two adjacent moments can be paired, and the position coordinates corresponding to the vehicle at the current moment can be determined using the number at the current moment.

[0098] 6) Based on the current location coordinates of each vehicle and the preset parking area and illegal parking time threshold, determine whether each vehicle is illegally parked at the current moment using the following methods:

[0099] 6-1) When the location coordinates of any vehicle are outside the preset parking area, record the current time as the first suspected illegal parking time of the vehicle, until the location coordinates of the vehicle are within the parking area, record the current time as the last suspected illegal parking time of the vehicle.

[0100] 6-2) If the difference between the final suspected illegal parking time and the initial suspected illegal parking time of the vehicle is greater than the preset illegal parking time threshold, then the vehicle is an illegally parked vehicle.

[0101] If a vehicle is illegally parked, an illegal parking number is generated for that vehicle and stored in the vehicle illegal parking database along with the vehicle's information.

[0102] If the vehicle is not illegally parked, determine whether the vehicle is parked haphazardly based on the standard parking direction range of the vehicle using the following methods:

[0103] ① Determine the parking direction of any vehicle based on its current directional characteristics;

[0104] ②If the vehicle is parked in the direction of the standard parking direction, then the vehicle is not parked haphazardly;

[0105] ③ If the vehicle is parked in a direction that is not within the standard parking direction range, then the vehicle is considered to be parked haphazardly.

[0106] If a vehicle is parked haphazardly, a haphazard parking number is generated for that vehicle and stored in the haphazard parking database along with the vehicle's information.

[0107] 7) After judging all vehicles at the current moment, one loop is completed. Repeat steps 2) to 6) at the next moment to continue the loop.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications made to the present invention by those skilled in the art without departing from the spirit of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for accurately monitoring vehicle parking violations and disorderly placement, characterized in that, The method comprises the following steps: 1) setting a standard parking direction range, a parking violation time threshold, and a plurality of parking areas, and establishing a vehicle parking violation database and a vehicle random placement database; 2) numbering all vehicles at the previous time, and extracting at least three corresponding feature data of each vehicle through a neural network model, combining the position coordinates corresponding to each vehicle, obtaining the vehicle information corresponding to each vehicle at the previous time, and forming a vehicle information data set at the previous time; 3) numbering all vehicles at the current time, and extracting at least three corresponding feature data of each vehicle through a neural network model, combining the position coordinates corresponding to each vehicle, obtaining the vehicle information corresponding to each vehicle at the current time, and forming a vehicle information data set at the current time, wherein the feature data comprises Re-ID features, color features, direction features, and vehicle type features of the vehicle; 4) using the Hungarian algorithm in the following manner, pairing all vehicle information of vehicles at adjacent two times in steps 2) and 3) according to the similarity of the feature data to obtain a maximum matching set of the vehicle information data sets at adjacent two times: ① setting a confidence classification threshold, and calculating the confidence of each vehicle at the current time, classifying the vehicle information of each vehicle at the current time according to the confidence classification threshold to obtain a high-confidence vehicle set and a low-confidence vehicle set; ② calculating the vehicle information similarity matrix of the high-confidence vehicle set and the vehicle information data set at the previous time, and pairing the vehicle information with similar feature data in the high-confidence vehicle set and the vehicle information data set at the previous time through recursive search for an augmented path to obtain a high-confidence matching set, and taking the vehicle information in the vehicle information data set at the previous time which does not participate in matching as a second round matching set; ③ calculating the vehicle information similarity matrix of the second round matching set and the low-confidence vehicle set, and pairing the vehicle information with similar feature data in the second round matching set and the low-confidence vehicle set through recursive search for an augmented path to obtain a low-confidence matching set; ④ combining the high-confidence matching set and the low-confidence matching set to obtain a maximum matching set; 5) according to the maximum matching set, the position coordinates of each numbered vehicle at the current time are determined by the corresponding number at the current time; 6) judging whether each vehicle at the current time is parked in violation of the regulations according to the position coordinates of each vehicle at the current time and the preset parking area and parking violation time threshold: ① if the vehicle is parked in violation of the regulations, generating a parking violation number of the vehicle, and storing the vehicle information of the vehicle in the vehicle parking violation database; ② if the vehicle is not parked in violation of the regulations, judging whether the vehicle is randomly placed according to the standard parking direction range, and if the vehicle is randomly placed, generating a random placement number of the vehicle, and storing the vehicle information of the vehicle in the vehicle random placement database; 7) completing a cycle after judging all vehicles at the current time, repeating steps 2) to 6), and cycling.

2. The method of claim 1, wherein, The specific way of judging whether each vehicle at the current time is parked in violation of the regulations is as follows: ①When the position coordinates of any vehicle are outside the preset parking area, record the current time as the initial suspected illegal parking time of the vehicle, and until the position coordinates of the vehicle are inside the parking area, record the current time as the final suspected illegal parking time of the vehicle; ②If the difference between the final suspected illegal parking time and the initial suspected illegal parking time of the vehicle is greater than the preset illegal parking time threshold, the vehicle is an illegal parking vehicle.

3. The method of claim 1, wherein, The feature data includes direction features, and if the vehicle does not illegally park, whether the vehicle is randomly placed is determined according to the standard parking direction range of the vehicle in the following manner: ①The parking direction of any vehicle is determined according to the direction features of the vehicle at the current time; ②If the parking direction of the vehicle is not within the standard parking direction range, the vehicle is a randomly placed vehicle; ③If the parking direction of the vehicle is within the standard parking direction range, the vehicle is not randomly placed.

4. The method of claim 1, wherein, Any element in the maximum matching set includes the number corresponding to the previous time and the number and position coordinates corresponding to the current time of the same vehicle.

5. The method of claim 1, wherein, The vehicle information similarity matrix is calculated in the following manner: ①The cosine similarity between the feature data of the two sets is calculated according to the following formula: where cos Θ is the cosine similarity, is the i-th feature data in one of the sets, is the j-th feature data in the other set. ②The cosine similarities obtained in step ① are used to form a feature similarity matrix that reflects the similarity of the feature data in the two sets; ③The rectangular frame of the corresponding vehicle is obtained according to the position coordinates of the vehicle, and the intersection-over-union ratio between the rectangular frames of the vehicles in the two sets is calculated according to the following formula: where S iou is the intersection over union of the rectangular boxes, S C is the intersection of the rectangular boxes of the two vehicles, S D is the union of the rectangular boxes of the two vehicles; ④The intersection-over-union ratios obtained in step ③ are used to form a position similarity matrix that reflects the similarity of the positions of the vehicles in the two sets; ⑤The product of the feature similarity matrix multiplied by the weight of the feature similarity is added to the product of the position similarity matrix multiplied by the weight of the position similarity, to obtain the vehicle information similarity matrix of the two sets.

6. The method of claim 5, wherein, The intersection of the rectangular frames of two vehicles is calculated according to the following formula: In the formula, S C The intersection of the rectangular bounding boxes of the two vehicles. and These are the coordinates of two non-adjacent vertices within the rectangle containing one of the vehicles. and Let each be a non-adjacent vertex within the rectangle containing the other car, and let the point be... point Points are vertices in the same direction as the rectangle. point These are the vertices of the rectangle in the same direction; The union of the rectangular frames of two vehicles is calculated according to the following formula: where S D is the union of the rectangular frames of the two vehicles, and are the coordinates of two non-adjacent vertices of the rectangular frame of one of the vehicles, respectively, and are the coordinates of two non-adjacent vertices of the rectangular frame of the other vehicle, respectively, and point point is a vertex of the same orientation of the rectangular frame, and point point is a vertex of the same orientation of the rectangular frame.

7. The method of claim 1, wherein, The vehicle information includes the feature data and position coordinates of the vehicle.

8. The method of claim 1, wherein, The confidence is obtained by a video acquisition unit using the YoloV5 algorithm, which obtains the position coordinates of the vehicle using the YoloV5 algorithm.

9. The method of claim 1, wherein, The Re-ID feature is obtained by the Re-ID feature recognition branch module, the color feature is obtained by the color feature recognition branch module, the direction feature is obtained by the direction feature recognition branch module, and the vehicle type feature is obtained by the vehicle type feature recognition branch module. The Re-ID feature recognition branch module, the color feature recognition branch module, the direction feature recognition branch module, and the vehicle type feature recognition branch module together form a neural network model for extracting feature data with the backbone network.

Images