A method and system for monitoring and managing classification of illegal parking behavior

Abstract

The application discloses a kind of illegal parking behavior classification monitoring management method and system, belong to wisdom traffic field, wherein method includes: using monitoring camera configuration monitoring illegal parking area, obtains monitoring video stream;Video stream is handled, and the global image of illegal parking area is obtained;According to illegal parking behavior classification standard, construct illegal parking behavior identification set;Traverse historical illegal parking image, using illegal parking behavior identification set is labeled to image, forms illegal parking sample image set;Based on illegal parking sample image set, construct target detection model;Illegal parking behavior detection and classification are carried out to real-time monitoring image using target detection model, realize automatic large-scale monitoring, provide real-time decision support for illegal parking management.The present application solves the technical problem that the prior art cannot accurately identify and classify illegal parking behavior, achieves the technical effect of realizing accurate, automatic, intelligent illegal parking management.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent transportation, in particular to a method and system for monitoring and managing illegal parking behavior classification. BACKGROUND

[0002] With the increase in the number of vehicles in cities, illegal parking problems are becoming increasingly serious, causing great distress to urban traffic management and public safety. The existing illegal parking management method relies on manual monitoring and identification, which is inefficient and cannot achieve large-scale monitoring. It is also difficult to accurately identify and classify different types of illegal parking behavior, resulting in unfair punishment. SUMMARY

[0003] The present application provides a method and system for monitoring and managing illegal parking behavior classification, aiming to solve the technical problem of being unable to accurately identify and classify illegal parking behavior in the prior art.

[0004] In view of the above problems, the present application provides a method and system for monitoring and managing illegal parking behavior classification.

[0005] The first aspect of the present application provides a method for monitoring and managing illegal parking behavior classification, which comprises: configuring a monitoring camera group, collecting images of an illegal parking area through the monitoring camera group, and obtaining a plurality of monitoring video streams; separating image frames of the plurality of monitoring video streams, and performing image registration based on the image frame separation result to obtain a global image of the illegal parking area; obtaining a set of illegal parking behavior identifiers based on the classification standard of illegal parking behavior; obtaining a set of historical illegal parking images based on the illegal parking area; traversing the set of historical illegal parking images to obtain a first illegal parking image, identifying the first illegal parking image based on the set of illegal parking behavior identifiers, and obtaining a set of illegal parking sample images; constructing a target detection model based on the set of illegal parking sample images; obtaining a set of real-time global images of the illegal parking area, inputting real-time monitoring images into the target detection model, and obtaining illegal parking behavior in the set of real-time global images; and managing illegal parking according to the illegal parking behavior.

[0006] In another aspect of the present application, a system for monitoring and managing illegal parking behavior classification is provided, which comprises: a monitoring image acquisition module configured to monitor a camera group, acquire multiple monitoring video streams by monitoring the camera group in an illegal parking area; a global image acquisition module configured to separate image frames from the multiple monitoring video streams, and perform image registration based on the image frame separation result to acquire a global image of the illegal parking area; a parking behavior identification module configured to acquire an illegal parking behavior identification set based on the classification standard of illegal parking behavior; a historical illegal parking data module configured to acquire a historical illegal parking image set based on the illegal parking area; an illegal parking sample module configured to traverse the historical illegal parking image set to acquire a first illegal parking image, identify the first illegal parking image based on the illegal parking behavior identification set, and acquire an illegal parking sample image set; a detection model construction module configured to construct a target detection model based on the illegal parking sample image set; an illegal parking behavior module configured to acquire a real-time global image set of the illegal parking area, input the real-time monitoring image into the target detection model, and acquire illegal parking behavior in the real-time global image set; and an illegal parking management module configured to manage illegal parking based on the illegal parking behavior.

[0007] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0008] The technical solution adopted in the present application first configures a monitoring camera to monitor an illegal parking area and acquire a monitoring video stream, then processes the video stream based on image processing technology to acquire a global image of the illegal parking area, and constructs an illegal parking behavior identification set according to the classification standard of illegal parking behavior, then traverses historical illegal parking images, labels the images using the illegal parking behavior identification set to form an illegal parking sample image set, then constructs a target detection model based on the illegal parking sample image set, and finally detects and classifies illegal parking behavior using the target detection model on real-time monitoring images to realize automatic large-scale monitoring and provide real-time decision support for illegal parking management, thereby solving the technical problem that the prior art cannot accurately identify and classify illegal parking behavior, and achieving the technical effect of realizing accurate, automatic and intelligent illegal parking management.

[0009] The above description is only a summary of the technical solutions of the present application, in order to more clearly understand the technical means of the present application, the present application can be implemented according to the content of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 A possible flowchart of an illegal parking behavior classification monitoring and management method is provided for the embodiments of the present application.

[0011] Figure 2 A possible flowchart of constructing a target detection model in the illegal parking behavior classification monitoring management method is provided for the embodiments of the present application.

[0012] Figure 3 A possible flowchart of performing second illegal identification in the illegal parking behavior classification monitoring management method is provided for the embodiments of the present application.

[0013] Figure 4 A possible structure diagram of the illegal parking behavior classification monitoring management system is provided for the embodiments of the present application.

[0014] Label explanation: monitoring image acquisition module 11, global image acquisition module 12, parking behavior identification module 13, historical illegal data module 14, illegal parking sample module 15, detection model construction module 16, illegal parking behavior module 17, illegal parking management module 18. DETAILED DESCRIPTION

[0015] The general idea of the technical solutions provided by the present application is as follows:

[0016] The embodiments of the present application provide an illegal parking behavior classification monitoring management method and system. Specifically, first, a monitoring camera is used to monitor an illegal parking area to obtain a monitoring video stream; then, image processing is performed on the video stream to obtain a global image of the illegal parking area; then, an illegal parking behavior identification set is constructed according to an illegal parking behavior classification standard; then, historical illegal parking images are traversed, and the images are labeled using the illegal parking behavior identification set to form an illegal parking sample image set; then, a target detection model is constructed based on the illegal parking sample image set; finally, real-time monitoring images are detected and classified for illegal parking behavior using the target detection model to achieve automatic large-scale monitoring.

[0017] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be specifically introduced in combination with the drawings of the specification. EMBODIMENT

[0018] As shown in the drawings, Figure 1 The embodiments of the present application provide an illegal parking behavior classification monitoring management method, which comprises:

[0019] Step S100: configure a monitoring camera group, and acquire a plurality of monitoring video streams by image acquisition of the illegal parking area through the monitoring camera group;

[0020] Specifically, a monitoring camera group is configured to continuously capture images of the illegally parked area, obtaining multiple monitoring video streams. The monitoring camera group refers to multiple camera devices deployed within the monitored illegally parked area to capture images of the scene within that area, with each camera device corresponding to one monitoring video stream. Each camera device in the monitoring camera group continuously captures images of the illegally parked area at a certain frame rate, forming a corresponding monitoring video stream to monitor the area. This results in multiple monitoring video streams, which together constitute the basic data for monitoring the illegally parked area.

[0021] By deploying surveillance cameras at multiple locations in the area to be monitored and continuously collecting multi-channel surveillance video streams, image information support is provided for the subsequent detection and identification of illegal parking.

[0022] Step S200: Perform image frame separation on the multiple monitoring video streams, and perform image registration based on the image frame separation results to obtain a global image of the illegal parking area;

[0023] Specifically, firstly, a time interval is set based on the frame rate of the surveillance video stream, for example, extracting one image frame every second. Within this time interval, one image frame is extracted from each surveillance video stream as the image frame separation result. For the separated image frames, a feature point detection algorithm is used to detect feature points. Then, based on the matching relationship between feature points in different image frames, the geometric transformation parameters of each image frame are calculated, such as translation, rotation, and scaling parameters. Finally, the image frames are projected according to the geometric transformation parameters, and the transformed image frames are stitched together to obtain a global image of the illegal parking area. This achieves the goal of extracting image frames from the surveillance video stream and obtaining a global view of the illegal parking area through image registration.

[0024] Step S300: Based on the classification criteria for illegal parking behavior, obtain the set of illegal parking behavior identifiers;

[0025] Specifically, the classification criteria for illegal parking refers to the rules or guidelines for classifying and defining different types of illegal parking. Based on these criteria, illegal parking can be divided into different categories, such as parking on the sidewalk, parking at intersections, and parking within marked lines. The illegal parking identifier set refers to the set of identifiers defined according to the classification criteria for illegal parking, used to identify and recognize different categories of illegal parking. Each identifier corresponds to a category of illegal parking.

[0026] Firstly, detailed illegal parking behavior classification standards are formulated, each classification corresponding to a type of illegal parking behavior; then, identifiers corresponding to each type of illegal parking behavior are defined, such as text labels of "occupying the road" and "parking at the intersection", or image icons representing different illegal parking manners, etc., and the set of these identifiers constitutes an illegal parking behavior identifier set.

[0027] By means of the classification standards of illegal parking behaviors, the illegal parking behavior identifier set is obtained, which provides a reference for identifying illegal parking behaviors in a monitoring video stream or a global image, and is conducive to improving the accuracy of subsequent illegal parking behavior detection and identification.

[0028] Step S400: based on the illegal parking area, a historical illegal parking image set is obtained;

[0029] Specifically, the illegal parking area refers to a region that may have illegal parking behaviors, which corresponds to the region monitored by the configured monitoring camera group. The historical illegal parking image set refers to a set of images containing illegal parking behaviors obtained from the monitoring camera group configured in the illegal parking area within a historical time period, which records the illegal parking behaviors occurring in the region within the historical time period, and provides sample data for constructing an illegal parking behavior detection model.

[0030] Firstly, image frames containing illegal parking behaviors are retrieved from the monitoring video of the illegal parking area within the historical time period; then, the retrieved image frames are filtered and integrated, irrelevant images of illegal parking behaviors are filtered out, and multiple images belonging to the same illegal parking behavior are integrated, to finally obtain the historical illegal parking image set.

[0031] By retrieving and integrating the historical illegal parking image set from the historical monitoring video, sample images required for constructing an illegal parking behavior detection model based on existing data are obtained, which provides sample data for training a target detection model.

[0032] Step S500: the historical illegal parking image set is traversed to obtain a first illegal parking image, and the first illegal parking image is identified based on the illegal parking behavior identifier set to obtain an illegal parking sample image set;

[0033] Specifically, each image in the historical illegal parking image set is accessed one by one, and each image is inspected to select one image as a first illegal parking image for processing; according to the illegal parking behavior in the first illegal parking image, an identifier corresponding thereto is found in the illegal parking behavior identifier set, and the first illegal parking image is labeled by using the identifier, and the labeled first illegal parking image contains corresponding illegal parking behavior category information.

[0034] First, a first illegal parking image is selected from a historical illegal parking image set, and then a corresponding identifier is found in an illegal parking behavior identifier set according to the illegal parking behavior in the image. The first illegal parking image is manually labeled or automatically labeled using the identifier to obtain a first illegal parking sample image. Then, more images are selected from the historical illegal parking image set, labeled according to the illegal parking behavior, and together with the first illegal parking sample image form an illegal parking sample image set.

[0035] By traversing the images in the historical illegal parking image set and selecting the correct class identifier according to the illegal parking behavior identifier set, it is ensured that each image in the illegal parking sample image set is labeled with an illegal parking type, providing sample data for training the target detection model.

[0036] Step S600: constructing a target detection model based on the illegal parking sample image set;

[0037] Specifically, a target detection algorithm in machine learning is used, and the illegal parking sample image set is used to train the target detection model. First, the sample images are divided into different categories according to the class labels in the sample image set, then the features of each sample image are extracted to represent the image content, and finally the class information is used to optimize the model parameters in the target detection algorithm, so that the model can correctly classify the images and detect the illegal parking behavior in the images based on the image features. The model parameters are continuously iterated and optimized, and when the model effect reaches the predetermined standard, the final target detection model is obtained.

[0038] By constructing a target detection model based on the illegal parking sample image set, the illegal parking behavior in the new input image can be automatically detected and the corresponding category information can be given, providing a basis for subsequent illegal parking behavior monitoring and identification.

[0039] Step S700: obtaining a real-time global image set of the illegal parking area, inputting the real-time monitoring image into the target detection model, and obtaining the illegal parking behavior in the real-time global image set;

[0040] Specifically, first, the configured monitoring camera group is used to monitor the illegal parking area in real time, the global image of the area is obtained through image processing technology, and the real-time global image set is formed. Then, each global image in the real-time global image set is input into the target detection model trained previously, and the model will detect and identify the illegal parking behavior in each input image, and give the results such as the category and corresponding position of the illegal parking behavior in the image. Finally, the real-time global image set corresponding to the illegal parking behavior and its specific position in the image and other information are obtained by integrating the detection and identification results of the model, and the illegal parking behavior in the real-time global image set is obtained.

[0041] By using the target detection model to detect and identify the actual monitoring data, the function of illegal parking behavior in real-time monitoring area is obtained, and intelligent identification and classification monitoring of illegal parking behavior are realized.

[0042] Step S800: illegal parking management is performed according to the illegal parking behavior.

[0043] Specifically, according to the specific content of the illegal parking behavior, such as category, location, time, etc., the detailed information of the illegal parking is recorded. Then, the illegal parking information is sent to the corresponding management institution through the set communication equipment, and the subsequent management work such as fine punishment is performed. At the same time, a display screen and other prompt devices are set in the illegal parking area, and the detected illegal parking information is used to prompt the driver or the owner to rectify.

[0044] By performing illegal parking management according to the illegal parking behavior, the function of taking management measures on the real-time illegal parking behavior obtained by detection and identification, such as recording information, notifying punishment, issuing removal prompt, etc., is realized, so as to achieve the purpose of standardizing social traffic order and realize accurate, automatic and intelligent illegal parking management.

[0045] Further, the embodiment of the application also includes:

[0046] Step S210: setting an extraction interval time, extracting key frames from each monitoring video stream based on the extraction interval time as an image frame separation result, wherein each image frame contains a corresponding acquisition time;

[0047] Step S220: feature point detection is performed on the extracted key frames to obtain image feature points;

[0048] Step S230: feature point matching is performed on the image frames under the same acquisition time based on the image feature points, and a feature point matching relationship is established;

[0049] Step S240: based on the feature point matching relationship, the geometric transformation parameters of each image frame are calculated;

[0050] Step S250: image frame registration is performed based on the geometric transformation parameters to obtain the global image.

[0051] Specifically, the extraction interval time refers to determining the time interval for extracting image frames from the monitoring video stream, and each interval corresponds to extracting an image frame, which can be set to several seconds to tens of seconds. Key frame images are selected from the video stream according to the set time interval. Each extracted key frame image records its corresponding acquisition time in the monitoring video stream, thereby forming an image frame separation result. Then, a feature point detection algorithm such as SIFT or SURF is used to automatically detect feature points in each key frame image. The feature points are points in the image that have obvious visual features, such as points with sharp changes in color tone and large gradient changes.

[0052] Next, images with similar acquisition times are selected as matching images, and feature points in the images are detected respectively; for each feature point in the image, a matching point is searched in the remaining frame images; whether the two feature points are matching points is determined according to a predetermined threshold value, such as whether the similarity between the descriptors is higher than the threshold value, and if so, it is determined that the feature points are matching, and the feature point matching relationship is obtained. Subsequently, according to the feature point matching relationship established between the key frame images, the transformation parameters required for the projection transformation from one image to another image are calculated, such as the rotation angle, the translation matrix, the size scaling ratio, etc., and the geometric transformation parameters of each image frame are obtained. Finally, according to the calculated geometric transformation parameters, coordinate transformation and projection transformation are performed on the key frame images, so that corresponding scene elements in different images are aligned, and the purpose of image registration is achieved.

[0053] By extracting key frame images from the monitoring video stream and obtaining the global image function through feature point detection, matching and geometric transformation, the information of multiple monitoring video streams is integrated to provide basic data for identifying illegal parking in the monitoring area.

[0054] Further, as shown in Figure 2 The embodiment of the present application further includes:

[0055] Step S610: dividing the illegal parking area to obtain a first area, a second area and a third area, wherein the first area is the illegal parking area that is prohibited to drive into at any time period, the second area is the illegal parking area that can be driven into, and the third area is the illegal parking area that can be parked for a predetermined time interval;

[0056] Step S620: constructing a first detection model, a second detection model and a third detection model according to the first area, the second area and the third area, respectively;

[0057] Step S630: fusing the first detection model, the second detection model and the third detection model into the target detection model.

[0058] Specifically, first, based on the attributes and functions of the illegal parking area, image segmentation technology is adopted to divide the area in combination with geographic information, different sub-areas are obtained, and three sub-areas are obtained, which are a first area, a second area and a third area. The first area is an illegal parking area that is prohibited to enter at any time, that is, an area where any vehicle is strictly prohibited to enter or stay; the second area is an illegal parking area that can be entered, that is, an area where vehicles are allowed to enter but are prohibited to stay; and the third area is an illegal parking area that can stay for a predetermined time interval, that is, an area where vehicles are allowed to enter and stay for a certain time.

[0059] Then, a first detection model is constructed for the first area to detect whether a vehicle enters or stays in the first area, a second detection model is constructed for the second area to detect whether a vehicle stays in the second area, and a third detection model is constructed for the third area to detect whether a vehicle stays in the third area for more than a predetermined time interval. The three detection models are individually set and trained according to the data characteristics of the respective areas, so that each detection model can accurately detect illegal behaviors in the corresponding area.

[0060] Then, the three single-area detection models are integrated to construct a unified illegal parking behavior detection model, so that the model can detect whether a vehicle enters or stays in the first area, and can also detect whether a vehicle in the second area and the third area stays for more than a predetermined time interval, so as to meet the detection requirements of illegal behaviors in the entire illegal parking area.

[0061] Further, the embodiments of the present application also include:

[0062] Step S510: performing regional division on the first illegal parking image according to the first area, the second area and the third area, and obtaining a regional division result;

[0063] The regional division result includes a first division area, a second division area and a third division area.

[0064] Step S520: constructing a vehicle feature set, the vehicle feature set containing a plurality of vehicle features;

[0065] Step S530: performing first illegal identification on the regional division result according to the vehicle feature set and the first division area;

[0066] Step S540: performing second illegal identification on the regional division result according to the vehicle feature set and the second division area;

[0067] Step S550: performing third illegal identification on the regional division result according to the vehicle feature set and the third division area;

[0068] Step S560: obtaining the illegal parking sample image according to the first illegal parking identification, the second illegal parking identification and the third illegal parking identification;

[0069] Step S570: adding the illegal parking sample image to the illegal parking sample image set.

[0070] Specifically, the first illegal parking image is segmented according to the three set regions to obtain corresponding three sub-regions, and the three sub-regions constitute the region division result. Then, various features of the vehicle are extracted to constitute a feature set, such as vehicle color, vehicle type, vehicle brand, license plate number, vehicle direction, time feature, etc.

[0071] Then, the vehicle features are matched in the first division region, and if a vehicle matching the vehicle features of the vehicle prohibited from entering the region is found, it is determined as the first illegal parking identification, i.e. the case of occupying the road. Similarly, the region division result is identified as the second illegal parking identification or the third illegal parking identification according to the vehicle feature set and the second division region or the third division region, for detecting the illegal parking case of staying too long. The detected vehicle features are matched with the vehicle feature set in the second division region, and if a vehicle feature matching the set longest passing time limit is found, it is determined as the second illegal parking identification, i.e. the case of staying too long in the region. The detected vehicle features are matched with the vehicle feature set in the third division region, and whether there is an illegal parking case of staying too long in the third division region is detected, and the third illegal parking identification is made in the third division region of the region division result accordingly. Finally, the image containing the illegal parking information obtained by region division and illegal parking identification is added to the illegal parking sample image set. The illegal parking sample image set is used to train the target detection model and contains image samples of various illegal parking cases.

[0072] By region division on the first illegal parking image and judging whether there is an illegal parking case according to different region types, the illegal parking identification is realized, and the final illegal parking sample image is obtained, providing sample training basic data for constructing an accurate target detection model.

[0073] Further, as shown in Figure 3 the embodiment of the present application further comprises:

[0074] Step S541: setting the minimum passing speed of the second region;

[0075] Step S542: judging whether the first illegal parking image has a second target vehicle;

[0076] Step S543: if the second region of the first illegal parking image has the second target vehicle, obtaining the second target vehicle features;

[0077] Step S544: obtaining an associated illegal parking image of the first illegal parking image based on the minimum passing speed;

[0078] Step S545: if the associated illegal parking image exists the second target vehicle feature, performing second illegal parking identification on the region division result.

[0079] Specifically, first, the minimum passing speed of the vehicle in the second region is determined according to the road type and traffic flow of the second region, for example, 30 kilometers per hour. Then, it is judged whether the vehicle is detected in the second division region of the first illegal parking image, and if so, it is taken as the second target vehicle for subsequent judgment. If it is judged that the second target vehicle exists in the second division region of the first illegal parking image, the features of the vehicle, such as the vehicle type, color, license plate number, etc., are further obtained for matching with other images to judge whether the second target vehicle is lower than the minimum passing speed. Subsequently, according to the minimum passing speed of the second region and the collection time of the first illegal parking image, the time period when the second target vehicle should have left the second region is calculated, and one or more images are obtained from the monitoring video in the time period as the associated illegal parking image. If the second target vehicle exists in the associated illegal parking image, it means that the second target vehicle is lower than the minimum passing speed and belongs to the illegal parking behavior. Finally, if the vehicle with the same features as the second target vehicle is detected in the associated image, it can be judged that the second target vehicle does not pass through the second region within the time period required by the minimum passing speed, and thus it is determined as illegal parking and the second illegal parking identification is performed on the second division region of the region division result.

[0080] By setting the minimum passing speed, obtaining the second target vehicle feature and the associated illegal parking image, etc., the function of judging whether the passing time of a specific vehicle meets the requirements according to the image and performing illegal parking identification accordingly is realized, and the illegal parking behavior of passing through the image beyond the time is effectively detected.

[0081] Further, the embodiments of the present application also include:

[0082] Step S551: setting the maximum parking time interval of the third region;

[0083] Step S552: judging whether the first illegal parking image exists the third target vehicle;

[0084] Step S553: if the third region of the first illegal parking image exists the third target vehicle, obtaining the third target vehicle feature;

[0085] Step S554: obtaining the collection time of the first illegal parking image;

[0086] Step S555: acquiring a first interval illegal parking image and a second interval illegal parking image according to the acquisition time and the maximum parking time interval;

[0087] Step S554: if the first interval illegal parking image or the second interval illegal parking image contains the third target vehicle feature, performing third illegal parking identification on the third partitioned region.

[0088] Specifically, first, the maximum allowed parking time of the vehicle in the third region is determined according to the parking space type and the traffic flow of the third region, for example, 30 minutes. Then, it is determined whether a vehicle is detected in the third partitioned region of the first illegal parking image, and if so, the vehicle is taken as the third target vehicle for subsequent judgment. If it is determined that the third partitioned region of the first illegal parking image contains the third target vehicle, the features of the vehicle, such as the vehicle model, color, license plate number, etc., are further acquired for matching with the interval illegal parking images acquired subsequently to determine whether the parking time of the third target vehicle exceeds the standard.

[0089] Then, the exact acquisition time of the first illegal parking image is recorded, and according to the acquisition time of the first illegal parking image, two time points are calculated with the maximum parking time interval as the step, and then two images are acquired from the corresponding time points as the first interval illegal parking image and the second interval illegal parking image. The acquisition of the two interval illegal parking images also requires that the third partitioned region is contained, and the acquisition time meets the requirement of determining whether the parking time of the third target vehicle exceeds the standard. If the same features of the third target vehicle are detected in the first interval illegal parking image or the second interval illegal parking image, it is determined that the parking time of the third target vehicle exceeds the requirement of the maximum parking time interval, and accordingly, third illegal parking identification is performed on the third partitioned region of the region partitioning result.

[0090] By setting the maximum parking time interval, acquiring the features of the third target vehicle and the two interval illegal parking images, etc., the function of determining whether the parking time of a specific vehicle meets the standard according to the images and performing illegal parking identification accordingly is realized, the over-time parking illegal behavior in the images is effectively detected, and thus high-accuracy illegal behavior detection is realized.

[0091] Further, the embodiments of the present application also include:

[0092] Step S621: acquiring a first illegal parking sample image set according to the first region and the first illegal parking identification;

[0093] Step S622: acquiring a second illegal parking sample image set according to the second region and the second illegal parking identification;

[0094] Step S623: obtaining a third illegal parking sample image set according to the third region and the third illegal parking identifier;

[0095] Step S624: constructing the first detection model, the second detection model and the third detection model according to the first illegal parking sample image set, the second illegal parking sample image set and the third illegal parking sample image set respectively.

[0096] Specifically, images identified by the first illegal identifier are searched in all monitoring images, and the found images are collected to form a first illegal parking sample image set. The first illegal parking sample image set is used to train the first detection model to detect illegal parking behavior in the first region. Images identified by the second illegal identifier are searched in all monitoring images, and the found images are collected to form a second illegal parking sample image set. The second illegal parking sample image set is used to train the second detection model to detect illegal parking behavior in the second region. Images identified by the third illegal identifier are searched in all monitoring images, and the found images are collected to form a third illegal parking sample image set. The third illegal parking sample image set is used to train the third detection model to detect illegal parking behavior in the third region.

[0097] The construction of the first detection model, the second detection model and the third detection model according to the first illegal parking sample image set, the second illegal parking sample image set and the third illegal parking sample image set respectively means that the three detection models are trained by using the corresponding sample image sets, so that the first detection model is suitable for detecting illegal parking in the first region, the second detection model is suitable for the second region, and the third detection model is suitable for the third region. Among them, the three detection models can adopt the same or different machine learning algorithms, and are optimized according to the data characteristics of each region to ensure that each detection model meets the predetermined accuracy requirement.

[0098] By constructing the illegal parking sample image set corresponding to each region and training the three region detection models according to the sample set, the individual optimization of model parameters and the pairing of region detection are realized, which lays a foundation for constructing the target detection model.

[0099] In summary, the illegal parking behavior classification monitoring and management method provided by the embodiments of the present application has the following technical effects:

[0100] The monitoring camera group is configured to perform image acquisition on the illegal parking area through the monitoring camera group, to obtain a plurality of monitoring video streams, to provide original data support for image processing and target detection, to perform image frame separation on the plurality of monitoring video streams, and to perform image registration based on the image frame separation result to obtain a global image of the illegal parking area, to provide input data for the target detection model, to obtain a set of illegal parking behavior identifiers based on the classification standard of illegal parking behavior, to provide labeling information for the construction of a set of sample images and the training of the target detection model, to obtain a set of historical illegal parking images based on the illegal parking area, to provide original data for the construction of a set of sample images of illegal parking, to traverse the set of historical illegal parking images to obtain a first illegal parking image, to identify the first illegal parking image based on the set of illegal parking behavior identifiers, to obtain a set of sample images of illegal parking, and to provide sample data for the training of the target detection model, and to construct the target detection model based on the set of sample images of illegal parking, to detect illegal parking behavior in real-time monitoring images, to obtain a set of real-time global images of the illegal parking area, to input the real-time monitoring images into the target detection model, to obtain illegal parking behavior in the set of real-time global images, and to manage illegal parking according to the illegal parking behavior, to detect illegal parking behavior in real-time monitoring images, and to perform corresponding management, thereby achieving the technical effect of accurate, automatic and intelligent illegal parking management. Embodiments

[0101] Based on the same inventive concept as the illegal parking behavior classification monitoring and management method in the foregoing embodiments, as shown in Figure 4 The embodiments of the present application provide an illegal parking behavior classification monitoring and management system, which comprises:

[0102] A monitoring image acquisition module 11 is configured to configure a monitoring camera group, perform image acquisition on an illegal parking area through the monitoring camera group, and obtain a plurality of monitoring video streams.

[0103] A global image acquisition module 12 is configured to perform image frame separation on the plurality of monitoring video streams, perform image registration based on the image frame separation result, and obtain a global image of the illegal parking area.

[0104] An illegal parking behavior identification module 13 is configured to obtain a set of illegal parking behavior identifiers based on the classification standard of illegal parking behavior.

[0105] A historical illegal parking data module 14 is configured to obtain a set of historical illegal parking images based on the illegal parking area.

[0106] An illegal parking sample module 15 is configured to traverse the set of historical illegal parking images to obtain a first illegal parking image, identify the first illegal parking image based on the set of illegal parking behavior identifiers, and obtain a set of sample images of illegal parking.

[0107] The detection model construction module 16 constructs a target detection model based on the set of sample images of illegal parking behaviors;

[0108] The illegal parking behavior module 17 is configured to acquire a set of real-time global images of the illegal parking area, input the real-time monitoring images into the target detection model, and acquire illegal parking behaviors in the set of real-time global images.

[0109] The illegal parking management module 18 is configured to perform illegal parking management according to the illegal parking behaviors.

[0110] Further, the global image acquisition module 12 includes the following execution steps:

[0111] An extraction interval time is set, and key frames are extracted from each monitoring video stream based on the extraction interval time as image frame separation results, wherein each image frame contains a corresponding acquisition time;

[0112] Feature point detection is performed on the extracted key frames to obtain image feature points;

[0113] Feature point matching is performed on image frames at the same acquisition time based on the image feature points to establish a feature point matching relationship;

[0114] Based on the feature point matching relationship, geometric transformation parameters of each image frame are calculated;

[0115] Image frame registration is performed based on the geometric transformation parameters to obtain the global image.

[0116] Further, the detection model construction module 16 includes the following execution steps:

[0117] The illegal parking area is divided to obtain a first area, a second area, and a third area, wherein the first area is the illegal parking area that is prohibited to enter at any time period, the second area is the illegal parking area that is allowed to enter, and the third area is the illegal parking area that is allowed to stay for a predetermined time interval;

[0118] The first detection model, the second detection model, and the third detection model are constructed based on the first area, the second area, and the third area, respectively;

[0119] The first detection model, the second detection model, and the third detection model are fused into the target detection model.

[0120] Further, the illegal parking sample module 15 includes the following execution steps:

[0121] The first illegal parking image is regionally divided according to the first area, the second area, and the third area to obtain a regionally divided result;

[0122] The region division result includes a first division region, a second division region, and a third division region.

[0123] A vehicle feature set is constructed, and the vehicle feature set contains a plurality of vehicle features.

[0124] The region division result is first identified according to the vehicle feature set and the first division region.

[0125] The region division result is second identified according to the vehicle feature set and the second division region.

[0126] The region division result is third identified according to the vehicle feature set and the third division region.

[0127] The first, second, and third violations are identified according to the first, second, and third violations.

[0128] The violation sample image is added to the violation sample image set.

[0129] Further, the violation sample module 15 further includes the following execution steps:

[0130] The minimum passing speed of the second region is set.

[0131] It is determined whether the first violation parking image has a second target vehicle.

[0132] If the second region of the first violation parking image has the second target vehicle, a second target vehicle feature is obtained.

[0133] Based on the minimum passing speed, an associated violation parking image of the first violation parking image is obtained.

[0134] If the associated violation parking image has the second target vehicle feature, the region division result is second identified.

[0135] Further, the violation sample module 15 further includes the following execution steps:

[0136] The maximum parking time interval of the third region is set.

[0137] It is determined whether the first violation parking image has a third target vehicle.

[0138] If the third region of the first violation parking image has the third target vehicle, a third target vehicle feature is obtained.

[0139] acquiring the acquisition time of the first illegal parking image;

[0140] acquiring a first interval illegal parking image and a second interval illegal parking image according to the acquisition time and the longest parking time interval;

[0141] if the first interval illegal parking image or the second interval illegal parking image contains the third target vehicle feature, performing third illegal parking identification on the area division result.

[0142] Further, the detection model construction module 16 further includes the following execution steps:

[0143] acquiring a first illegal parking sample image set according to the first area and the first illegal parking identification;

[0144] acquiring a second illegal parking sample image set according to the second area and the second illegal parking identification;

[0145] acquiring a third illegal parking sample image set according to the third area and the third illegal parking identification;

[0146] constructing the first detection model, the second detection model and the third detection model according to the first illegal parking sample image set, the second illegal parking sample image set and the third illegal parking sample image set respectively.

[0147] Any step of the above method can be stored as computer instructions or programs in an unlimited computer memory and can be called and recognized by an unlimited computer processor to realize any method in the embodiments of the present application, and no more limitation is made herein.

[0148] Further, the above first or second may not only represent the order relationship, but also may represent a specific concept, and / or refer to the selection of multiple elements individually or collectively. Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the present application and its equivalent technologies, the present application intends to include these modifications and variations.

Claims

1. A method of monitoring and managing the classification of illegal parking behavior, characterized by, The method is applied to a violation parking behavior classification monitoring management system, and the method comprises the following steps: A monitoring camera group is configured, image acquisition of a violation parking area is performed by the monitoring camera group, and a plurality of monitoring video streams are acquired; Image frame separation is performed on the plurality of monitoring video streams, and image registration is performed based on the image frame separation result to acquire a global image of the violation parking area; Based on a classification standard of violation parking behavior, a violation parking behavior identifier set is acquired; Based on the violation parking area, a historical violation parking image set is acquired; The historical violation parking image set is traversed to acquire a first violation parking image, the first violation parking image is identified based on the violation parking behavior identifier set, and a violation parking sample image set is acquired; A target detection model is constructed based on the violation parking sample image set; Real-time global image sets of the violation parking area are acquired, real-time monitoring images are input into the target detection model, and violation parking behaviors in the real-time global image sets are acquired; Violation parking management is performed according to the violation parking behaviors; The target detection model is constructed based on the violation parking sample image set, which comprises the following steps: The violation parking area is divided to acquire a first area, a second area and a third area, wherein the first area is the violation parking area that is prohibited to drive into at any time period, the second area is the violation parking area that can be driven into, and the third area is the violation parking area that can be parked for a predetermined time interval; First, second and third detection models are respectively constructed according to the first area, the second area and the third area; The first, second and third detection models are fused into the target detection model.

2. The method of claim 1, wherein, The image frame separation is performed on the plurality of monitoring video streams, and the global image of the violation parking area is acquired based on the image frame separation result, which comprises the following steps: An extraction interval time is set, key frames are extracted from each monitoring video stream based on the extraction interval time as the image frame separation result, wherein each image frame contains a corresponding acquisition time; Feature point detection is performed on the extracted key frames to acquire image feature points; Feature point matching is performed on the image frames at the same acquisition time based on the image feature points to establish a feature point matching relationship; Geometric transformation parameters of each image frame are calculated based on the feature point matching relationship; Image frame registration is performed based on the geometric transformation parameters to acquire the global image.

3. The method of claim 1, wherein, The first violation parking image is identified based on the violation parking behavior identifier set to acquire the violation parking sample image set, which comprises the following steps: The first violation parking image is regionally divided according to the first area, the second area and the third area to acquire a regional division result; The regional division result comprises a first division area, a second division area and a third division area; A vehicle feature set is constructed, and the vehicle feature set contains a plurality of vehicle features; The first violation identification is performed on the regional division result based on the vehicle feature set and the first division area. According to the vehicle feature set and the second division area, a second illegal parking identification is performed on the area division result; According to the vehicle feature set and the third division area, a third illegal parking identification is performed on the area division result; According to the first illegal parking identification, the second illegal parking identification, and the third illegal parking identification, the illegal parking sample image is obtained; The illegal parking sample image is added to the illegal parking sample image set.

4. The method of claim 3, wherein, The second illegal parking identification performed on the area division result according to the vehicle feature set and the second division area comprises: A minimum passing speed of the second area is set; It is judged whether the first illegal parking image has a second target vehicle; If the second area of the first illegal parking image has the second target vehicle, a second target vehicle feature is obtained; Based on the minimum passing speed, an associated illegal parking image of the first illegal parking image is obtained; If the associated illegal parking image has the second target vehicle feature, a second illegal parking identification is performed on the area division result.

5. The method of claim 3, wherein, The third illegal parking identification performed on the area division result according to the vehicle feature set and the third division area comprises: A maximum parking time interval of the third area is set; It is judged whether the first illegal parking image has a third target vehicle; If the third area of the first illegal parking image has the third target vehicle, a third target vehicle feature is obtained; A collection time of the first illegal parking image is obtained; According to the collection time and the maximum parking time interval, a first interval illegal parking image and a second interval illegal parking image are obtained; If the first interval illegal parking image or the second interval illegal parking image has the third target vehicle feature, a third illegal parking identification is performed on the area division result.

6. The method of claim 3, wherein, The first detection model, the second detection model, and the third detection model are respectively constructed according to the first area, the second area, and the third area, which comprises: According to the first area and the first illegal parking identification, a first illegal parking sample image set is obtained; According to the second area and the second illegal parking identification, a second illegal parking sample image set is obtained; According to the third area and the third illegal parking identification, a third illegal parking sample image set is obtained; The first detection model, the second detection model, and the third detection model are respectively constructed according to the first illegal parking sample image set, the second illegal parking sample image set, and the third illegal parking sample image set.

7. A classification and monitoring management system for illegal parking behavior, characterized in that, A system for implementing the method for classifying and monitoring illegal parking behavior according to any one of claims 1-6, the system comprising: A monitoring image acquisition module, the monitoring image acquisition module is configured to configure a monitoring camera group, and image acquisition is performed on an illegal parking area through the monitoring camera group to obtain a plurality of monitoring video streams; A global image acquisition module, the global image acquisition module is configured to separate image frames from the plurality of monitoring video streams, and perform image registration based on the image frame separation result to obtain a global image of the illegal parking area; A parking behavior identification module is configured to obtain a parking violation behavior identification set based on classification criteria of parking violation behaviors; A historical parking violation data module is configured to obtain a historical parking violation image set based on the parking violation area; A parking violation sample module is configured to traverse the historical parking violation image set to obtain a first parking violation image, identify the first parking violation image based on the parking violation behavior identification set, and obtain a parking violation sample image set; A detection model construction module is configured to construct a target detection model based on the parking violation sample image set; A parking violation behavior module is configured to obtain a real-time global image set of the parking violation area, input a real-time monitoring image into the target detection model, and obtain a parking violation behavior in the real-time global image set; A parking violation management module is configured to manage parking violation based on the parking violation behavior.

Images