Illegal parking detection method, device and equipment and storage medium

Abstract

The embodiment of the invention provides an illegal parking detection method and device, equipment and a storage medium. The method is applied to the technical field of vehicle traffic, and comprises the following steps: acquiring a real-time video stream in traffic gate monitoring, and extracting image frames in the real-time video stream according to a preset time interval; parking space identification is carried out based on the image frame, a parking space area detection result is generated, and the range of a no-parking area is extracted reversely; before, after, or simultaneously; vehicle and license plate detection is carried out based on the image frame to obtain current position coordinates and license plate information of the vehicle; according to whether the position coordinates are in the range of the parking forbidding area, whether illegal parking occurs is judged; if yes, a report is automatically generated, and system prompt information with illegal parking signs is generated. In this way, the problems that in traditional illegal parking inspection, the labor cost is too high, and the working efficiency and precision are low can be solved, the labor and material resource cost is reduced, and the working efficiency, management stability and precision are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer communication technology, and further to the field of vehicle traffic technology, and in particular to a method, device, equipment and storage medium for detecting illegal parking. Background Art

[0002] In the prior art, there are many emergency solutions to solve the problem of illegal parking, but they all have corresponding defects. For example: First, the labor cost is high. Traditional illegal parking inspections mainly rely on patrol personnel or set up fixed surveillance cameras, which requires a lot of manpower and material costs, especially for densely populated urban areas. Due to the large number of vehicles and complex driving routes, the workload and difficulty of patrol personnel are increased; second, the work efficiency is low. Traditional illegal parking inspections are mainly patrolled by patrol personnel, which is inefficient and cannot meet the basic needs of modern society for efficiency, speed and accuracy; third, the management is difficult. Illegal parking phenomena are sudden and difficult to predict in advance. Therefore, in terms of management, sudden illegal parking situations must be dealt with quickly, which also increases the difficulty at the management level; fourth, the instability is high. Traditional illegal parking inspections are affected by natural conditions such as weather and light, and are also affected by human factors such as fatigue, and there are many abnormal detection results; fifth, the accuracy is not high. Traditional illegal parking inspections mainly rely on manual patrols and visual judgments, and there are errors such as subjective cognition of personnel, which leads to limited accuracy. Therefore, a more intelligent and automated technology is urgently needed to improve the efficiency of traffic monitoring. Summary of the invention

[0003] The present disclosure provides a method, device, equipment and storage medium for detecting illegal parking, which solve the technical problem of low efficiency in detecting and processing illegal parking behaviors.

[0004] According to a first aspect of the present disclosure, a method for detecting illegal parking is provided. The method comprises: Acquire a real-time video stream from traffic checkpoint monitoring, and extract image frames from the real-time video stream according to a preset time interval; Perform parking space recognition based on the image frame, generate parking space area detection results, and reversely extract the range of the prohibited parking area; before, after, or at the same time; Perform vehicle and license plate detection based on the image frame to obtain the current position coordinates and license plate information of the vehicle; Determining whether the vehicle is illegally parked according to whether the current position coordinates of the vehicle are within the range of the prohibited parking area; If the vehicle is parked illegally, a report is automatically generated, and a system prompt message with an illegal parking sign is generated.

[0005] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the performing parking space recognition based on the image frame, generating a parking space area detection result, and reversely extracting the range of the prohibited parking area comprises: Input the image frame into the trained DeepLab V3+ semantic segmentation algorithm model to obtain a parking area detection result; Based on the parking area detection result, the coordinate range of the no-parking area is extracted in reverse order, the coordinate data of all key points in the no-parking area are sorted, and then the points constituting the boundary of the no-parking area are identified and sorted, and the boundary information of the no-parking area is constructed using the sorted boundary points, wherein: The no-parking area data includes: coordinate data of all key points in the no-parking area and boundary information of the no-parking area.

[0006] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the vehicle and license plate detection based on the image frame and obtaining the current position coordinates and license plate information of the vehicle include: Input the image frame into a trained YOLOv8 target detection model, locate the vehicle and license plate area in the image, and obtain the current position coordinates of the vehicle; The license plate area is cropped to generate a license plate candidate image, and the license plate candidate image is segmented into characters, the final results are merged, and the license plate number is output.

[0007] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the character segmentation of the license plate candidate image comprises: The EASTNet model is combined with the OCR text detection model to perform joint training on shared features to obtain the EASTNet-OCR model, where: The training method includes: obtaining a data set including license plate images and corresponding text labels, wherein the data set includes a variety of license plate types, character arrangement methods, and interference factors; The combined model is trained based on the data set, part of the network structure is shared during the joint training process, and the parameters of the model are adjusted to minimize the joint loss function, wherein the formula of the joint loss function is as follows: , in is the Iou loss function, A represents the prediction box, and B represents the range of the annotation box; is the cross entropy loss function, N is the total number of character samples, C is the total number of character categories, It isi The sample in c The true labels of the categories, It is i The sample in c The predicted probability of each category, is the overall measure of the difference between the model prediction and the true label, and λ is the weight; Preprocessing the license plate candidate image to obtain a preprocessed license plate candidate image; The preprocessed license plate candidate image is input into the EASTNet-OCR model, so that the output of the EASTNet model can be directly used as the input feature of the OCR text detection model, thereby obtaining a list containing character or text line prediction results.

[0008] The preprocessed license plate candidate image is input into the EASTNet-OCR model, so that the output of the EASTNet model can be directly used as the input feature of the OCR text detection model, thereby obtaining a list containing character or text line prediction results.

[0009] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein judging whether the vehicle is illegally parked according to whether the current position coordinates of the vehicle are within the range of the prohibited parking area comprises: If the current position coordinates of the vehicle are within the range of the prohibited parking area, it is further determined whether the position of the vehicle changes within a preset time threshold, wherein: The judgment method includes: determining the vehicle information of the vehicle, the specific time of arrival at the illegal parking area and the specific location of the vehicle based on the current position coordinates, license plate number, license plate position and detection time obtained after the vehicle is first identified by the image acquisition device, and then continuously tracking and comparing the position information of the vehicle within a preset time threshold. If the position information of the vehicle does not change within the preset time threshold, it is determined that the vehicle is illegally parked.

[0010] According to the above aspects and any possible implementation, an implementation is further provided, wherein if the vehicle is illegally parked, a report is automatically generated, and a system prompt message with an illegal parking sign is generated, including: If the vehicle is parked illegally, the vehicle's license plate number, current location coordinates, illegal parking area information, detection time and illegal parking snapshot are automatically obtained, and the obtained information is integrated into a formatted report. The report is stored in the illegal parking database of the traffic management department, and a system prompt message with an illegal parking sign is generated.

[0011] According to the above aspects and any possible implementation, an implementation is further provided, wherein if the vehicle is illegally parked, a report is automatically generated, and a system prompt message with an illegal parking sign is generated, including: If the vehicle is parked illegally, the vehicle's license plate number, current location coordinates, illegal parking area information, detection time and illegal parking snapshot are automatically obtained, and the above information is integrated into a formatted report. The report is stored in the illegal parking database of the traffic management department and a system prompt message with an illegal parking sign is generated.

[0012] According to the above aspects and any possible implementation manner, there is further provided an implementation manner, after determining that the vehicle is parked illegally, the method further includes: Send out an alarm and generate an alarm log, wherein the alarm log includes at least: the vehicle license plate number, current location coordinates, illegal parking area information, illegal parking time, illegal parking snapshot and penalty information; and / or, Record the sending status of the alarm log and / or the feedback information of the vehicle owner.

[0013] According to a second aspect of the present disclosure, a device for detecting illegal parking is provided. The device comprises: The image acquisition module 210 is used to acquire the real-time video stream in the traffic checkpoint monitoring and extract the image frames in the real-time video stream according to the preset time interval; The area extraction module 220 is used to perform parking space recognition based on the image frame, generate a parking space area detection result, and reversely extract the range of the prohibited parking area; before, after, or at the same time; The information acquisition module 230 is used to detect the vehicle and the license plate based on the image frame, and obtain the current position coordinates and license plate information of the vehicle; A violation judgment module 240, configured to judge whether the vehicle is illegally parked according to whether the current position coordinates of the vehicle are within the range of the prohibited parking area; The report generation module 250 is used to automatically generate a report if the vehicle is illegally parked, and generate a system prompt message with an illegal parking sign.

[0014] According to a third aspect of the present disclosure, an electronic device is provided, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, the method described above is implemented.

[0015] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method according to the first aspect and / or the second aspect of the present disclosure is implemented.

[0016] In the embodiments of the present disclosure, firstly, the image frames of the preset time interval can be extracted by acquiring the real-time video stream in the traffic checkpoint monitoring; and the parking space is identified based on the image frame to generate the parking space area detection result, and the range of the prohibited parking area is extracted in reverse; before, after, or at the same time; the vehicle and license plate detection is performed based on the image frame to obtain the current position coordinates and license plate information of the vehicle; secondly, it can be judged whether the vehicle is illegally parked according to whether the current position coordinates of the vehicle are within the range of the prohibited parking area; finally, if the vehicle is illegally parked, a report is automatically generated, and a system prompt message with an illegal parking sign is generated. In this way, the problems of high labor costs, low work efficiency and accuracy in traditional illegal parking inspections can be solved, which not only reduces the manpower and material costs and management difficulty in the inspection process, but also improves work efficiency, management stability and supervision accuracy.

[0017] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein: Figure 1 A flow chart of a method for detecting illegal parking provided by an embodiment of the present disclosure is shown; Figure 2 A structural diagram of a parking violation detection device provided by an embodiment of the present disclosure is shown; Figure 3 A structural diagram of an exemplary electronic device capable of implementing an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0020] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0021] In the embodiments of the present disclosure, firstly, the image frames of the preset time interval can be extracted by acquiring the real-time video stream in the traffic checkpoint monitoring; and the parking space is identified based on the image frame to generate the parking space area detection result, and the range of the prohibited parking area is extracted in reverse; before, after, or at the same time; the vehicle and license plate detection is performed based on the image frame to obtain the current position coordinates and license plate information of the vehicle; secondly, it can be judged whether the vehicle is illegally parked according to whether the current position coordinates of the vehicle are within the range of the prohibited parking area; finally, if the vehicle is illegally parked, a report is automatically generated, and a system prompt message with an illegal parking sign is generated. In this way, the problems of high labor costs, low work efficiency and accuracy in traditional illegal parking inspections can be solved, which not only reduces the manpower and material costs and management difficulty in the inspection process, but also improves work efficiency, management stability and supervision accuracy.

[0022] In conjunction with the accompanying drawings, a method, device, equipment and storage medium for detecting illegal parking provided by the embodiments of the present disclosure are described in detail below through specific embodiments.

[0023] Figure 1 A flowchart of a method for detecting illegal parking provided by an embodiment of the present disclosure is shown. Figure 1 As shown, a parking violation detection method 100 may include the following steps: S110, obtaining a real-time video stream in traffic checkpoint monitoring, and extracting image frames in the real-time video stream according to a preset time interval.

[0024] Exemplarily, the video stream data in the monitoring scene of the traffic checkpoint is captured and obtained in real time by means of monitoring equipment installed at the traffic checkpoint, wherein the monitoring equipment includes but is not limited to a high-definition camera, an infrared camera or an intelligent monitoring camera, which can capture the video information of the traffic checkpoint around the clock and at all times; Preset a time interval, which is used to determine the frequency of extracting image frames from the real-time video stream, wherein the preset time interval can be adjusted according to actual needs to adapt to scenarios with different traffic flows, monitoring requirements or image processing capabilities; According to the preset time interval, the video stream data acquired in real time is processed, and the corresponding image frames are accurately extracted from the video stream in units of time intervals, wherein before extracting the image frames, the real-time video stream may be subjected to preprocessing operations such as denoising, enhancement or format conversion to improve the efficiency and accuracy of subsequent image processing; The extracted image frames are stored or transmitted in real time through a data transmission network to a designated processing terminal or storage device for further analysis, processing or archiving, wherein the extracted image frames can be classified and stored or transmitted according to classification criteria such as time sequence, vehicle type, traffic violation, etc., to facilitate subsequent retrieval and analysis.

[0025] S120, performing parking space recognition based on the image frame, generating a parking space area detection result, and reversely extracting the range of the prohibited parking area.

[0026] In some embodiments, the image frame is input into a trained DeepLab V3+ semantic segmentation algorithm model to obtain a parking area detection result; Based on the parking area detection result, the coordinate range of the no-parking area is extracted in reverse order, the coordinate data of all key points in the no-parking area are sorted, and then the points constituting the boundary of the no-parking area are identified and sorted, and the boundary information of the no-parking area is constructed using the sorted boundary points, wherein: The no-parking area data includes: coordinate data of all key points in the no-parking area and boundary information of the no-parking area.

[0027] For example, the DeepLab V3+ algorithm is based on the encoder-decoder architecture, which uses atrous convolution and ASPP (Atrous Spatial Pyramid Pooling) modules to enhance feature extraction capabilities and achieve high-precision semantic segmentation.

[0028] Exemplarily, the DeepLab V3+ semantic segmentation algorithm model is specifically trained for traffic scenes and parking space recognition to improve the accuracy and efficiency of parking area detection.

[0029] Exemplarily, based on the parking area detection result, the coordinate range of the no-parking area is reversely determined and extracted, and the no-parking area is an area outside the parking area or where parking is not allowed under specific regulations, wherein the specific location and range of the no-parking area are determined by comparing the parking area detection result with preset no-parking rules or map information; the present invention automatically identifies parking spaces through a semantic segmentation algorithm, thereby reducing reliance on manually pre-marking no-parking areas.

[0030] Organizing the coordinate data of all key points in the prohibited parking area, using image processing algorithms or machine learning techniques to automatically identify and extract key points constituting the boundary of the prohibited parking area, ensuring the accuracy and robustness of boundary identification, and sorting these boundary points to ensure the continuity of the boundary; The sorted boundary points are used to construct boundary information of the no-parking area, which can clearly describe the shape, size and location of the no-parking area; the generated no-parking area data is verified and optimized to improve the accuracy and reliability of no-parking area identification, including but not limited to removing noise points, correcting boundary errors or merging adjacent no-parking areas.

[0031] S130, performing vehicle and license plate detection based on the image frame to obtain the current position coordinates and license plate information of the vehicle.

[0032] In some embodiments, the image frame is input into a trained YOLOv8 target detection model to locate the vehicle and the license plate area in the image to obtain the current position coordinates of the vehicle; The license plate area is cropped to generate a license plate candidate image, and the license plate candidate image is segmented into characters, the final results are merged, and the license plate number is output.

[0033] In some embodiments, performing character segmentation on the license plate candidate image includes: The EASTNet model is combined with the OCR text detection model to perform joint training on shared features to obtain the EASTNet-OCR model, where: The training method includes: obtaining a data set including license plate images and corresponding text labels, wherein the data set includes a variety of license plate types, character arrangement methods, and interference factors; The combined model is trained based on the data set, part of the network structure is shared during the joint training process, and the parameters of the model are adjusted to minimize the joint loss function, wherein the formula of the joint loss function is as follows: , in is the Iou loss function, A represents the prediction box, and B represents the range of the annotation box; is the cross entropy loss function, N is the total number of character samples, C is the total number of character categories, It is i The sample in c The true labels of the categories, It is i The sample in c The predicted probability of each category, is the overall measure of the difference between the model prediction and the true label, and λ is the weight; Preprocessing the license plate candidate image to obtain a preprocessed license plate candidate image; The preprocessed license plate candidate image is input into the EASTNet-OCR model, so that the output of the EASTNet model can be directly used as the input feature of the OCR text detection model, thereby obtaining a list containing character or text line prediction results.

[0034] The preprocessed license plate candidate image is input into the EASTNet-OCR model, so that the output of the EASTNet model can be directly used as the input feature of the OCR text detection model, thereby obtaining a list containing character or text line prediction results.

[0035] For example, EASTNet is a bus-type local network that uses the Acknowledging Ethernet principle to provide users with service primitives for low-level inter-process communication mechanisms. Users can use these service primitives to compile higher-level communication protocols and high-level application software.

[0036] The EAST (Efficient and Accurate Scene Text Detector) model abandons the traditional complex intermediate steps, such as candidate box extraction and text formatting, and directly predicts the text area in the image through a fully convolutional network (FCN). It adopts a multi-scale feature fusion structure to cope with text lines of different scales, thereby improving the detection capability of the model.

[0037] The OCR (Optical Character Recognition) text detection model is a model based on artificial intelligence technology. Its core function is to automatically identify and extract text content from images or scans and convert them into a computer-editable text format. Its basic principle is to locate, segment and recognize text in images through image processing and pattern recognition algorithms. This process usually includes preprocessing, text detection, text recognition and post-processing steps.

[0038] The purpose of the present invention is to integrate vehicle identification and license plate recognition into a unified step, which can, on the one hand, reduce errors and omissions in the identification process and improve the ability to accurately detect illegally parked vehicles; on the other hand, it can reduce data transmission and processing time, realize instant identification and processing of vehicles and license plates, and improve the system's real-time response capability to illegal parking behaviors.

[0039] Exemplarily, this step may be performed before, after, or simultaneously with step S120; The YOLOv8 object detection model is specifically trained for traffic scenes and license plate recognition to improve the accuracy and efficiency of vehicle and license plate area detection.

[0040] The EASTNet-OCR model achieves an organic combination of text detection and character recognition through joint training, thereby improving the overall performance of license plate character segmentation and recognition; wherein the preprocessing of the license plate candidate image includes: verifying and correcting the recognized license plate number to improve the accuracy and reliability of license plate recognition.

[0041] S140: Determine whether the vehicle is illegally parked based on whether the current position coordinates of the vehicle are within the range of the prohibited parking area.

[0042] In some embodiments, if the current position coordinates of the vehicle are within the range of the no-parking area, it is further determined whether the position of the vehicle changes within a preset time threshold, wherein: The judgment method includes: determining the vehicle information of the vehicle, the specific time of arrival at the illegal parking area and the specific location of the vehicle based on the current position coordinates, license plate number, license plate position and detection time obtained after the vehicle is first identified by the image acquisition device, and then continuously tracking and comparing the position information of the vehicle within a preset time threshold. If the position information of the vehicle does not change within the preset time threshold, it is determined that the vehicle is illegally parked.

[0043] Exemplarily, the current position coordinates of the vehicle are compared with the range of a preset no-parking area to determine whether the vehicle is located in the no-parking area. If the vehicle is located in the no-parking area, continuous tracking of the vehicle is initiated, the time when the vehicle is first identified is recorded as the start time, and a preset time threshold is set; within the preset time threshold, the position information of the vehicle is continuously acquired and updated, and the position information of the vehicle within the preset time threshold is compared to determine whether the position of the vehicle has changed, wherein the method for determining whether the position has changed includes: based on the current position coordinates, license plate number, license plate position and detection time obtained after the vehicle is first identified by an image acquisition device, determining the vehicle information of the vehicle, the specific time of arrival at the illegal parking area and the specific position of the vehicle, and then calculating the distance or angle change between the position coordinates of the vehicle at consecutive time points. If the distance or angle change is less than the preset threshold, it is considered that the position of the vehicle has not changed.

[0044] Exemplarily, the preset time threshold is set based on factors such as traffic management requirements, road congestion conditions, or vehicle parking habits, and can be a fixed time period or a dynamically adjusted time range.

[0045] S150: If the vehicle is parked illegally, a report is automatically generated, and a system prompt message with an illegal parking sign is generated.

[0046] In some embodiments, if the vehicle is illegally parked, the vehicle's license plate number, current location coordinates, illegal parking area information, detection time, and illegal parking snapshot are automatically obtained, and the obtained information is integrated into a formatted report. The report is stored in the illegal parking database of the traffic management department, and a system prompt message with an illegal parking sign is generated.

[0047] In some embodiments, after determining that the vehicle is illegally parked, the method further includes: Send out an alarm and generate an alarm log, wherein the alarm log includes at least: the vehicle license plate number, current location coordinates, illegal parking area information, illegal parking time, illegal parking snapshot and penalty information; and / or, Record the sending status of the alarm log and / or the feedback information of the vehicle owner.

[0048] In the present disclosure, firstly, the image frames of preset time intervals can be extracted by acquiring the real-time video stream in the traffic checkpoint monitoring; and the parking space is identified based on the image frame to generate the parking space area detection result, and the range of the prohibited parking area is extracted in reverse; before, after, or at the same time; the vehicle and license plate detection is performed based on the image frame to obtain the current position coordinates and license plate information of the vehicle; secondly, it can be judged whether the vehicle is illegally parked according to whether the current position coordinates of the vehicle are within the range of the prohibited parking area; finally, if the vehicle is illegally parked, a report is automatically generated, and a system prompt message with an illegal parking sign is generated. In this way, the problems of high labor costs, low work efficiency and accuracy in traditional illegal parking inspections can be solved, which not only reduces the manpower and material costs and management difficulty in the inspection process, but also improves work efficiency, management stability and supervision accuracy.

[0049] In conjunction with a specific embodiment, a method 100 for detecting illegal parking provided by the present disclosure is described in detail below, as shown below: By means of monitoring equipment arranged at the traffic checkpoint, the video stream data in the monitoring scene of the traffic checkpoint is captured and obtained in real time, and a time interval is preset, and the video stream data obtained in real time is processed according to the preset time interval, and the corresponding image frames are accurately extracted from the video stream in units of time intervals, wherein, before extracting the image frames, the real-time video stream can be subjected to pre-processing operations such as denoising, enhancement or format conversion, so as to improve the efficiency and accuracy of subsequent image processing; The extracted image frames are stored or transmitted in real time through a data transmission network to a designated processing terminal or storage device for further analysis, processing or archiving, wherein the image frames can be classified and stored or transmitted according to classification criteria such as time sequence, vehicle type, traffic violation, etc., to facilitate subsequent retrieval and analysis.

[0050] Input the image frame into the trained DeepLab V3+ semantic segmentation algorithm model to obtain a parking area detection result; and based on the parking area detection result, reversely extract the coordinate range of the no-parking area, organize the coordinate data of all key points in the no-parking area, and use image processing algorithms or machine learning techniques to automatically identify and extract key points constituting the boundary of the no-parking area to ensure the accuracy and robustness of boundary recognition, and sort these boundary points to ensure the continuity of the boundary; The sorted boundary points are used to construct boundary information of the no-parking area, which can clearly describe the shape, size and location of the no-parking area; and the generated no-parking area data is verified and optimized to improve the accuracy and reliability of no-parking area recognition, including but not limited to removing noise points, correcting boundary errors or merging adjacent no-parking areas, wherein, The no-parking area data includes: coordinate data of all key points in the no-parking area and boundary information of the no-parking area.

[0051] Before, after, or simultaneously with the above steps, you can also: The image frame is input into a trained YOLOv8 target detection model, the vehicle and the license plate area in the image are located, and the current position coordinates of the vehicle are obtained; the license plate area is cropped to generate a license plate candidate image, and the license plate candidate image is segmented by characters, the final results are merged, and the license plate number is output, wherein the character segmentation of the license plate candidate image includes: The EASTNet model is combined with the OCR text detection model to perform shared feature joint training to obtain the EASTNet-OCR model, wherein the training method includes: obtaining a data set including license plate images and their corresponding text labels, wherein the data set covers a variety of license plate types, character arrangement methods, and interference factors; The combined model is trained based on the data set, part of the network structure is shared during the joint training process, and the parameters of the model are adjusted to minimize the joint loss function, wherein the formula of the joint loss function is as follows: , in is the Iou loss function, A represents the prediction box, and B represents the range of the annotation box; is the cross entropy loss function,N is the total number of character samples, C is the total number of character categories, It is i The sample in c The true labels of the categories, It is i The sample in c The predicted probability of each category, is the overall measure of the difference between the model prediction and the true label, and λ is the weight; Preprocessing the license plate candidate image to obtain a preprocessed license plate candidate image; The preprocessed license plate candidate image is input into the EASTNet-OCR model, so that the output of the EASTNet model can be directly used as the input feature of the OCR text detection model, thereby obtaining a list containing character or text line prediction results.

[0052] The preprocessed license plate candidate image is input into the EASTNet-OCR model, so that the output of the EASTNet model can be directly used as the input feature of the OCR text detection model, thereby obtaining a list containing character or text line prediction results.

[0053] The YOLOv8 object detection model is specifically trained for traffic scenes and license plate recognition to improve the accuracy and efficiency of vehicle and license plate area detection.

[0054] The EASTNet-OCR model achieves an organic combination of text detection and character recognition through joint training, thereby improving the overall performance of license plate character segmentation and recognition; wherein the preprocessing of the license plate candidate image includes: verifying and correcting the recognized license plate number to improve the accuracy and reliability of license plate recognition.

[0055] Judging whether the vehicle is illegally parked according to whether the current position coordinates of the vehicle are within the range of the prohibited parking area includes: Compare the current position coordinates of the vehicle with the preset no-parking area range to determine whether the vehicle is located in the no-parking area. If the vehicle is located in the no-parking area, start continuous tracking of the vehicle, record the time when the vehicle is first identified as the start time, and set a preset time threshold; within the preset time threshold, continuously obtain and update the position information of the vehicle, and compare the position information of the vehicle within the preset time threshold to determine whether the position of the vehicle has changed, wherein the method for determining whether the position has changed includes: based on the current position coordinates, license plate number, license plate position and detection time obtained after the vehicle is first identified by the image acquisition device, determine the vehicle information of the vehicle, the specific time of arrival at the illegal parking area and the specific position of the vehicle, and then calculate the distance or angle change between the position coordinates of the vehicle at consecutive time points. If the distance or angle change is less than the preset threshold, it is considered that the position of the vehicle has not changed; Exemplarily, the preset time threshold is set based on factors such as traffic management requirements, road congestion conditions, or vehicle parking habits, and can be a fixed time period or a dynamically adjusted time range.

[0056] If the vehicle is parked illegally, the license plate number, current location coordinates, illegal parking area information, detection time and illegal parking snapshot of the vehicle are automatically obtained, and the obtained information is integrated into a formatted report, and the report is stored in the illegal parking database of the traffic management department, and a system prompt message with an illegal parking sign is generated, and / or, If the vehicle is parked illegally, an alarm is issued and an alarm log is generated, wherein the alarm log includes at least: the vehicle's license plate number, current location coordinates, illegal parking area information, illegal parking time, illegal parking snapshot and penalty information; and / or, Record the sending status of the alarm log and / or the feedback information of the vehicle owner.

[0057] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0058] The above is an introduction to the method embodiment. The following is a further explanation of the scheme disclosed in the present invention through an apparatus embodiment.

[0059] Figure 2FIG. 4 shows a structural diagram of a parking violation detection device provided by an embodiment of the present disclosure, such as Figure 2 As shown, a parking violation detection device 200 may include: The image acquisition module 210 is used to acquire the real-time video stream in the traffic checkpoint monitoring and extract the image frames in the real-time video stream according to the preset time interval; The area extraction module 220 is used to perform parking space recognition based on the image frame, generate a parking space area detection result, and reversely extract the range of the prohibited parking area; before, after, or at the same time; The information acquisition module 230 is used to detect the vehicle and the license plate based on the image frame, and obtain the current position coordinates and license plate information of the vehicle; A violation judgment module 240, configured to judge whether the vehicle is illegally parked according to whether the current position coordinates of the vehicle are within the range of the prohibited parking area; The report generation module 250 is used to automatically generate a report if the vehicle is illegally parked, and generate a system prompt message with an illegal parking sign.

[0060] Understandably, Figure 2 Each module / unit in the illegal parking detection device 200 shown has the following functions: Figure 1 The functions of each step in the illegal parking detection method 100 shown can achieve its corresponding technical effects, and for the sake of brevity, they will not be repeated here.

[0061] Figure 3 A block diagram of an exemplary electronic device capable of implementing an embodiment of the present disclosure is shown. The electronic device 300 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 300 may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0062] like Figure 3As shown, the electronic device 300 may include a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0063] A number of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0064] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above, such as method 100. For example, in some embodiments, the method 100 may be implemented as a computer program product, including a computer program, which is tangibly contained in a computer-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the method 100 described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform the method 100 in any other appropriate manner (e.g., by means of firmware).

[0065] The various embodiments described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0066] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0067] In the context of the present disclosure, a computer-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0068] It should be noted that the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute method 100 and achieve the corresponding technical effect achieved by the embodiments of the present disclosure executing the method. For the sake of brevity, they are not repeated here.

[0069] In addition, the present disclosure also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method 100 is implemented.

[0070] To provide interaction with a user, the above-described embodiments may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0071] The embodiments described above may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0072] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0073] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of this disclosure can be achieved, and this document is not limited here.

[0074] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for detecting illegal parking, characterized in that: include: Acquire a real-time video stream from a traffic checkpoint monitoring station, and extract image frames from the real-time video stream according to a preset time interval; Perform parking space recognition based on the image frame, generate parking space area detection results, and reversely extract the range of the prohibited parking area; before, after, or at the same time; Perform vehicle and license plate detection based on the image frame to obtain the current position coordinates and license plate information of the vehicle; Determining whether the vehicle is illegally parked according to whether the current position coordinates of the vehicle are within the range of the prohibited parking area; If the vehicle is parked illegally, a report is automatically generated, and a system prompt message with an illegal parking sign is generated.

2. The method according to claim 1, characterized in that The performing of parking space recognition based on the image frame, generating a parking space area detection result, and reversely extracting the range of the prohibited parking area includes: Input the image frame into the trained DeepLab V3+ semantic segmentation algorithm model to obtain a parking area detection result; Based on the parking area detection result, the coordinate range of the no-parking area is extracted in reverse order, the coordinate data of all key points in the no-parking area are sorted, and then the points constituting the boundary of the no-parking area are identified and sorted, and the boundary information of the no-parking area is constructed using the sorted boundary points, wherein: The no-parking area data includes: coordinate data of all key points in the no-parking area and boundary information of the no-parking area.

3. The method according to claim 1, characterized in that The detecting of the vehicle and the license plate based on the image frame to obtain the current position coordinates and the license plate information of the vehicle includes: Input the image frame into a trained YOLOv8 target detection model, locate the vehicle and license plate area in the image, and obtain the current position coordinates of the vehicle; The license plate area is cropped to generate a license plate candidate image, and the license plate candidate image is segmented into characters, the final results are merged, and the license plate number is output.

4. The method according to claim 3, characterized in that: The character segmentation of the license plate candidate image comprises: The EASTNet model is combined with the OCR text detection model to perform joint training on shared features to obtain the EASTNet-OCR model, where: The training method includes: obtaining a data set including license plate images and corresponding text labels, wherein the data set includes a variety of license plate types, character arrangement methods, and interference factors; The combined model is trained based on the data set, part of the network structure is shared during the joint training process, and the parameters of the model are adjusted to minimize the joint loss function, wherein the formula of the joint loss function is as follows: ， in is the Iou loss function, A represents the prediction box, and B represents the range of the annotation box; is the cross entropy loss function, N is the total number of character samples, C is the total number of character categories, It is i The sample in c The true labels on the categories, It is i The sample in c The predicted probability of each category, is the overall measure of the difference between the model prediction and the true label, and λ is the weight; Preprocessing the license plate candidate image to obtain a preprocessed license plate candidate image; The preprocessed license plate candidate image is input into the EASTNet-OCR model, so that the output of the EASTNet model can be directly used as the input feature of the OCR text detection model, thereby obtaining a list containing character or text line prediction results.

5. The method according to claim 3, characterized in that: The determining whether the vehicle is illegally parked according to whether the current position coordinates of the vehicle are within the range of the prohibited parking area includes: If the current position coordinates of the vehicle are within the range of the prohibited parking area, it is further determined whether the position of the vehicle changes within a preset time threshold, wherein: The judgment method includes: determining the vehicle information of the vehicle, the specific time of arrival at the illegal parking area and the specific location of the vehicle based on the current position coordinates, license plate number, license plate position and detection time obtained after the vehicle is first identified by the image acquisition device, and then continuously tracking and comparing the position information of the vehicle within a preset time threshold. If the position information of the vehicle does not change within the preset time threshold, it is determined that the vehicle is illegally parked.

6. The method according to claim 1, characterized in that If the vehicle is parked illegally, a report is automatically generated, and a system prompt message with an illegal parking sign is generated, including: If the vehicle is parked illegally, the vehicle's license plate number, current location coordinates, illegal parking area information, detection time and illegal parking snapshot are automatically obtained, and the obtained information is integrated into a formatted report. The report is stored in the illegal parking database of the traffic management department, and a system prompt message with an illegal parking sign is generated.

7. The method according to claim 1, characterized in that After determining that the vehicle is parked illegally, the method further includes: Send out an alarm and generate an alarm log, wherein the alarm log includes at least: the vehicle license plate number, current location coordinates, illegal parking area information, illegal parking time, illegal parking snapshot and penalty information; and / or, Record the sending status of the alarm log and / or the feedback information of the vehicle owner.

8. A parking violation detection device, characterized in that: include: An image acquisition module is used to acquire a real-time video stream from traffic checkpoint monitoring and extract image frames from the real-time video stream according to a preset time interval; A region extraction module is used to perform parking space recognition based on the image frame, generate a parking space region detection result, and reversely extract the range of the prohibited parking area; before, after, or at the same time; An information acquisition module, used to detect the vehicle and the license plate based on the image frame, and acquire the current position coordinates and license plate information of the vehicle; A violation judgment module, used to judge whether the vehicle is illegally parked according to whether the current position coordinates of the vehicle are within the range of the prohibited parking area; The report generation module is used to automatically generate a report if the vehicle is illegally parked, and generate a system prompt message with an illegal parking sign.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images