Vehicle illegal parking detection method and device, storage medium and electronic equipment

Abstract

This disclosure provides a method, apparatus, storage medium, and electronic device for detecting illegally parked vehicles, relating to the field of visual detection technology. The method includes: acquiring video data of a monitored area, wherein the monitored area includes a no-parking zone; monitoring for the presence of a target vehicle entering the no-parking zone based on the video data; if a target vehicle is present, acquiring its characteristic information; determining the target vehicle's location information at multiple time points based on the characteristic information and video data; determining whether the target vehicle's dwell time in the no-parking zone exceeds a preset time based on the target vehicle's location information at multiple time points; if it exceeds the preset time, determining that the target vehicle is illegally parked. In this embodiment, the characteristic information of the target vehicle is introduced to identify it, which can effectively determine whether the same vehicle is parked in the no-parking zone, reducing the false detection rate and making the detection results more accurate.

Description

Technical Field

[0001] This disclosure relates to the field of visual inspection technology, and in particular to a method and device for detecting illegally parked vehicles, a storage medium, and an electronic device. Background Technology

[0002] With the development of society and economy, the number of vehicles has increased dramatically, and the contradiction between parking demand and parking space supply has become increasingly prominent, leading to illegal parking in real life.

[0003] Illegal parking can seriously obstruct traffic and affect public safety.

[0004] With the development of video detection technology and computer vision technology, surveillance equipment is increasingly being used to detect illegally parked vehicles in no-parking zones. However, current detection methods can generally only detect the presence of vehicles in no-parking zones, but cannot effectively determine whether a vehicle is parked in a no-parking zone, resulting in low accuracy of the detection results.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this disclosure is to provide a method, device, storage medium, and electronic device for detecting illegal parking of vehicles, which at least to some extent overcomes the problem that related technologies can only detect whether a vehicle exists in a no-parking area, but cannot effectively determine whether a vehicle is parked in a no-parking area, resulting in low accuracy of the detection results.

[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0008] According to one aspect of this disclosure, a method for detecting illegally parked vehicles is provided, comprising:

[0009] Collect video data of the monitored area, which includes: no-parking zones for vehicles;

[0010] Based on video data, monitor for the presence of target vehicles entering the no-parking zone;

[0011] If a target vehicle exists, obtain its characteristic information;

[0012] Based on the target vehicle's feature information and video data, determine the target vehicle's location information at multiple time points;

[0013] Based on the location information of the target vehicle at multiple time points, determine whether the duration of the target vehicle's stay in the no-parking zone exceeds the preset time limit;

[0014] If the preset time limit is exceeded, the target vehicle will be identified as an illegally parked vehicle.

[0015] In one embodiment of this disclosure, obtaining the feature information of the target vehicle includes:

[0016] The video data is input into a pre-trained neural network model, which outputs the feature information of the target vehicle.

[0017] In one embodiment of this disclosure, determining the location information of the target vehicle at multiple time points based on the target vehicle's feature information and video data includes:

[0018] The video data is input into the target detection model, and the detection box information of multiple video frame images is output, where each video frame image corresponds to a time point;

[0019] Based on the detection bounding box information in multiple video frame images, the location information of the target vehicle at multiple time points is determined.

[0020] In one embodiment of this disclosure, the target detection model also outputs confidence information, which represents the confidence level that the detected object within the detection box is a vehicle.

[0021] Based on the bounding box information in the video frame image, the location information of the target vehicle is determined, including:

[0022] The location information of the target vehicle is determined based on the detection box information with a confidence level greater than the preset confidence level.

[0023] In one embodiment of this disclosure, after monitoring whether a target vehicle has entered a no-parking zone based on video data, the method further includes:

[0024] Based on the vehicle's first position at time T-1 in the video data, predict the vehicle's position at time T to obtain the predicted position;

[0025] If the distance between the vehicle's second position and the predicted position at time T is greater than the first preset distance value, the distance between the second position and the first position is calculated.

[0026] Obtain the characteristic information of the target vehicle, including:

[0027] If the distance between the second position and the first position is less than a second preset distance value, the feature information of the target vehicle is obtained.

[0028] In one embodiment of this disclosure, determining the location information of the target vehicle at multiple time points based on the target vehicle's feature information and video data includes:

[0029] Based on the feature information of the target vehicle, N first images are determined from multiple video frame images in the video data. The first image is a video frame image containing the target vehicle, and N is a positive integer.

[0030] Obtain the location information of the target vehicle in each first image and the time information associated with that location information.

[0031] In one embodiment of this disclosure, N first images are determined from video frame images of video data based on feature information of the target vehicle, including:

[0032] The video frame images of the detected video data are used to determine M second images from multiple video frame images. The second images are video frame images containing vehicles, where M is a positive integer and M is not less than N.

[0033] Obtain feature information of the vehicle in the second image;

[0034] Based on the feature information of the vehicle in the second image and the feature information of the target vehicle, N first images are determined from M second images, and the similarity value between the feature information of the vehicle in the second image and the feature information of the target vehicle is greater than a preset similarity threshold.

[0035] According to another aspect of this disclosure, a vehicle illegal parking detection device is provided, comprising:

[0036] The acquisition module is used to acquire video data of the monitored area, which includes: no-parking zones for vehicles;

[0037] The monitoring module is used to monitor whether there are target vehicles entering the no-parking zone based on video data;

[0038] The acquisition module is used to acquire the characteristic information of the target vehicle if it exists.

[0039] The first determining module is used to determine the location information of the target vehicle at multiple time points based on the target vehicle's feature information and video data;

[0040] The judgment module is used to determine whether the target vehicle stays in the no-parking zone for a longer period of time than a preset time, based on the location information of the target vehicle at multiple time points.

[0041] The second determination module is used to determine that the target vehicle is illegally parked if a preset time period is exceeded.

[0042] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-described vehicle illegal parking detection method by executing the executable instructions.

[0043] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described method for detecting illegally parked vehicles.

[0044] The vehicle illegal parking detection method provided in this disclosure, after detecting a target vehicle entering a no-parking zone, acquires the target vehicle's feature information; and determines the target vehicle's location information at multiple time points based on the target vehicle's feature information and video data; based on the target vehicle's location information at multiple time points, it determines whether the target vehicle's dwell time in the no-parking zone exceeds a preset time; if it exceeds the preset time, the target vehicle is determined to be illegally parked. Thus, by introducing the target vehicle's feature information to identify the target vehicle in the video data, it can effectively determine whether the same vehicle is parked in the no-parking zone, thereby more accurately identifying target vehicles whose dwell time exceeds the preset time as illegally parked vehicles, reducing the false detection rate, and making the detection results more accurate.

[0045] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0047] Figure 1 This diagram illustrates a computer system architecture according to an embodiment of the present disclosure;

[0048] Figure 2 This diagram illustrates a flowchart of a method for detecting illegally parked vehicles according to an embodiment of the present disclosure;

[0049] Figure 3 A detailed flowchart of step 240 in an embodiment of this disclosure is shown;

[0050] Figure 4 This diagram illustrates another method for detecting illegally parked vehicles in an embodiment of this disclosure.

[0051] Figure 5 This diagram illustrates a flowchart of the target vehicle identification process in an embodiment of this disclosure.

[0052] Figure 6 This diagram illustrates a vehicle detection device for illegally parked vehicles according to an embodiment of the present disclosure.

[0053] Figure 7A structural block diagram of a computer device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0054] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0055] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0056] As described in the background section, current detection methods can generally only detect whether a vehicle exists in a no-parking zone, but cannot effectively determine whether a vehicle is parked in a no-parking zone, resulting in low accuracy of the detection results.

[0057] Specifically, the inventors discovered that in related technologies, illegal parking detection based on surveillance cameras involves detecting vehicles in frames T_1 and T_2 of a video sequence, obtaining their detection boxes respectively, and if the vehicle is in a no-parking zone, the intersection-over-union (IOU) ratio of the two frames is used to determine whether the vehicle is stopped. If the IOU is greater than a threshold, the vehicle is considered not to be moving and is in a stopped state, thus constituting illegal parking.

[0058] This method determines whether a vehicle has stopped solely based on the intersection-union ratio of the vehicle detection bounding boxes, failing to confirm whether it is the same vehicle or different vehicles passing through the same location, resulting in a high false alarm rate for illegal parking detection. Although some methods utilize license plate detection to address this issue, they are difficult to implement in scenarios with obstructions, or in recognition scenarios involving long distances or high angles.

[0059] Furthermore, when a vehicle is obstructed at a certain moment, the detection frame will change significantly, resulting in a lower overlap ratio between the two detection frames, making it impossible to determine if the vehicle has stopped, and leading to a high false negative rate.

[0060] In the solution provided in this disclosure, after detecting a target vehicle entering a no-parking zone, the feature information of the target vehicle is acquired; and based on the feature information and video data, the location information of the target vehicle at multiple time points is determined; based on the location information of the target vehicle at multiple time points, it is determined whether the duration of the target vehicle's stay in the no-parking zone exceeds a preset time; if it exceeds the preset time, the target vehicle is determined to be an illegally parked vehicle. In this embodiment, the feature information of the target vehicle is introduced to identify the target vehicle, which can effectively determine whether the same vehicle is parked in the no-parking zone, reducing the false detection rate and making the detection results more accurate.

[0061] To facilitate understanding, the following is an explanation of several terms used in this disclosure.

[0062] Computer vision technology is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multi-dimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D (3D) technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0063] Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0064] A program model can be a formalized representation of a solution to a real-world problem. It can be a set of ordered formulas for solving the problem, a problem-solving process, or a computer language program module used to solve a specific real-world problem.

[0065] The solutions provided in this disclosure relate to technologies such as computer vision, and are specifically illustrated through the following embodiments:

[0066] Figure 1 This is a schematic diagram of the structure of a computer system provided in an exemplary embodiment of this disclosure. The system includes: a plurality of terminals 120 and a server cluster 140.

[0067] Terminal 120 can be a mobile terminal such as a mobile phone, game console, tablet computer, e-book reader, smart glasses, MP4 (Moving Picture Experts Group Audio Layer IV) player, smart home device, AR (Augmented Reality) device, VR (Virtual Reality) device, etc. Alternatively, terminal 120 can also be a personal computer (PC), such as a laptop computer and a desktop computer, etc.

[0068] The terminal 120 may contain an application for providing video data of the monitored area.

[0069] Terminal 120 is connected to server cluster 140 via a communication network. Optionally, the communication network can be a wired network or a wireless network.

[0070] Server cluster 140 is a single server, or a combination of several servers, or a virtualization platform, or a cloud computing service center. Server cluster 140 provides background services for applications that provide… Optionally, server cluster 140 undertakes the primary computing work, and terminal 120 undertakes secondary computing work; or, server cluster 140 undertakes secondary computing work, and terminal 120 undertakes the primary computing work; or, terminal 120 and server cluster 140 collaborate on computing using a distributed computing architecture.

[0071] In some alternative embodiments, server cluster 140 is used to store… information.

[0072] Optionally, the application clients installed on different terminals 120 may be the same, or the application clients installed on two terminals 120 may be clients of the same type of application on different control system platforms. Depending on the terminal platform, the specific form of the application client may also differ; for example, the application client may be a mobile client, a PC client, or a World Wide Web (WWW) client.

[0073] Those skilled in the art will understand that the number of terminals 120 described above can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more. This disclosure does not limit the number of terminals or the type of device.

[0074] Optionally, the system may also include a management device ( Figure 1 (Not shown), the management device is connected to the server cluster 140 via a communication network. Optionally, the communication network can be a wired network or a wireless network.

[0075] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0076] The following will describe in more detail each step of the vehicle illegal parking detection method in this exemplary embodiment, with reference to the accompanying drawings and embodiments.

[0077] Figure 2 This diagram illustrates a flowchart of a vehicle illegal parking detection method according to an embodiment of this disclosure. The method provided in this embodiment can be executed by any electronic device with computing power, such as... Figure 1 Terminal 120 and / or server cluster 140 in the example. In the following illustration, terminal 120 is used as the execution subject.

[0078] like Figure 2As shown, the method for detecting illegally parked vehicles includes the following steps:

[0079] Step S210: Collect video data of the monitored area, wherein the monitored area includes: no-parking zones for vehicles;

[0080] Step S220: Based on the video data, monitor whether there are any target vehicles entering the no-parking zone;

[0081] Step S230: If a target vehicle exists, obtain the characteristic information of the target vehicle;

[0082] Step S240: Determine the location information of the target vehicle at multiple time points based on the characteristic information and video data of the target vehicle;

[0083] Step S250: Based on the location information of the target vehicle at multiple time points, determine whether the duration of the target vehicle's stay in the no-parking zone exceeds the preset duration.

[0084] Step S260: If the preset time exceeds the limit, the target vehicle is determined to be an illegally parked vehicle.

[0085] The vehicle illegal parking detection method provided in this disclosure, after detecting a target vehicle entering a no-parking zone, acquires the target vehicle's feature information; and determines the target vehicle's location information at multiple time points based on the target vehicle's feature information and video data; based on the target vehicle's location information at multiple time points, it determines whether the target vehicle's dwell time in the no-parking zone exceeds a preset time; if it exceeds the preset time, the target vehicle is determined to be illegally parked. Thus, by introducing the target vehicle's feature information to identify the target vehicle in the video data, it can effectively determine whether the same vehicle is parked in the no-parking zone, thereby more accurately identifying target vehicles whose dwell time exceeds the preset time as illegally parked vehicles, reducing the false detection rate, and making the detection results more accurate.

[0086] The above steps are explained in detail below:

[0087] In step S210 above, video data of the monitored area is collected. There are various ways to collect video data, such as through an image acquisition device on the terminal. Alternatively, a separate image acquisition device can be used to collect video data of the monitored area. This image acquisition device is connected to the terminal, and the terminal obtains the video data of the monitored area through this image acquisition device.

[0088] In one example, to facilitate later processing of the video data, video data from a fixed area can be captured. This makes it easier to determine the location information during data processing. To capture video data from a fixed area, a fixed-position acquisition device can be used. By fixing the position of the acquisition device, the acquisition angle is fixed, thus obtaining video data from a fixed position.

[0089] Here, the no-parking zone can be an area that is manually set up. The size and shape of this area are not limited.

[0090] In the example above, since video data from a fixed area is collected, the location of the no-stopping zone for vehicles remains unchanged in the resulting video frame images, which facilitates subsequent processing of the video data.

[0091] In step S220, based on the video data, it is monitored whether any target vehicles have entered the no-parking zone. This can be achieved using a computer program model. Specific monitoring methods are detailed in the following embodiments and will not be repeated here.

[0092] The target vehicle's characteristic information can be information that distinguishes the target vehicle from other vehicles. This characteristic information may include one or more of the following: vehicle type, vehicle color, vehicle model, etc.

[0093] The vehicle type can include trucks, buses, and other types, as well as public buses and cars.

[0094] The video data includes multiple video frame images. In step S240, the position of the target vehicle in the video frame image and the corresponding time point can be determined based on the feature information of the target vehicle and the time point corresponding to the multiple video frame images. The position information of the target vehicle can be determined based on the position of the target vehicle in the video frame image.

[0095] It should be noted that the target vehicle's location information can be either its position within a video frame or its actual location determined based on the video data. When the acquired video data is from a fixed angle, the target vehicle's location information can be its position within the video frame.

[0096] The duration of a target vehicle's stay in a no-parking zone can be determined based on the video frames containing the target vehicle mentioned earlier. Each video frame containing the target vehicle corresponds to a time point, and the duration of stay can be calculated based on consecutive video frames.

[0097] Setting a preset time limit can prevent abnormal detection results caused by slow vehicle movement. Therefore, setting a preset time limit, and determining the target vehicle as illegally parked if the preset time limit is exceeded, can further improve the accuracy of detection results.

[0098] In some embodiments, to improve the efficiency of detecting illegally parked vehicles, step S230 can be implemented based on a computer program model. Therefore, obtaining the feature information of the target vehicle in step S230 may include:

[0099] The video data is input into a pre-trained neural network model, which outputs the feature information of the target vehicle.

[0100] As an example, the neural network model can be a Pytorch deep learning model. Before using this model, it can be pre-trained based on labeled video data; the specific training process will not be detailed here.

[0101] Here, after the video data is input into a pre-trained neural network model, the neural network model can identify the features of the target vehicle in the video data based on the learned prior knowledge, and then output the feature information of the target vehicle.

[0102] In this embodiment of the disclosure, the feature information of the target vehicle is identified by a pre-trained neural network model. The feature extraction process can be implemented automatically, which can effectively improve the speed of extracting the feature information of the target vehicle, thereby improving the efficiency of detecting illegally parked vehicles.

[0103] In some embodiments, to further improve detection efficiency, step S240 can also be implemented using a computer program model. Based on this, as... Figure 3 As shown, step S240 above may include:

[0104] Step S302: Input the video data into the target detection model so that the target detection model outputs the detection box information in multiple video frame images of the video data, wherein each video frame image corresponds to a time point;

[0105] Step S304: Based on the detection box information in multiple video frame images, determine the location information of the target vehicle at multiple time points.

[0106] The target detection model can be the YOLOv5 target detection model.

[0107] Specifically, in conjunction with the above embodiments, a deep neural network can be trained using the YOLOv5 object detection method based on the PYTORCH deep learning framework to achieve real-time vehicle object detection.

[0108] The model takes T-frame video data as input, performs object detection using YOLOv5, and outputs vehicle detection information for that frame, denoted as . in For the i-th vehicle target being detected, the location information of the target is included. This indicates the coordinates of the top-left point of the detection box. Indicates the width of the detection box. This indicates the height of the detection frame.

[0109] In some embodiments, the target detection model described above also outputs confidence information, which represents the confidence that the detected object within the detection box is a vehicle.

[0110] Step S304 may specifically include determining the location information of the target vehicle based on the detection box information whose confidence information is greater than a preset confidence level.

[0111] Here, confidence information is used to filter reliable detection box information, making the data detected by users more accurate, thereby improving the accuracy of detecting illegally parked vehicles.

[0112] Furthermore, in this embodiment of the present disclosure, the detection box information in the video frame image is obtained by using the target detection model, and the location information of the target vehicle is obtained through the detection box information, which can effectively improve the speed of obtaining the location information of the target vehicle, thereby improving the efficiency of detecting illegally parked vehicles.

[0113] In some embodiments, the method may further include a monitoring process for the target vehicle, such as... Figure 4 As shown, the process may include the following steps:

[0114] Step S410: Based on the vehicle's first position at time T-1 in the video data, predict the vehicle's position at time T to obtain the predicted position;

[0115] Step S420: If the distance between the second position and the predicted position of the vehicle at time T is greater than the first preset distance value, calculate the distance between the second position and the first position.

[0116] Accordingly, step S230 in the above embodiments may include:

[0117] If the distance between the second position and the first position is less than a second preset distance value, the feature information of the target vehicle is obtained.

[0118] The monitoring process for the aforementioned target vehicles is described in detail below; please refer to it accordingly. Figure 4 and Figure 5 .

[0119] When T = 0, the vehicle trajectory information Track0 is initialized with Detection0; when T ≥ 1, the state at time T is predicted using the Kalman filter algorithm based on the vehicle information at time T-1.

[0120] Assume the vehicle's status information is as follows:

[0121] x T-1 =[t T-1 ,l T-1 ,α T-1 ,h T-1 ,v t ,v l ,v α ,v h ] T (1)

[0122] in, The aspect ratio is 1.

[0123] The covariance matrix is

[0124]

[0125] Define the prediction matrix F k for:

[0126]

[0127] The predicted result is

[0128] Based on the predicted trajectory information With detection box T The IOU value is calculated using the following formula:

[0129]

[0130] Based on the IOU matrix, the Hungarian matching algorithm is used, along with a set threshold T. IOU Determine the set of matching trajectories S matched Trajectory set S for which no associated items were found unmatched_track and the set of detection boxes S for which no related items were found. unmatched_detection .

[0131] For the matched S matched The set is used to update the predicted state of the Track using the current detection bounding box, i.e.:

[0132]

[0133] P′ k =P k -K′H k Pk (6)

[0134]

[0135] Among them, H k It is an identity matrix.

[0136] Update vehicle trajectory collection information

[0137] Track_set T =S matched +S unmatched_track +S unmatched_detection

[0138] Among them, for S unmatched_track Add time Δt to its attribute time_loss, S matched Set time_loss to 0, S unmatched_detection Set time_loss to 0. Delete time_loss>T loss The trajectory information.

[0139] The trajectory set S mentioned above matched This refers to the set of trajectories of the target vehicle. The process of updating the vehicle trajectory set information is as described in the previous embodiment: if a preset time period is exceeded, the target vehicle is determined to be an illegally parked vehicle.

[0140] In this embodiment of the disclosure, the target vehicle can be accurately identified, providing accurate data for subsequent detection and thus improving the accuracy of the detection results.

[0141] Furthermore, this embodiment utilizes a ResNet-50 neural network to extract vehicle feature information, effectively determining whether the same vehicle is parked in a no-parking zone and whether the parking duration exceeds a threshold. Even when obstructions exist, the system can trace the vehicle and accumulate illegal parking duration using saved historical vehicle information after the obstruction is removed, thereby reducing the false negative and false positive rates and improving the accuracy of illegal parking detection.

[0142] The following section provides a detailed description of the method for detecting illegally parked vehicles in this disclosure, using the specific embodiments for target vehicle identification described above as examples.

[0143] For S matched Detecting illegally parked vehicles in the cluster.

[0144] For S matched The matching track and detection are determined by calculating whether the offset of the center point of their detection boxes is less than a threshold T. distance This is used to determine whether the vehicle is moving. The formula for calculating the center point is as follows:

[0145]

[0146] The formula for calculating the offset is as follows:

[0147]

[0148] If the vehicle is not moving, the ResNet-50 network is used to extract the vehicle's feature information, and the similarity between the two is calculated using cosine distance. If the similarity is greater than a threshold T, the similarity is considered. similary If the time interval Δt is not found, it is considered to be the same vehicle, and the time_illegal value in the Track attribute is increased by the time interval Δt.

[0149] For S unmatched_track With S unmatched_detection The vehicles in the dataset are processed by extracting their feature information using a ResNet-50 network, and those with a similarity greater than a threshold T are selected. similary The track and detection are correlated, and S is updated. matched S unmatched_track , tection The set is used to determine the offset of the center points of the two sets and whether they have moved. If they have not moved, the time interval Δt is added to the time_illegal attribute of the track.

[0150] Filter the Track set for tracks where the duration of illegal parking is greater than T. parking Vehicles that are illegally parked beyond a preset time limit will be recorded and processed.

[0151] In some embodiments, step S240 above can also be implemented as follows:

[0152] Based on the feature information of the target vehicle, N first images are determined from multiple video frame images in the video data. The first image is a video frame image containing the target vehicle, and N is a positive integer.

[0153] Obtain the location information of the target vehicle in each first image and the time information associated with that location information.

[0154] Specifically, determining N first images from the video frame images of the video data based on the feature information of the target vehicle can include:

[0155] The video frame images of the detected video data are used to determine M second images from multiple video frame images. The second images are video frame images containing vehicles, where M is a positive integer and M is not less than N.

[0156] Obtain feature information of the vehicle in the second image;

[0157] Based on the feature information of the vehicle in the second image and the feature information of the target vehicle, N first images are determined from M second images, and the similarity value between the feature information of the vehicle in the second image and the feature information of the target vehicle is greater than a preset similarity threshold.

[0158] In this embodiment of the disclosure, the location information of the target vehicle and its associated time information can be accurately obtained based on the characteristic information and video data of the target vehicle, providing accurate data for subsequent detection and processing, and improving the accuracy of illegal parking vehicle detection.

[0159] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.

[0160] It should be noted that the vehicle illegal parking detection method provided in this disclosure can also be executed by a vehicle illegal parking detection device, or by a control module within that device for executing the method. This disclosure uses the example of a vehicle illegal parking detection device executing the method to illustrate the vehicle illegal parking detection device provided in this disclosure.

[0161] Figure 6 A schematic diagram of a vehicle illegal parking detection device according to an embodiment of this disclosure is shown. Figure 6 As shown, the vehicle illegal parking detection device 600 includes:

[0162] The acquisition module 610 is used to acquire video data of the monitored area, which includes: a no-parking zone for vehicles;

[0163] The monitoring module 620 is used to monitor whether there are target vehicles entering the no-parking zone based on video data;

[0164] The acquisition module 630 is used to acquire the feature information of the target vehicle if the target vehicle exists.

[0165] The first determining module 640 is used to determine the location information of the target vehicle at multiple time points based on the characteristic information of the target vehicle and video data.

[0166] The judgment module 650 is used to determine whether the duration of the target vehicle's stay in the no-parking zone exceeds a preset time limit based on the target vehicle's location information at multiple time points.

[0167] The second determination module 660 is used to determine that the target vehicle is an illegally parked vehicle if a preset time period is exceeded.

[0168] In some embodiments, the acquisition module 630 may be specifically used to input video data into a pre-trained neural network model and output feature information of the target vehicle.

[0169] In some embodiments, the first determining module 640 may include:

[0170] The input unit is used to input video data into the target detection model and output the detection box information in multiple video frame images of the video data, wherein each video frame image corresponds to a time point;

[0171] The first determining unit is used to determine the location information of the target vehicle at multiple time points based on the detection box information in multiple video frame images.

[0172] In some embodiments, the target detection model in the input unit can also output confidence information, which represents the confidence that the detected object within the detection box is a vehicle.

[0173] The first determining unit can be specifically used to determine the location information of the target vehicle based on the detection box information with confidence information greater than a preset confidence level.

[0174] In some embodiments, the vehicle illegal parking detection device 600 may further include:

[0175] The prediction module can be used to predict the vehicle's position at time T based on the vehicle's first position at time T-1 in the video data, thus obtaining the predicted position.

[0176] The calculation module can be used to calculate the distance between the second position and the first position when the distance between the second position and the predicted position of the vehicle at time T is greater than the first preset distance value.

[0177] Accordingly, the acquisition module 630 can be specifically used to acquire the feature information of the target vehicle when the distance between the second position and the first position is less than a second preset distance value.

[0178] In some embodiments, the first determining module 640 may include:

[0179] The second determining unit is used to determine N first images from multiple video frame images of video data based on the feature information of the target vehicle. The first images are video frame images containing the target vehicle, and N is a positive integer.

[0180] The acquisition unit is used to acquire the location information of the target vehicle in each first image and the time information associated with the location information.

[0181] In some embodiments, the second determining unit may be specifically implemented as follows:

[0182] The video frame images of the detected video data are used to determine M second images from multiple video frame images. The second images are video frame images containing vehicles, where M is a positive integer and M is not less than N.

[0183] Obtain feature information of the vehicle in the second image;

[0184] Based on the feature information of the vehicle in the second image and the feature information of the target vehicle, N first images are determined from M second images, and the similarity value between the feature information of the vehicle in the second image and the feature information of the target vehicle is greater than a preset similarity threshold.

[0185] The vehicle illegal parking detection device provided in this embodiment can be used to execute the vehicle illegal parking detection methods provided in the above-described method embodiments. The implementation principle and technical effect are similar, and will not be described in detail here for the sake of brevity.

[0186] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuits,” “modules,” or “systems.”

[0187] The following reference Figure 7 To describe an electronic device 700 according to this embodiment of the present invention. Figure 7 The electronic device 700 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0188] like Figure 7 As shown, the electronic device 700 is manifested in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including storage unit 720 and processing unit 710).

[0189] The storage unit stores program code that can be executed by the processing unit 710, causing the processing unit 710 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 710 can perform actions such as... Figure 2Step 210, as shown, involves collecting video data from the monitored area, which includes a no-parking zone; Step 220, based on the video data, monitoring for the presence of a target vehicle entering the no-parking zone; Step 230, if a target vehicle is present, acquiring its characteristic information; Step 240, based on the target vehicle's characteristic information and the video data, determining the target vehicle's location information at multiple time points; Step 250, based on the target vehicle's location information at multiple time points, determining whether the target vehicle's dwell time in the no-parking zone exceeds a preset time limit; Step 260, if the preset time limit is exceeded, determining that the target vehicle is an illegally parked vehicle.

[0190] Storage unit 720 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 7201 and / or cache memory 7202, and may further include a read-only memory (ROM) 7203.

[0191] The storage unit 720 may also include a program / utility 7204 having a set (at least one) program module 7205, such program module 7205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0192] Bus 730 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0193] Electronic device 700 can also communicate with one or more external devices 800 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 700, and / or with any device that enables electronic device 700 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 750. Furthermore, electronic device 700 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 760. As shown, network adapter 760 communicates with other modules of electronic device 700 via bus 730. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0194] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0195] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section of this specification.

[0196] A program product for implementing the above-described method according to embodiments of the present invention is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0197] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0198] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0199] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0200] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0201] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0202] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0203] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0204] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A method for detecting illegally parked vehicles, characterized in that, include: Video data of a monitored area is collected by a fixed-location acquisition device. The monitored area includes a no-parking zone for vehicles. The video data is collected in monitoring scenarios where license plate information is difficult to obtain effectively. Based on the video data, monitor whether any target vehicles have entered the no-parking zone. Based on the vehicle's first position at time T-1 in the video data, the vehicle's position at time T is predicted to obtain the predicted position. If the distance between the second position and the predicted position of the vehicle at time T is greater than a first preset distance value, the distance between the second position and the first position is calculated; if the distance between the second position and the first position is less than a second preset distance value, the feature information of the target vehicle is obtained. The feature information of the target vehicle is feature information that can distinguish the target vehicle from other vehicles. The feature information includes one or more of the following: vehicle type, vehicle color, and vehicle model. Based on the feature information of the target vehicle and the video data, the location information of the target vehicle at multiple time points is determined; wherein, the similarity value between the feature information of the vehicles corresponding to the location information at the multiple time points is greater than a preset similarity threshold. Based on the location information of the target vehicle at multiple time points, determine whether the duration of the target vehicle's stay in the vehicle no-parking area exceeds a preset duration; If the preset time limit is exceeded, the target vehicle is determined to be an illegally parked vehicle.

2. The method according to claim 1, characterized in that, The acquisition of the feature information of the target vehicle includes: The video data is input into a pre-trained neural network model, which outputs the feature information of the target vehicle.

3. The method according to claim 1, characterized in that, The step of determining the location information of the target vehicle at multiple time points based on the feature information of the target vehicle and the video data includes: The video data is input into the target detection model, and the detection box information in multiple video frame images of the video data is output, wherein each video frame image corresponds to a time point; Based on the detection box information in the multiple video frame images, the location information of the target vehicle at multiple time points is determined.

4. The method according to claim 3, characterized in that, The target detection model also outputs confidence information, which indicates the confidence level that the detected object within the detection box is a vehicle. Determining the location information of the target vehicle based on the detection bounding box information in the video frame image includes: The location information of the target vehicle is determined based on the detection box information whose confidence level is greater than the preset confidence level.

5. The method according to claim 1, characterized in that, The step of determining the location information of the target vehicle at multiple time points based on the feature information of the target vehicle and the video data includes: Based on the feature information of the target vehicle, N first images are determined from multiple video frame images of the video data. The first images are video frame images containing the target vehicle, and N is a positive integer. The location information of the target vehicle in each first image and the time information associated with the location information are obtained.

6. The method according to claim 5, characterized in that, The step of determining N first images from the video frame images of the video data based on the feature information of the target vehicle includes: Detect video frame images of the video data, and determine M second images from the multiple video frame images, where the second images are video frame images containing vehicles, M is a positive integer, and M is not less than N; Obtain the feature information of the vehicle in the second image; Based on the feature information of the vehicle in the second image and the feature information of the target vehicle, N first images are determined from the M second images, wherein the similarity value between the feature information of the vehicle in the second image and the feature information of the target vehicle is greater than a preset similarity threshold.

7. A vehicle illegal parking detection device, characterized in that, include: The acquisition module is used to acquire video data of a monitored area through acquisition devices at fixed locations. The monitored area includes a no-parking zone for vehicles. The video data is acquired in monitoring scenarios where license plate information is difficult to obtain effectively. The monitoring module is used to monitor whether there are target vehicles entering the no-parking zone based on the video data; The prediction module is used to predict the vehicle's position at time T based on the vehicle's first position at time T-1 in the video data, thus obtaining the predicted position. The calculation module is used to calculate the distance between the second position and the first position when the distance between the second position and the predicted position of the vehicle at time T is greater than the first preset distance value. The acquisition module is used to acquire feature information of the target vehicle when the distance between the second position and the first position is less than a second preset distance value. The feature information of the target vehicle is feature information that can distinguish the target vehicle from other vehicles. The feature information includes one or more of the following: vehicle type, vehicle color, and vehicle model. The first determining module is used to determine the location information of the target vehicle at multiple time points based on the feature information of the target vehicle and the video data; wherein the similarity value between the feature information of the vehicles corresponding to the location information at the multiple time points is greater than a preset similarity threshold. The judgment module is used to determine whether the duration of the target vehicle's stay in the vehicle no-parking area exceeds a preset duration based on the target vehicle's location information at multiple time points. The second determining module is used to determine that the target vehicle is an illegally parked vehicle if a preset time period is exceeded.

8. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the vehicle illegal parking detection method according to any one of claims 1 to 6 by executing the executable instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the vehicle illegal parking detection method according to any one of claims 1 to 6.

Images