Vehicle illegal parking detection method, device, electronic device and storage medium

Abstract

The present invention relates to the field of video processing technology, and provides a method, device, electronic device and storage medium for detecting illegal parking of vehicles. The method for detecting illegal parking of vehicles comprises: obtaining multiple video streams to be detected; patrolling the multiple video streams to obtain target images containing vehicle targets of the corresponding video streams in each patrol cycle; when the time span of the target image of a video stream is greater than the illegal parking duration threshold, clustering all target images of the video stream according to the position coordinates of the contained vehicle targets to obtain at least one target image group; performing similarity comparison on each target image group, and when the number of similar target images in the same group is greater than the quantity threshold, determining that there is an illegally parked vehicle target in the corresponding target image group. The present invention saves computing resources while improving the accuracy of illegal parking detection of vehicles through patrol detection, image grouping based on position coordinate clustering and similarity comparison, thereby achieving cost reduction and efficiency improvement.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a vehicle illegal parking detection method, device, electronic equipment and storage medium. Background Art

[0002] Illegal parking can threaten road traffic, urban public management and even the lives and property of citizens. Especially in places with heavy traffic, illegal parking can seriously hinder traffic and affect public safety.

[0003] With the rapid development of science and technology and the gradual improvement of video image processing technology, video surveillance has been widely used in vehicle parking detection. Figure 1 As shown: S101, obtain the vehicle detection result, specifically find all the vehicles in the camera image through the target detection algorithm; then track each vehicle through the target tracking algorithm, calculate the stay time of each vehicle respectively, and judge whether it is illegal parking, specifically including: S102, compare the similarity with the previous vehicle detection result; S103, judge whether the similarity exceeds the corresponding threshold; if so, execute S104, accumulate the stay time of the current vehicle, if not, return to obtain the next vehicle detection result (that is, no longer track the current vehicle); after accumulating the stay time, continue to execute S105, judge whether the stay time exceeds the corresponding threshold, if so, execute S106, alarm, if not, continue to obtain the next vehicle detection result, repeat the above process to track the current vehicle.

[0004] It can be seen that in the process of tracking a vehicle by the target tracking algorithm, as long as the two detection results are not similar, the residence time will no longer be accumulated, resulting in the current vehicle being lost; corresponding to the actual scene, for example, if the vehicle is temporarily blocked by other objects, the target tracking algorithm will lose the current vehicle, and thus recalculate the residence time of the current vehicle, resulting in inaccurate calculation of the residence time; in addition, in order to ensure the detection accuracy of the target tracking algorithm, real-time tracking calculation is required, and one-way algorithm can only support one video screen, with high calculation overhead and high resource consumption.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] In view of this, the present invention provides a vehicle illegal parking detection method, device, electronic device and storage medium, which can solve the problem of high calculation overhead of target tracking algorithm and inaccurate calculation of parking duration.

[0007] One aspect of the present invention provides a method for detecting illegal parking of vehicles, comprising: obtaining multiple video streams to be detected; patrolling the multiple video streams to obtain target images containing vehicle targets of the corresponding video streams in each patrol cycle; when the time span of the target image of a video stream is greater than an illegal parking duration threshold, clustering all target images of the video stream according to the position coordinates of the contained vehicle targets to obtain at least one target image group; performing a similarity comparison on each target image group, and when the number of similar target images in the same group is greater than a quantity threshold, determining that the corresponding target image group contains an illegally parked vehicle target.

[0008] In some embodiments, when the patrol detects multiple video streams, the timestamp of the target image of the corresponding video stream in a patrol cycle and the position coordinates of the vehicle target are taken as an element and stored in the fixed-length queue of the corresponding video stream; the judgment condition when the time span of the target image of a video stream is greater than the illegal parking duration threshold is: the fixed-length queue of the video stream is full.

[0009] In some embodiments, the vehicle illegal parking detection method also includes: after determining that the corresponding target image group contains illegally parked vehicle targets, clearing the full fixed-length queue; and if it is determined that there are no illegally parked vehicle targets in each target image group, as the patrol cycle progresses, when the full fixed-length queue is updated, the step of clustering all target images of the video stream according to the position coordinates of the contained vehicle targets is executed according to the updated full fixed-length queue.

[0010] In some embodiments, the vehicle illegal parking detection method further includes: monitoring a fixed-length queue of each of the video streams; and clearing the fixed-length queue when the time interval for storing elements in the fixed-length queue exceeds a parking illegal parking interval threshold.

[0011] In some embodiments, the fixed capacity LQ of the fixed-length queue satisfies: Wherein, T is the illegal parking duration threshold, W is the expansion coefficient, and P is the patrol period. The round-up character.

[0012] In some embodiments, obtaining a target image containing a vehicle target in a corresponding video stream in each patrol cycle includes: in a current patrol cycle, obtaining a current video segment of a corresponding current video stream; extracting frames from the current video segment, and filtering out a detection result image having a vehicle target in a target screen area through target detection; and generating a target image containing a frame timestamp and the position coordinates of the vehicle target based on the detection result image.

[0013] In some embodiments, clustering all target images of the video stream according to the position coordinates of the vehicle targets contained therein includes: obtaining the position coordinates of the vehicle targets of all target images of the video stream; clustering the obtained position coordinates; grouping all target images of the video stream according to the position coordinate clustering results, and when a target image contains multiple vehicle targets, generating multiple target images each containing the multiple vehicle targets, and respectively dividing the multiple target images into corresponding target image groups.

[0014] In some embodiments, after obtaining at least one target image group, the method further includes: filtering out target image groups in which the number of target images is less than a preset threshold value L, and the preset threshold value L satisfies: Wherein, T is the illegal parking duration threshold, P is the patrol period, The floor character.

[0015] In some embodiments, the similarity comparison for each target image group includes: arranging the target images of each target image group into a time series according to timestamps; performing similarity comparison on the first target image of each time series with subsequent target images of the time series in sequence, and accumulating the number of similar target images whose comparison results exceed a similarity threshold.

[0016] In some embodiments, the quantity threshold satisfies: R=n*E; wherein R is the quantity threshold, n is the number of target images in the target image group currently being compared, and E is the error coefficient.

[0017] In some embodiments, after determining that there is an illegally parked vehicle target in the corresponding target image group, it also includes: based on similar target images of the target image group, outputting an illegal parking detection result including a screenshot and illegal parking duration of the illegally parked vehicle target, wherein the illegal parking duration is calculated based on the timestamp of the similar target image.

[0018] Another aspect of the present invention provides a vehicle illegal parking detection device, comprising: a video reading module, used to obtain multiple video streams to be detected; a patrol detection module, used to patrol the multiple video streams to obtain target images containing vehicle targets in the corresponding video streams in each patrol cycle; an image clustering module, used to cluster all target images of a video stream according to the position coordinates of the contained vehicle targets when the time span of the target image of a video stream is greater than the illegal parking duration threshold, to obtain at least one target image group; an illegal parking discrimination module, used to perform a similarity comparison on each target image group, and when the number of similar target images in the same group is greater than the quantity threshold, determine that the corresponding target image group has an illegally parked vehicle target.

[0019] Another aspect of the present invention provides an electronic device, comprising: a processor; a memory, wherein the memory stores executable instructions; wherein when the executable instructions are executed by the processor, any of the above-mentioned implementations of the vehicle illegal parking detection method is implemented.

[0020] Another aspect of the present invention provides a computer-readable storage medium for storing a program, wherein the program, when executed by a processor, implements any of the above-mentioned methods for detecting illegal parking of vehicles.

[0021] The beneficial effects of the present invention compared with the prior art include at least:

[0022] Through patrol detection, one algorithm can support multiple video streams without the need for real-time tracking and calculation, reducing computing overhead and saving computing resources; by grouping images based on the location coordinates of vehicle targets, moving objects and non-moving objects in the target image can be effectively separated; by performing similarity comparison on the images within the group after clustering, the problem of inaccurate calculation of the dwelling time caused by temporary occlusion of vehicle targets can be effectively solved, thereby improving the accuracy of vehicle illegal parking detection, avoiding missed detections and false alarms, and achieving cost reduction and efficiency improvement.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present invention, and together with the specification are used to explain the principles of the present invention. Obviously, the accompanying drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0025] Figure 1 A schematic diagram showing a flow chart of vehicle illegal parking detection in the prior art;

[0026] Figure 2 A schematic diagram showing the steps of a vehicle parking violation detection method according to an embodiment of the present invention;

[0027] Figure 3 A schematic diagram showing a scene of obtaining a target image of a current video stream in a current patrol cycle in an embodiment of the present invention;

[0028] Figure 4 A schematic diagram showing the steps of a method for detecting illegal parking of a vehicle in another embodiment of the present invention;

[0029] Figure 5 A schematic diagram of a module of a vehicle illegal parking detection device according to an embodiment of the present invention is shown;

[0030] Figure 6 A schematic diagram showing the structure of an electronic device in one embodiment of the present invention is shown;

[0031] Figure 7 A schematic diagram showing the structure of a computer-readable storage medium in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0032] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments set forth herein. Instead, these embodiments are provided to make the present invention comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art.

[0033] The accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0034] In addition, the processes shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps can be decomposed, some steps can be combined or partially combined, and the actual execution order may change according to the actual situation. The words "first", "second" and similar words used in the specific description do not indicate any order, quantity or importance, but are only used to distinguish different components. It should be noted that the embodiments of the present invention and the features in different embodiments can be combined with each other without conflict.

[0035] Figure 2 The main steps of a vehicle parking violation detection method in one embodiment are shown, referring to Figure 2 As shown, the vehicle illegal parking detection method includes: step S210, obtaining multiple video streams to be detected; step S220, patrolling the multiple video streams to obtain the target image containing the vehicle target of the corresponding video stream in each patrol cycle; step S230, when the time span of the target image of a video stream is greater than the illegal parking time threshold, clustering all the target images of the video stream according to the position coordinates of the contained vehicle targets to obtain at least one target image group; step S240, performing a similarity comparison on each target image group, and when the number of similar target images in the same group is greater than the number threshold, it is determined that the corresponding target image group has an illegally parked vehicle target.

[0036] One video stream is captured by one camera, and the camera can upload the captured video stream to the server in real time or at intervals. The server can be a physical server or a cloud server. The multiple video streams to be detected can belong to the same area, such as the video streams captured by cameras at three different locations on the main road of a park, or they can belong to different areas, such as the video streams captured by cameras at five different locations on two different main roads of a park. In addition, illegal parking detection can be performed only on multiple video streams within a certain historical period, or illegal parking monitoring can be performed by continuously reading multiple video streams continuously received from the server without real-time reading.

[0037] Therefore, the above-mentioned vehicle illegal parking detection method can be applied to any place where illegal parking detection is required, without the need for real-time tracking and calculation. Through patrol detection, one algorithm can support multiple video streams, reduce computing overhead, and save computing resources. In particular, in some places with dense traffic, such as main roads in communities and parks, it is possible to effectively separate moving objects and non-moving objects in the target image by grouping pictures based on the location coordinates of vehicle targets, and by performing similarity comparison on the pictures in the group after clustering, it is possible to effectively solve the problem of inaccurate calculation of the length of stay caused by the temporary obstruction of vehicle targets in places with dense traffic. In addition, in some places with high requirements for the installation of IoT devices and high difficulty in actual deployment, the above-mentioned vehicle illegal parking detection method can be used without changing the original camera terminal. It only needs to use the video stream uploaded by the camera to the server to detect vehicle illegal parking, which is highly feasible. Even in some places where network deployment is difficult, the camera terminal can transmit the video stream back to the cloud server through 5G and other communication methods to achieve cloud-network integration.

[0038] Patrol detection of multiple video streams refers to detecting each video stream in turn according to the set patrol cycle. In one embodiment, a fixed-length queue is introduced to store the relevant information of the acquired target image. Specifically, when patrol detection of multiple video streams is performed, the timestamp of the target image of the corresponding video stream in a patrol cycle and the position coordinates of the vehicle target are taken as an element and stored in the fixed-length queue of the corresponding video stream (that is, in a patrol cycle, no matter how many object information is detected, it is only stored as an element in the corresponding fixed-length queue); accordingly, the judgment condition that the time span of the target image of a video stream is greater than the illegal parking duration threshold is: the fixed-length queue of the video stream is full.

[0039] The timestamp of the target image and the position coordinates of the vehicle target therein (hereinafter referred to as spatiotemporal information) can be obtained when the target image is acquired from the video stream. The specific acquisition method will be described below in conjunction with target detection. One video stream corresponds to a fixed-length queue to ensure that when the fixed-length queue is full, the corresponding video stream has acquired the target image with a suitable time span for illegal parking determination.

[0040] In this embodiment, taking five video streams to be detected (the first video stream I1 to the fifth video stream I5) as an example, the patrol detection process includes: in the first patrol cycle, obtaining the target image containing the vehicle target in the first video stream I1, and using the temporal and spatial information of the obtained target image as an element i 11 Stored in the fixed-length queue Q1 of the first video stream I1; in the second patrol cycle, obtain the target image containing the vehicle target in the second video stream I2, and use the temporal and spatial information of the obtained target image as an element i 22 Stored in the fixed-length queue Q2 of the second video stream I2; ... and so on, obtain the target image of the corresponding video stream in each patrol cycle, and store the time and space information in the corresponding fixed-length queue.

[0041] After the first video stream I1 to the fifth video stream I5 have completed a round of patrol detection, the next round of patrol detection is continued, that is, in the sixth patrol cycle P6, the first video stream I1 is returned to obtain the target image containing the vehicle target, and the spatiotemporal information of the obtained target image is used as an element i 16 Stored in its fixed-length queue Q1; ... and so on, as the patrol cycle progresses, the target image of the corresponding video stream is continuously obtained and the spatiotemporal information is stored in the corresponding fixed-length queue. In this process, if there is no vehicle target in the corresponding video stream in a patrol cycle, the target image of the video stream will not be obtained, and no element will be stored in its corresponding fixed-length queue.

[0042] Furthermore, during the patrol detection process, the fixed-length queue of each video stream is also monitored; when the time interval for storing elements in a fixed-length queue exceeds the illegal parking interval threshold, it indicates that there is no vehicle target in the corresponding video stream for a sufficiently long continuous time, and it is even less likely that illegal parking will occur. Therefore, the fixed-length queue is cleared to reduce the amount of calculation for subsequent illegal parking judgments. The illegal parking interval threshold can be set as needed, for example, 5 minutes. In addition, the illegal parking duration threshold is the basis for determining the duration of illegal parking of a vehicle target. If a vehicle stays in a prohibited parking area for longer than the illegal parking duration threshold, it is determined to be illegally parked; the illegal parking duration threshold can also be set as needed, or according to the relevant local traffic regulations, for example, 10 minutes.

[0043] In one embodiment, the fixed capacity LQ of the fixed-length queue specifically satisfies: Wherein, T is the duration value of the illegal parking duration threshold, W is the expansion coefficient, and P is the duration value of the patrol cycle in the same unit as the illegal parking duration threshold (in minutes in this embodiment). is the rounding symbol. For example, the illegal parking duration threshold is 10 minutes (then T=10); the expansion factor is 1.2 (then W=1.2), which is introduced to ensure the accuracy of the algorithm, so as to appropriately lengthen the illegal parking duration threshold, and a suitable value slightly greater than 1 can be taken; the patrol cycle is 30s (i.e. 0.5 minutes, P=0.5); thus, the fixed capacity LQ, that is, the length of the fixed-length queue is 10*1.2 / 0.5=24. In other embodiments, the above values ​​can be set as needed, not limited to those listed here.

[0044] As the patrol cycle progresses, when a fixed-length queue is full, indicating that the video stream corresponding to the fixed-length queue has acquired a sufficient number of target images suitable for illegal parking determination, all elements in the fixed-length queue are read and combined with all target images of the corresponding video stream to make illegal parking determination. The illegal parking determination specifically adopts a method combining position coordinate clustering and image similarity comparison, which will be explained in detail below.

[0045] If multiple fixed-length queues are full at the same time, multi-threaded parallel processing can be used to perform illegal parking judgments on multiple video streams separately.

[0046] In one embodiment, based on the illegal parking determination result of a video stream, if it is determined that a certain or several target image groups of the video stream contain illegally parked vehicle targets, the full fixed-length queue of the video stream is cleared to save memory space and reduce the amount of calculation for the next illegal parking determination of the video stream; subsequently, as the patrol cycle proceeds, the spatiotemporal information of the target images obtained from the video stream continues to be stored in the corresponding patrol cycle.

[0047] In one embodiment, based on the illegal parking determination result of a video stream, if it is determined that there are no illegally parked vehicle targets in each target image group of the video stream, then as the patrol cycle progresses, when the full fixed-length queue of the video stream is updated, a step of clustering all target images of the video stream (all target images are target images corresponding to all elements in the updated full fixed-length queue) according to the position coordinates of the contained vehicle targets is executed according to the updated full fixed-length queue.

[0048] By utilizing the principle that when a fixed-length queue is full, each time a new element is stored, the first old element stored is squeezed out. If the illegally parked vehicle target is not identified in one illegal parking judgment, all elements in the corresponding full fixed-length queue are retained, and illegal parking judgment is continued when the full fixed-length queue is updated next time, so as to ensure the accuracy of illegal parking judgment through time continuity.

[0049] In one embodiment, a target detection algorithm is used to obtain a target image according to a video stream. Specifically, obtaining a target image containing a vehicle target in a corresponding video stream in each patrol cycle includes: obtaining a current video segment of a corresponding current video stream in a current patrol cycle; extracting frames from the current video segment, and filtering out a detection result image containing a vehicle target in a target screen area through target detection; and generating a target image containing a frame timestamp and the position coordinates of the vehicle target according to the detection result image.

[0050] Figure 3 FIG. 1 shows a scene of obtaining a target image of a current video stream in a current patrol cycle in an embodiment, referring to FIG. Figure 3 As shown, the current video segment 310 corresponding to the current patrol cycle of the current video stream is obtained, and the current video segment 310 is subjected to frame extraction detection, that is, the image is extracted and sent to the target detection algorithm to detect whether there is a vehicle target in the target screen area of ​​the image. The target detection algorithm has been trained in advance and can output the detection frame and corner point coordinates of all vehicle targets in its target screen area according to the input image. The specific principle of the target detection algorithm is an existing technology, so it will not be described in detail.

[0051] by Figure 3 As shown in the figure, for example, image 320 is extracted from the current video segment 310 (from the perspective of the actual scene, the main road area of ​​image 320 indicated by a black bold dashed line contains a vehicle indicated by a black bold frame). After target detection, it is identified that the target screen area (i.e., the main road area) of image 320 has a vehicle target (i.e., a vehicle), and the detection result image is output, which is actually the detection frame corresponding to the vehicle target 320' and the corner point coordinates of the detection frame, as shown in the figure, which shows the coordinates of the upper left corner point (x1, y1) and the coordinates of the lower right corner point (x2, y2). Then, based on the detection result image, a target image 330 is generated, which includes the timestamp of the frame corresponding to image 320 and the position coordinates of the vehicle target 320' (the position coordinates (x, y) of the center point of the vehicle target 320' can be calculated based on the corner point coordinates of the detection frame to facilitate subsequent position coordinate clustering).

[0052] Continuing the frame sampling detection, no vehicle target is detected in the target image area of ​​image 350, and no detection result image is generated. Two vehicle targets are detected in the target image area of ​​image 360, and detection frames corresponding to the two vehicle targets 360' and 360" and the corner point coordinates of the detection frames (not specifically marked) are generated, and then the corresponding target image 370 is generated.

[0053] In one embodiment, after obtaining a suitable number of target images of a video stream, all target images of the video stream are clustered according to the position coordinates of the vehicle targets contained therein, including: obtaining the position coordinates of the vehicle targets of all target images of the video stream (in combination with the above-mentioned fixed-length queue embodiment, that is, taking out all elements from the full fixed-length queue of the video stream, and the timestamps and position coordinates in the fixed-length queue correspond one to one, for example, can be stored in a key-value manner); clustering the obtained position coordinates; grouping all target images of the video stream according to the position coordinate clustering results, and when a target image contains multiple vehicle targets, generating multiple target images each containing the multiple vehicle targets, and dividing the multiple target images into corresponding target image groups.

[0054] For example, combining Figure 3 As shown, the target image 370 includes two vehicle targets 360' and 360", and two target images are generated based on the target image 370 (for example, the image area corresponding to the vehicle target 360' / 360" can be cut out from the target image 370), so that one of the target images includes the vehicle target 360' and the other target image includes the vehicle target 360", and the two generated target images are divided into corresponding target image groups according to the position coordinate clustering results.

[0055] In addition, when grouping the target images, a capture operation may be performed on all the target images, that is, the screen area corresponding to the vehicle target is captured, and the captured screen images are grouped to facilitate subsequent image similarity comparison.

[0056] Alternatively, in one embodiment, when generating the target image according to the detection result image, the picture area corresponding to the vehicle target can be cut out from the original image as the target image.

[0057] Furthermore, in one embodiment, after obtaining at least one target image group by grouping images clustered based on the position coordinates of the vehicle target, the method further includes: filtering out the target image group whose number of target images is less than a preset threshold value L, and the preset threshold value L specifically satisfies: The method for obtaining the illegal parking duration threshold T and the patrol period P can refer to the above-mentioned embodiment. The floor character.

[0058] If the number of images in the target image group is insufficient, it indicates that there is a high probability that there are no long-term static objects in the group, that is, there are no illegally parked vehicles. Therefore, by screening the target image groups after clustering and grouping, the target image groups with insufficient number of images in the group are filtered out, the calculation amount of subsequent similarity comparison is reduced, and the illegal parking identification process is accelerated.

[0059] For the retained target image groups, a similarity comparison of images within the group is performed, specifically including: arranging the target images of each target image group into a time series according to timestamps; performing a similarity comparison on the first target image of each time series with the subsequent target images of the time series in turn, and accumulating the number of similar target images whose comparison results exceed the similarity threshold.

[0060] For example, a target image group contains 10 target images (only as an example, not a limitation on the number of target images in the group). After being arranged into a time series according to timestamps, the first target image is sequentially compared with the second, third, ... tenth target images for vehicle target similarity. If the comparison result exceeds the similarity threshold, the number of similar target images is accumulated. Finally, the number of similar target images is compared with the quantity threshold to determine whether there are illegally parked vehicle targets in the target image group.

[0061] By comparing the similarity of images within the group, when moving objects and stationary objects are grouped together (corresponding to actual scenarios, such as when an illegally parked vehicle is temporarily blocked by a passing vehicle), the impact caused by the temporary blocking of moving objects can be ignored, and the illegally parked vehicle targets can be accurately identified. If the number of similar target images within the group exceeds the quantity threshold R, it indicates that there are illegally parked vehicles in the group. The quantity threshold R specifically satisfies: R = n*E; where n is the number of target images in the target image group currently being compared, and E is the error coefficient. The value range of E is [0,1]. The larger the value, the more accurate the recognition result.

[0062] The image similarity comparison specifically uses an existing method to compare the similarity of given images and return a similarity value, which will not be further described here.

[0063] Furthermore, in one embodiment, after determining that there is an illegally parked vehicle target in the corresponding target image group, the method further includes: outputting an illegal parking detection result including a screenshot and illegal parking duration of the illegally parked vehicle target according to similar target images in the target image group, wherein the illegal parking duration is calculated according to the timestamp of the similar target image. Thus, on the basis of determining that there is an illegally parked vehicle, the illegal parking duration of the vehicle is further obtained.

[0064] Figure 4 The steps of a method for detecting illegal parking of a vehicle in a preferred embodiment are shown in FIG. Figure 4 As shown, in combination with the screen patrol, target detection, spatial position clustering and image similarity comparison described in the above embodiment, the process of detecting illegal parking of vehicles in this embodiment specifically includes:

[0065] The first step S410, screen patrol: pre-read the screen configuration of all video streams that need to detect illegal parking (for example, in this embodiment, there are a total of I camera video streams that need to be detected), and then read a video stream according to the patrol cycle and video stream order.

[0066] The second step S420, target detection: perform frame detection on the video stream, generate a target image based on the target detection result, and store the corresponding spatiotemporal information in the fixed-length queue of the video stream; when the reading operation of a video stream is completed, store the spatiotemporal information in its detection result into the corresponding fixed-length queue, and continue with the reading operation of the next video stream.

[0067] The third step S430: determine whether there is a fixed-length queue that is full; if not, continue to read the next video stream as the patrol cycle progresses; if so, read all elements in the full fixed-length queue, perform subsequent illegal parking judgments, and continue the patrol reading of each video stream at the same time. Before the next reading operation of the video stream corresponding to the full fixed-length queue, the illegal parking judgment of the video stream can usually be completed. If it cannot be completed, when the screen patrol reaches the next reading operation of the video stream, it can be paused to wait for the illegal parking judgment to end, and then continue the reading operation of the video stream.

[0068] Figure 4 In the figure, the dotted arrows indicate the continuation of the screen tour.

[0069] The fourth step S440 is spatial position clustering: combining the elements in the full fixed-length queue and the corresponding target images, the position coordinates of the vehicle target are clustered first, and then the target images are grouped according to the position coordinate clustering results to obtain target image groups corresponding to different picture areas of the video stream, which can effectively separate the static objects and moving objects in the shooting picture of the video stream; after clustering and grouping, the number of target images in each group is counted. If the preset threshold is not reached, it indicates that there is a high probability that there is no long-term static object in the group, and the target image group whose number of target images in the group exceeds the preset threshold is retained.

[0070] The fifth step S450, image similarity comparison: perform similarity comparison on the target images in the group, and determine whether the number of similar target images in the group exceeds the quantity threshold. If it indicates that there are illegally parked vehicle targets in the group, an alarm is output. In this way, when a moving object may be grouped with a stationary object, the impact caused by the temporary occlusion of the moving object is ignored, and the illegally parked vehicle is accurately identified.

[0071] It should be noted that Figure 4 Only a brief process of screen patrol, target detection, spatial position clustering and image similarity comparison during vehicle illegal parking detection is shown. The specific methods and principles can be referred to the description of the above embodiments and will not be repeated here.

[0072] In summary, compared with the target tracking algorithm used in the prior art, which requires real-time calculation, one-way algorithm can only support one-way video screen, and may cause vehicle target loss due to occlusion and inaccurate calculation of residence time, the present invention adopts a vehicle illegal parking detection method that combines screen patrol, target detection, spatial position clustering and image similarity comparison, which has the following advantages and effects:

[0073] Adopting the method based on spatial position coordinate clustering and image similarity comparison instead of tracking can better separate the stationary objects and moving objects in the video screen, effectively solve the problem of inaccurate calculation of the dwelling time caused by the occlusion of the vehicle target, improve the detection accuracy, reduce the false alarm rate and the missed detection rate, and can be applied to any traffic-intensive places. The detection accuracy specifically refers to the proportion of the results with occlusion in the detected illegal parking results; at the same time, because no tracking algorithm is used, real-time detection is not required, the computational overhead is low, and the screen patrol can realize the patrol detection of multiple video streams, which plays the role of supporting one algorithm for multiple video streams, reducing costs and increasing efficiency; in addition, the vehicle illegal parking detection method of the present invention is highly robust, can be applied to various scenarios, is less affected by weather factors, and realizes accurate illegal parking detection.

[0074] The embodiment of the present invention further provides a vehicle illegal parking detection device, which can be used to implement the vehicle illegal parking detection method described in any of the above embodiments. The features and principles of the vehicle illegal parking detection method described in any of the above embodiments can be applied to the following vehicle illegal parking detection device embodiments. In the following vehicle illegal parking detection device embodiments, the features and principles of vehicle illegal parking detection that have been explained will not be repeated.

[0075] Figure 5 The main modules of a vehicle parking violation detection device in one embodiment are shown, referring to Figure 5 As shown, the vehicle illegal parking detection device 500 includes: a video reading module 510, which is used to obtain multiple video streams to be detected; a patrol detection module 520, which is used to patrol and detect multiple video streams to obtain target images containing vehicle targets in the corresponding video stream in each patrol cycle; an image clustering module 530, which is used to cluster all target images of a video stream according to the position coordinates of the contained vehicle targets when the time span of the target image of the video stream is greater than the illegal parking duration threshold, so as to obtain at least one target image group; an illegal parking discrimination module 540, which is used to perform a similarity comparison on each target image group, and when the number of similar target images in the same group is greater than the quantity threshold, it is determined that the corresponding target image group has an illegally parked vehicle target.

[0076] Furthermore, the vehicle illegal parking detection device 500 may also include modules for implementing other process steps of the above-mentioned vehicle illegal parking detection method embodiments, for example, including modules for implementing Figure 4The modules of each step process of the embodiment of the vehicle illegal parking detection method shown in the figure, and the specific principles of each module can be referred to the description of the above-mentioned embodiments of the vehicle illegal parking detection method, which will not be repeated here.

[0077] As described above, the vehicle illegal parking detection device of the present invention can realize one-way algorithm supporting multiple video streams through patrol detection, without the need for real-time tracking and calculation, thereby reducing computing overhead; by grouping pictures based on the location coordinates of vehicle targets, it can effectively separate moving objects and non-moving objects in the target image; by performing similarity comparison on the pictures within the group after clustering, it can effectively solve the problem of inaccurate calculation of the stay time caused by the temporary occlusion of the vehicle target, thereby improving the accuracy of vehicle illegal parking detection and achieving cost reduction and efficiency improvement.

[0078] An embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein executable instructions are stored in the memory, and when the executable instructions are executed by the processor, the vehicle illegal parking detection method described in any of the above embodiments is implemented.

[0079] As described above, the electronic device of the present invention can realize one-way algorithm supporting multiple video streams through patrol detection, without the need for real-time tracking and calculation, thereby reducing computing overhead; by grouping pictures based on the location coordinates of vehicle targets, it can effectively separate moving objects and non-moving objects in the target image; by performing similarity comparison on the pictures within the group after clustering, it can effectively solve the problem of inaccurate calculation of the stay time caused by the temporary occlusion of the vehicle target, thereby improving the accuracy of vehicle illegal parking detection and achieving cost reduction and efficiency improvement.

[0080] Figure 6 is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. It should be understood that: Figure 6 The modules are only schematically shown, and these modules may be virtual software modules or actual hardware modules. The combination and splitting of these modules and the addition of other modules are all within the protection scope of the present invention.

[0081] like Figure 6 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0082] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps of the vehicle parking violation detection method described in any of the above embodiments. For example, the processing unit 610 can execute the following steps: Figure 2 and Figure 4 Steps shown.

[0083] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0084] The storage unit 620 may also include a program / utility 6204 having one or more program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0085] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0086] The electronic device 600 can also communicate with one or more external devices 700, which can be one or more of a keyboard, a pointing device, a Bluetooth device, etc. These external devices 700 enable the user to communicate interactively with the electronic device 600. The electronic device 600 can also communicate with one or more other computing devices, and the computer device shown includes a router and a modem. This communication can be performed through an input / output (I / O) interface 650. In addition, the electronic device 600 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through a network adapter 660. The network adapter 660 can communicate with other modules of the electronic device 600 through a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0087] The embodiment of the present invention further provides a computer-readable storage medium for storing a program, which, when executed, implements the vehicle parking violation detection method described in any of the above embodiments. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the vehicle parking violation detection method described in any of the above embodiments.

[0088] As described above, the computer-readable storage medium of the present invention can realize one-way algorithm supporting multiple video streams through patrol detection, without the need for real-time tracking and calculation, thereby reducing computing overhead; by grouping pictures based on the location coordinates of vehicle targets, the moving objects and non-moving objects in the target image can be effectively separated; by performing similarity comparison on the pictures within the group after clustering, the problem of inaccurate calculation of the dwell time caused by the temporary occlusion of the vehicle target can be effectively solved, thereby improving the accuracy of vehicle illegal parking detection and achieving cost reduction and efficiency improvement.

[0089] Figure 7 Schematic diagram of the structure of the computer-readable storage medium of the present invention. Figure 7 As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0090] The program product may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media include, but are not limited to: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0091] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0092] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device, such as through the Internet using an Internet service provider.

[0093] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A vehicle illegal parking detection method, characterized in that: include: Obtain multiple video streams to be detected; patrolling to detect the multiple video streams, and obtaining a target image containing a vehicle target in the corresponding video stream in each patrol cycle; When the time span of a target image of a video stream is greater than a parking violation duration threshold, clustering all target images of the video stream according to the position coordinates of the vehicle targets contained therein to obtain at least one target image group; A similarity comparison is performed on each target image group. When the number of similar target images in the same group is greater than a quantity threshold, it is determined that the corresponding target image group contains illegally parked vehicle targets.

2. The vehicle illegal parking detection method according to claim 1, characterized in that: When the patrol detects multiple video streams, the timestamp of the target image of the corresponding video stream in one patrol cycle and the position coordinates of the vehicle target are stored as an element in the fixed-length queue of the corresponding video stream; The determination condition when the time span of the target image of a video stream is greater than the illegal parking duration threshold is: the fixed-length queue of the video stream is full.

3. The vehicle illegal parking detection method according to claim 2, characterized in that: Also includes: After determining that there is an illegally parked vehicle target in the corresponding target image group, clearing the full fixed-length queue; as well as If it is determined that there are no illegally parked vehicle targets in each target image group, as the patrol cycle progresses, when the full fixed-length queue is updated, the step of clustering all target images in the video stream according to the position coordinates of the contained vehicle targets is executed based on the updated full fixed-length queue.

4. The vehicle illegal parking detection method according to claim 2, characterized in that: Also includes: Monitoring the fixed-length queue of each video stream; When the time interval for storing elements in a fixed-length queue exceeds a parking violation interval threshold, the fixed-length queue is cleared.

5. The vehicle illegal parking detection method according to claim 2, characterized in that: The fixed capacity LQ of the fixed-length queue satisfies: Wherein, T is the illegal parking duration threshold, W is the expansion coefficient, and P is the patrol period. The round-up character.

6. The vehicle illegal parking detection method according to claim 1, characterized in that: The step of obtaining a target image containing a vehicle target in a corresponding video stream in each patrol cycle includes: In a current patrol cycle, obtaining a current video segment of a corresponding current video stream; Extract frames from the current video segment, and select detection result images having vehicle targets in the target picture area through target detection; A target image including a frame timestamp and position coordinates of a vehicle target is generated according to the detection result image.

7. The vehicle illegal parking detection method according to claim 1, characterized in that: The clustering of all target images of the video stream according to the position coordinates of the vehicle targets contained therein comprises: Obtaining the position coordinates of the vehicle target of all target images of the video stream; Clustering the obtained location coordinates; According to the position coordinate clustering results, all target images of the video stream are grouped. When a target image contains multiple vehicle targets, multiple target images each containing the multiple vehicle targets are generated, and the multiple target images are respectively divided into corresponding target image groups.

8. The vehicle illegal parking detection method according to claim 1, characterized in that: After obtaining at least one target image group, the method further includes: Eliminate the target image group whose number of target images is less than a preset threshold L, and the preset threshold L satisfies: Wherein, T is the illegal parking duration threshold, P is the patrol period, The floor character.

9. The vehicle illegal parking detection method according to claim 1, characterized in that: The similarity comparison for each target image group includes: Arranging the target images of each target image group into a time sequence according to timestamps; The first target image of each time series is sequentially compared with subsequent target images of the time series for similarity, and the number of similar target images whose comparison results exceed the similarity threshold is accumulated.

10. The vehicle illegal parking detection method according to claim 1, characterized in that: The quantity threshold satisfies: R=n*E; Wherein, R is the quantity threshold, n is the number of target images in the target image group currently being compared, and E is the error coefficient.

11. The vehicle illegal parking detection method according to claim 1, characterized in that: After determining that the corresponding target image group contains illegally parked vehicle targets, the method further includes: According to the similar target images of the target image group, the illegal parking detection result including the screen screenshot and the illegal parking duration of the illegally parked vehicle target is output, and the illegal parking duration is calculated based on the timestamp of the similar target images.

12. A vehicle illegal parking detection device, characterized in that: include: A video reading module, used to obtain multiple video streams to be detected; A patrol detection module, used for patrol detection of multiple video streams, to obtain a target image containing a vehicle target in the corresponding video stream in each patrol cycle; An image clustering module, configured to cluster all target images of a video stream according to the position coordinates of the vehicle targets contained therein to obtain at least one target image group when the time span of the target image of a video stream is greater than a parking violation duration threshold; The illegal parking identification module is used to perform similarity comparison on each target image group. When the number of similar target images in the same group is greater than a quantity threshold, it is determined that there is an illegally parked vehicle target in the corresponding target image group.

13. An electronic device, characterized in that: include: A processor; a memory storing executable instructions; Wherein, when the executable instruction is executed by the processor, the vehicle illegal parking detection method as described in any one of claims 1-11 is implemented.

14. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by the processor, the vehicle illegal parking detection method as described in any one of claims 1-11 is implemented.

Images