Multi-terminal fusion non-continuous vehicle illegal parking detection method and system

Abstract

The application discloses a multi-terminal discontinuous vehicle illegal parking detection method and system. The method comprises the following steps: acquiring pictures of the same area taken by terminals mounted on different devices at different time periods and on different routes to obtain multiple discontinuous pictures; performing space-time clustering and correction, positioning the vehicle by using a static landmark extraction technology, determining the vehicle area, and constructing a position fingerprint for cross-period comparison to determine whether the vehicle is in a stationary state; wherein, multi-view information fusion is performed on the basis of the corrected discontinuous pictures to fuse complete scene description information; when the vehicle is in a stationary state, the parking area and the environmental characteristics are determined; it is judged whether the vehicle has an illegal parking behavior; if yes, the illegal parking type of the vehicle is subdivided; and a structured illegal parking report is created. By implementing the method of the application, comprehensive and efficient monitoring of different types of illegal parking behaviors can be realized, and the accuracy and reliability of the monitoring data are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to a vehicle parking violation detection method, more particularly to a multi-terminal non-continuous vehicle parking violation detection method and system. BACKGROUND

[0002] Currently, the monitoring of vehicle parking violations mainly relies on high-altitude cameras for shooting. However, this method has certain limitations. Since the camera installation position is fixed, its shooting angle and coverage range are limited, resulting in a blind spot in the field of view, which causes missed shots and misshots. Specifically, in the case of target obstruction, it is difficult to accurately determine whether the vehicle is parked in violation of the rules, or the vehicle is incorrectly identified as a violation due to slow driving speed. In addition, data analysis based on static cameras requires high algorithm requirements, but due to limited data volume, it is difficult to accurately assess the degree of vehicle parking violation, and it is also difficult to determine whether the parked vehicle directly caused road congestion. Therefore, a more efficient and accurate method is needed to determine whether a vehicle is parked in violation of the rules and to determine the degree of violation in order to update the parking violation information to the traffic management department for subsequent processing.

[0003] Existing solutions mainly include the use of multiple high-altitude cameras to expand the monitoring range and improve the recognition accuracy, but this significantly increases the system cost. Another attempt is to use mobile monitoring terminals by installing cameras on the top of vehicles to monitor parking violations in a specific area. Although this approach can reduce some costs, it is limited in terms of monitoring quantity, efficiency and range, and can only monitor specific angles, so it cannot fully solve the problem.

[0004] Therefore, it is necessary to design a new method to overcome the shortcomings of the prior art and achieve comprehensive and efficient monitoring of different types of parking violations while ensuring the accuracy and reliability of the monitoring data. SUMMARY

[0005] The present application aims to overcome the shortcomings of the prior art and provide a multi-terminal non-continuous vehicle parking violation detection method and system.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solution: a multi-terminal non-continuous vehicle parking violation detection method, comprising:

[0007] Obtaining pictures of the same area taken by terminals mounted on different devices at different time periods and on different routes to obtain multiple non-continuous pictures;

[0008] The multiple non-continuous pictures are spatio-temporally clustered and corrected, a static landmark extraction technique is used to locate the vehicle, a vehicle area is determined, and a location fingerprint is constructed for cross-period comparison to determine whether the vehicle is in a static state; wherein multi-view information fusion is performed based on the corrected non-continuous pictures to fuse the multi-terminal and multi-angle non-continuous pictures into complete scene description information;

[0009] When the vehicle is in a static state, a parking space area is divided and identified based on historical parking line information or a current image recognition method, and road panoramic information is identified and recorded to obtain environmental characteristics;

[0010] Based on the environmental characteristics, the parking space area, and the vehicle area, whether the vehicle has a violation parking behavior is determined according to a no-parking area rule, a parking space relationship, and a period state;

[0011] When the vehicle has a violation parking behavior, a violation parking type of the vehicle is subdivided;

[0012] A structured violation parking report is created based on the violation parking type, wherein the violation parking report includes vehicle information, a violation parking type, a time and location, a confidence score, and an evidence chain.

[0013] A further technical solution is that the spatio-temporal clustering and correction of the multiple non-continuous pictures, the use of a static landmark extraction technique to locate the vehicle, the determination of a vehicle area, and the construction of a location fingerprint for cross-period comparison to determine whether the vehicle is in a static state, includes:

[0014] Based on a dynamic digital map, non-continuous pictures corresponding to terminals of different views are fused to locate the vehicle;

[0015] The dynamic digital map is an interactive digital sand table established for the area, including road geometry, lane lines, numbered parking spaces, and static facilities, for obtaining semantic information by analyzing non-continuous pictures corresponding to terminals of different views, and updating the situation on the interactive digital sand table in real time with standardized markers.

[0016] A further technical solution is that the device includes different types of vehicles, including taxis, buses, vans, and delivery vehicles; and the installation position and installation angle of the terminal are not limited.

[0017] A further technical solution is that the spatio-temporal clustering and correction of the multiple non-continuous pictures, the use of a static landmark extraction technique to locate the vehicle, the determination of a vehicle area, and the construction of a location fingerprint for cross-period comparison to determine whether the vehicle is in a static state, includes:

[0018] grouping pictures in the plurality of non-continuous pictures that have a distance within a predetermined range based on GPS and timestamp information to obtain a monitoring area;

[0019] converting the non-continuous pictures corresponding to the monitoring area into distortion-free overhead views by a camera calibration matrix to obtain corrected non-continuous pictures;

[0020] performing vehicle positioning on the corrected non-continuous pictures using static landmarks to obtain a vehicle area;

[0021] calculating a relative position vector of a target vehicle with respect to the static landmarks within a predetermined range for pictures at different time points in the corrected non-continuous pictures to obtain vehicle position fingerprints at different time points;

[0022] determining whether the vehicle is in a stationary state based on the vehicle position fingerprints at different time points.

[0023] Further technical solutions of the method are as follows:

[0024] identifying and extracting static landmarks that are fixed and invariant on the corrected non-continuous pictures using a feature point detection algorithm, wherein the static landmarks include long-term invariant features, including parking line corner points and manhole covers;

[0025] determining a position of the vehicle in the corrected non-continuous pictures based on license plate recognition or vehicle feature points to obtain a vehicle area, wherein the vehicle area includes position information represented by a vehicle geometric center point or a license plate center point.

[0026] Further technical solutions of the method are as follows:

[0027] when a difference between the vehicle position fingerprints at different time points is within a set tolerance range and a time interval exceeds a preset minimum judgment interval, determining that the vehicle is in a stationary state.

[0028] Further technical solutions of the method are as follows:

[0029] judging whether the vehicle has a parking violation behavior based on the environment features, the parking area, and the vehicle area according to a no-parking area rule, a parking space relationship, and a time period state.

[0030] Further technical solutions are as follows: the no-parking area rule comprises checking whether the vehicle is located in a special channel or a specific place where parking is absolutely prohibited to determine a corresponding illegal parking type.

[0031] Further technical solutions are as follows: the parking space relationship comprises analyzing the relationship between the vehicle area and the parking space area, and determining whether the vehicle has a boundary-crossing illegal parking behavior according to the vehicle area proportion; and the time period state comprises marking a long-term parked vehicle in combination with a time factor.

[0032] The application further provides a fusion multi-terminal discontinuous vehicle illegal parking detection system, comprising:

[0033] An acquisition unit is configured to acquire pictures of the same area taken by terminals mounted on different devices at different time periods and on different routes, to obtain a plurality of discontinuous pictures;

[0034] A stillness determination unit is configured to perform space-time clustering and correction on the plurality of discontinuous pictures, to locate a vehicle by using a static landmark extraction technology, to determine a vehicle area, and to construct a location fingerprint for cross-time period comparison, so as to determine whether the vehicle is in a still state; wherein, multi-view information fusion is performed on the basis of the corrected discontinuous pictures, to fuse the multi-terminal and multi-angle discontinuous pictures into complete scene description information;

[0035] A parking space and environment identification unit is configured to, when the vehicle is in a still state, divide and identify a parking space area based on historical parking line information or a current image recognition method, and to identify and record road panoramic information, to obtain environmental features;

[0036] An illegal parking behavior judgment unit is configured to, based on the environmental features, the parking space area and the vehicle area, determine whether the vehicle has an illegal parking behavior according to a no-parking area rule, a parking space relationship and a time period state;

[0037] A type determination unit is configured to, when the vehicle has an illegal parking behavior, subdivide an illegal parking type of the vehicle;

[0038] A report generation unit is configured to create a structured illegal parking report based on the illegal parking type, wherein the illegal parking report comprises vehicle information, an illegal parking type, a time and a place, a confidence score and an evidence chain.

[0039] Compared with the prior art, the present application has the beneficial effects that: the present application collects non-continuous pictures taken by different devices at different times and routes, and uses the spatio-temporal clustering and correction technology combined with static landmark extraction to accurately locate the vehicle position, constructs the position fingerprint for cross-period comparison to determine whether the vehicle is stationary; then, the multi-view information is fused to form a complete scene description, the parking space area is divided and the environmental features are recorded based on the parking line and image recognition technology, the illegal parking behavior is judged according to the no-parking rule, parking space relationship and period state, the illegal parking types are subdivided, and the structured illegal parking report is generated. This way overcomes the problems in the prior art that a single data source cannot provide comprehensive scene information and it is difficult to accurately determine the vehicle state, realizes comprehensive and efficient monitoring of different types of illegal parking behaviors, and ensures the accuracy and reliability of the monitoring data.

[0040] The present application will be further described below in conjunction with the drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0042] Figure 1 The flowchart of the multi-terminal non-continuous vehicle illegal parking detection method provided by the embodiment of the present application is shown.

[0043] Figure 2 The shooting schematic diagram of the terminal provided by the embodiment of the present application is shown.

[0044] Figure 3 The schematic diagram of the illegal parking detection example provided by the embodiment of the present application is shown.

[0045] Figure 4 The schematic block diagram of the multi-terminal non-continuous vehicle illegal parking detection system provided by the embodiment of the present application is shown.

[0046] Figure 5 The schematic block diagram of the computer device provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0048] It should be understood that the terms "comprises" and "comprising," when used in this specification and the following claims, indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0049] It should also be understood that the terms used in the specification and the appended claims are intended to describe particular embodiments and do not intend to limit the present application. As used in the specification and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0050] It should further be understood that the term "and / or" used in the specification and the appended claims means one or more of the associated listed items as well as all possible combinations of the items and includes the combinations.

[0051] Please refer to Figure 1 , Figure 1 The schematic flow chart of the fusion multi-terminal non-continuity vehicle illegal parking detection method provided by the embodiment of the present application. The fusion multi-terminal non-continuity vehicle illegal parking detection method is applied to a server, and by integrating pictures of the same area taken by different devices at different times and places, and using time-space clustering and correction technology, static landmark extraction technology for vehicle positioning and state judgment, constructing position fingerprint to realize cross-period comparison, ensuring accurate identification of vehicle static state. Further combined with dynamic digital map, historical parking line information and image recognition technology to divide parking area and record road panoramic information, according to the existence and type of illegal parking behavior, the structured illegal parking report containing detailed evidence is finally generated. This method overcomes the problem of incomplete monitoring data and low accuracy in the prior art, realizes comprehensive, efficient and accurate monitoring of different types of illegal parking behavior, and improves the reliability and adaptability of the illegal parking detection system.

[0052] Figure 1 The flow chart of the fusion multi-terminal non-continuity vehicle illegal parking detection method provided by the embodiment of the present application. As shown in Figure 1 The method comprises the following steps S110 to S160.

[0053] S110, acquiring pictures of the same area taken by terminals mounted on different devices at different time periods and on different routes to obtain a plurality of non-continuity pictures.

[0054] In this embodiment, the multiple non-continuous pictures refer to photos of the same area taken by terminal devices installed on different types of vehicles (such as taxis, buses, coaches, delivery vehicles, etc.) at different time periods and on different routes. These photos form a non-continuous data set due to the differences in time, location, and angle of shooting.

[0055] The device includes different types of vehicles, including taxis, buses, coaches, delivery vehicles; the installation position and installation angle of the terminal are not limited.

[0056] Specifically, the terminal device can be installed on any movable vehicle without specific angle or position restrictions, which enables image information of an area to be obtained from multiple perspectives. For example, a taxi may pass through and take pictures of a street at a certain time in the morning, while a bus may pass through the same street for shooting at another time in the afternoon. Unlike traditional continuous video stream monitoring, this detection method relies on scattered, non-real-time static pictures. This means that even the same vehicle will only be captured when its path randomly passes through the target area. Therefore, even the monitoring of the same vehicle is non-continuous, and algorithms are needed to associate the pictures taken at different times.

[0057] Although pictures from different terminals may overlap, the system will perform preliminary screening and deduplication on these pictures based on GPS positioning information, time stamps, and picture content similarity. This can not only ensure the effectiveness and diversity of the data, but also reduce redundant data and reduce server burden.

[0058] Because these terminal devices can be installed on a variety of different vehicles and are not limited to a specific installation angle, they can provide a rich and diverse data source. Such flexibility provides a solid foundation for subsequent image analysis and vehicle parking violation judgment. For example, high-angle pictures can help identify parking lines and the overall layout of the vehicle, while low-angle pictures can help clearly read license plate numbers.

[0059] The non-continuous pictures obtained will be clustered and corrected according to their attached spatio-temporal information (i.e. GPS coordinates and time stamps) to facilitate comparison and analysis in the same coordinate system. This process is crucial for accurately determining whether a vehicle is in a stationary state, determining the specific location of the vehicle, and its relationship with the surrounding environment.

[0060] Specifically, as shown in Figure 2 The system design allows terminal devices to be installed on various types of vehicles, such as taxis, buses, coaches, and even delivery vehicles, etc. These terminals can adapt to multiple installation angles and can be quickly installed through front windshield suction, ensuring the flexibility and wide coverage of data collection.

[0061] Each terminal has edge computing capabilities, which can perform preliminary screening on the captured pictures locally. This includes blur detection, brightness / exposure assessment, low-resolution filtering, noise detection, and motion blur recognition. Through these algorithms, low-quality or irrelevant pictures can be effectively removed, reducing the amount of data uploaded to the server.

[0062] Although the angles of the pictures taken by different terminals are different, in order to prevent information redundancy, a similarity-based deduplication algorithm is used. Specifically, when the similarity of two pictures exceeds 95% (considering shooting coordinates, content similarity and information redundancy), they are considered highly similar and are processed for deduplication. In addition, secondary screening and deduplication are also performed on the server side to further optimize the quality of the data set. Considering the difference in angles, the deduplication here is mainly based on shooting coordinates, picture content similarity, and information redundancy.

[0063] Since the terminal devices can be installed on vehicles of different types and heights, they can provide diverse perspectives and data sources. For example, pictures taken at high angles can help identify parking lines and overall layout, while pictures taken at low angles are more conducive to clearly reading license plate numbers.

[0064] Even if the pictures are taken at the same location, the information obtained will differ due to factors such as shooting time, weather conditions, etc. For example, a red car may not have pedestrian interference when photographed in the morning, but there may be pedestrians in the picture when photographed in the afternoon. That is, pictures taken at the same location at different times may differ due to factors such as personnel movement, which can assist in determining the status of the vehicle. This diversity provides rich background information and support for subsequent algorithm recognition. The pictures taken are data for machine learning. The data collected by different terminals at the same location, the time of shooting, and the angle of shooting are different. The light and shadow in the pictures taken at different times will also be different, including weather or other items entering.

[0065] Unlike the traditional continuous video stream monitoring method, the non-continuity of the system is reflected in the analysis of dispersed static pictures. Specifically, by different terminals shooting target vehicle pictures at different times, the picture collection of the target vehicle at different time periods and different angles is realized, and these static pictures can be analyzed to determine the vehicle static or dynamic state, etc. For example, for a red car on the roadside, different types of terminals (such as taxis and buses) will pass through and take pictures of this area at different time periods, rather than continuously recording. This way makes it possible to obtain the parking state of the same car at different times and angles, and further realize more accurate illegal parking judgment through algorithm means. Non-continuity is to collect data. For example, a red car on the roadside. Normal manual inspection or vehicle video inspection is to shoot along the road, back and forth, which is a continuous process. The non-continuity of the system is reflected in that at this location, different terminals such as taxis or buses that pass through the red car may take pictures in the morning and in the afternoon, and upload the pictures, rather than recording and analyzing in the form of video.

[0066] For the multi-terminal non-continuity situation, after completing the existing illegal parking detection steps, the optimized detection standard can be developed according to the difference of the pictures collected by different terminals, and the detection result can be optimized to make it more in line with the actual road conditions.

[0067] The received multi-terminal pictures will first be spatio-temporally clustered according to the GPS positioning information and time stamp, and then be uniformly converted into standard overhead views for comparison and analysis. Next, the system will extract structured semantic information from each picture, such as vehicle outline, license plate number, parking space marking line, and static landmark position, etc., and superimpose it on the dynamic digital map.

[0068] For the multi-terminal non-continuity feature, a multi-view information fusion and scene enhancement mechanism is introduced. This mechanism does not simply remove redundant pictures, but makes full use of the unique perspectives provided by each terminal to eliminate uncertainty. For example, when a terminal takes a picture with a very clear license plate but the parking space line is blocked, another terminal may capture the complete parking space line but cannot identify the license plate. By combining the information from these two perspectives, a more complete and accurate scene description can be obtained, achieving the effect of 1+1>2. Simple static analysis of pictures may not be accurate, such as a car being parked and being photographed, which may be a parking space line. Through a picture, it is not known whether the car is moving or stationary. But through multiple pictures, it can be known that, for example, the car is still pressing the line at 10:00 and 10:10, it is considered that the car is stationary and pressing the line. Of course, in the actual processing, all pictures of the red car will be spatio-temporally clustered and analyzed. In this way, the originally static single picture can form a complete one under different times and angles.

[0069] That is, for pictures taken at different time points in the same monitoring area, the server runs a powerful deep learning model for secondary vehicle recognition, license plate recognition, and environmental feature point extraction, assigns different weights to different information types, and uses them for subsequent vehicle overlay analysis to achieve cross-terminal vehicle tracking. Each terminal vehicle can capture road pictures in real time, and after edge computing, upload pictures containing vehicles to the server. Unlike traditional picture collection, this terminal picture collection has three advantages: first, the terminal has strong adaptability to vehicles and is not limited by the type of vehicle and the installation angle. It can be installed on various vehicles at any angle; second, the collection frequency is high, 3 pictures per second (frequency adjustable, maximum 1s30 pictures, or 1 picture per 2 meters according to driving distance), and the terminal has edge computing capabilities, which can perform initial screening and deduplication on pictures, uploading only pictures containing vehicles to save upload traffic and reduce server pressure; third, the terminal has strong installation adaptability and can be widely deployed on different terminal vehicles to provide data collected by different terminals and data collected by different terminals at the same location for algorithm recognition stage, making it easier to understand vehicle parking status at different time periods. After receiving pictures taken by multiple terminals in the same area at different angles, the platform does not perform traditional deduplication, but performs multi-view information fusion and scene enhancement, uses differences to eliminate uncertainty, and fuses fragmented information from multiple terminals and multiple angles into complete and accurate scene descriptions, achieving a 1+1>2 effect. For example, terminal A (low-angle taxi) captures a clear license plate but the parking line is blocked, and terminal B (high-angle bus) cannot capture the license plate but can see the parking line completely. The system associates and binds the high-confidence information from the two terminals on the dynamic digital map to obtain vehicle information with exact license plate number and exact location, solving the problem of incomplete information from a single terminal.

[0070] In summary, this method not only emphasizes the uniqueness of data collection, but also highlights its advantages in improving the accuracy, coverage, and efficiency of illegal parking detection. By reasonably utilizing data sources from multiple terminals and multiple time periods, and combining advanced image processing techniques, the level of illegal parking monitoring in urban traffic management can be significantly improved. At the same time, the system uses multi-view information fusion and scene enhancement mechanisms, avoiding traditional "deduplication" and creatively proposing a semantic information overlay mechanism based on standard overhead view, achieving maximum utilization of data.

[0071] This step S110 not only emphasizes the uniqueness of data collection methods, but also highlights the advantages of this method in improving the accuracy, coverage, and efficiency of illegal parking detection.

[0072] S120, spatio-temporal clustering and correction is performed on the multiple non-continuous pictures, a static landmark extraction technique is used to locate the vehicle, a vehicle area is determined, and a location fingerprint is constructed for cross-period comparison to determine whether the vehicle is in a static state; wherein multi-view information fusion is performed based on the corrected non-continuous pictures to fuse the non-continuous pictures of multiple terminals and multiple angles into complete scene description information.

[0073] In this embodiment, on the basis of a dynamic digital map, non-continuous pictures corresponding to terminals of different views are fused to locate a vehicle.

[0074] The dynamic digital map is an interactive digital sand table established for the region, containing road geometry, lane lines, numbered parking spaces, and static facilities, and is used to obtain semantic information by analyzing non-continuous pictures corresponding to terminals of different views, and update the situation on the interactive digital sand table in real time with standardized markers.

[0075] Specifically, spatio-temporal clustering and image correction are performed on multiple non-continuous pictures, vehicle positioning is completed using a static landmark extraction technique, a location fingerprint is constructed and cross-period comparison is performed to determine whether the vehicle is in a static state; multi-view information fusion is performed based on the corrected pictures to integrate pictures collected non-continuously from different terminals and different views into complete scene description information.

[0076] This embodiment completes the above fusion under the support of a DDM (Dynamic Digital Map), which establishes an interactive "digital sand table" for the region, with road geometry, lane lines, numbered parking spaces, fire hydrants, manhole covers, and other static facilities fixed at the bottom; each non-continuous picture is regarded as an "intelligence sampling", and only semantic information (vehicle outline, position, license plate, confidence, etc.) is extracted to refresh the DDM in real time with standardized markers, realizing a leap from "pixel-level difference" to "semantic-level unity".

[0077] In an embodiment, the above step S120 can include steps S121-S125.

[0078] S121, group pictures with geographical positions meeting the requirements in the multiple non-continuous pictures based on GPS and timestamp information to obtain a monitoring area.

[0079] In this embodiment, a monitoring area refers to a series of pictures with adjacent geographical positions and close shooting times, which are grouped into one group according to the GPS positioning information and time stamp of the pictures. Specifically, if each picture in the series is located within a specific geographical area (for example, a circular area with a radius of 15 meters) and the shooting time interval is short, these pictures are considered to belong to the same "monitoring area". This concept is the basis for spatio-temporal clustering, which helps subsequent analysis of data at different time points of the same geographical location.

[0080] First, a series of pictures with adjacent geographical positions and close shooting times are grouped into a "monitoring area" using the GPS positioning information and time stamp attached to each picture. For example, all pictures within a specific geographical area (such as a circular area with a radius of 15 meters) and with a short shooting time interval are considered to belong to the same monitoring area. This step is the basis for subsequent spatio-temporal clustering, which helps to concentrate on the analysis of data at different time points of the same geographical location.

[0081] S122, converting the non-continuous pictures corresponding to the monitoring area into a distortion-free overhead view through a camera calibration matrix to obtain corrected non-continuous pictures.

[0082] In this embodiment, the corrected non-continuous pictures refer to images processed by the camera calibration matrix, which are converted from the original shooting angle to the standard overhead view. This process eliminates image distortion caused by differences in installation location and angle of different terminal devices, allowing all pictures to be compared in a unified two-dimensional coordinate system. The purpose of this is to ensure that pictures from different angles and positions can be accurately positioned and features extracted within the same reference framework.

[0083] Next, for each picture in the monitoring area, a camera calibration matrix is used to convert it into a distortion-free standard overhead view. This process eliminates image distortion caused by differences in installation location and angle of different terminal devices, allowing all pictures to be compared in a unified two-dimensional coordinate system. The purpose of this is to ensure that pictures from different angles and positions can be accurately positioned and features extracted within the same reference framework.

[0084] On the original map, the two lane lines are not parallel, and through the correction algorithm, the lane lines are converted to parallel. When the lane lines are parallel, the entire image can be considered as a high-altitude overhead view.

[0085] Specifically, because the camera's field of view is a slanted, downward view, the image appears trapezoidal, with distant objects appearing smaller and nearby objects larger. To address this issue, after image acquisition, a calibration matrix is ​​used to calibrate the camera, and this matrix is ​​then used to correct the original image, transforming the final image into a top-down view and ensuring all objects have the same pixel accuracy. The calibration and correction are based on prior information about lane lines. In the original image, the lane lines are not parallel; after correction, they are made parallel, thus achieving a top-down perspective conversion. This process utilizes distortion correction methods, mathematically based on single-application transformation processing.

[0086] S123. Use static landmarks to locate vehicles in the corrected discontinuous image to obtain the vehicle region.

[0087] In this embodiment, the vehicle area refers to the location range of the target vehicle determined by image recognition technology in the corrected image. The vehicle area is typically represented by the geometric center of the vehicle or the center of the license plate, and it is used to calculate the relative positional relationship between the vehicle and surrounding static landmarks. Accurately defining the vehicle area is crucial for determining whether a vehicle is correctly parked in a parking space and whether there is any illegal parking.

[0088] In one embodiment, step S123 described above may include steps S1231 to S1232.

[0089] S1231. Use a feature point detection algorithm to identify and extract fixed static landmarks on the corrected discontinuous image, wherein the static landmarks include long-term unchanging features, including parking space line corners and manhole covers.

[0090] In this embodiment, static landmarks are extracted from images taken at different times. These landmarks include corner points of parking lines, manhole covers on the road surface, specific markings or cracks on the ground, roadside fire hydrants, utility poles, and other features that remain unchanged over a long period of time and can be accurately located.

[0091] S1232. Based on license plate recognition or vehicle feature points, determine the position of the vehicle in the corrected discontinuous image to obtain a vehicle region, wherein the vehicle region includes position information represented by the vehicle's geometric center point or the license plate's center point.

[0092] Meanwhile, for the target vehicle in the picture, its accurate position in the overhead view can be determined through license plate recognition or vehicle feature point, which can be represented by the vehicle geometric center point or the license plate center point. Then the relative position "fingerprint" is calculated, i.e. for a picture taken at a certain time, the relative position vector of the target vehicle relative to its N nearest static landmarks is calculated. For example, for the picture P1 taken at t1, the relative position vector of the vehicle relative to the nearest three landmarks L1, L2 and L3 is calculated, and similarly, the corresponding vector can also be calculated for the picture P2 taken at t2.

[0093] S124, calculate the relative position vector of the target vehicle relative to the static landmark with a distance meeting the requirements in the corrected non-continuous picture at different times, to obtain the vehicle position fingerprint at different times.

[0094] In this embodiment, the vehicle position fingerprint at different times refers to the set of relative position vectors calculated for the target vehicle in the pictures taken at different time points (such as t1 and t2) relative to its N nearest static landmarks (such as the nearest three landmarks L1, L2 and L3). Each position fingerprint contains a series of information describing the position of the vehicle, such as the distance and direction of the vehicle center point to each landmark. By comparing the position fingerprints at different times, it can be determined whether the vehicle remains stationary, i.e. whether the vehicle has not changed significantly in position between the two time points. This technique is particularly suitable for long-term monitoring of the parking of a vehicle to identify possible illegal parking behavior.

[0095] For the corrected pictures taken at different times (such as t1 and t2), the relative position vector of the target vehicle relative to its N nearest static landmarks (such as the nearest three landmarks L1, L2 and L3) is calculated, thereby forming the vehicle position fingerprint at that time. Each position fingerprint contains a series of information describing the position of the vehicle, such as the distance and direction of the vehicle center point to each landmark. By comparing the position fingerprints at different times, it can be determined whether the vehicle remains stationary.

[0096] S125, determine whether the vehicle is in a stationary state based on the vehicle position fingerprints at different times.

[0097] In this embodiment, when the difference between the vehicle position fingerprints at different times is within a set tolerance range and the time interval exceeds a preset minimum judgment interval, it is determined that the vehicle is in a stationary state.

[0098] Finally, by comparing the vehicle's position fingerprints at different times, if the positional difference between two points in time is within a set tolerance range (e.g., within 10 centimeters), and the time interval exceeds a preset minimum judgment interval (e.g., 1 minute), it can be highly certain that the vehicle was stationary during that period. This method is particularly suitable for long-term monitoring of vehicle parking to identify potential illegal parking.

[0099] Throughout the process, the dynamic digital map serves as the underlying infrastructure, providing an interactive digital sandbox for real-time updates of the status of vehicles, parking spaces, and other static facilities. Once images from different terminals are parsed into semantic information, the system updates the corresponding status on the dynamic digital map using standardized markers. This not only improves data processing efficiency but also enhances the system's robustness and adaptability, enabling the effective fusion of discontinuous images from multiple terminals and angles, ultimately improving the accuracy and reliability of vehicle illegal parking detection.

[0100] In this embodiment, after receiving images from different terminals, the server clusters them based on GPS location information and timestamps, grouping images with similar geographical locations (such as within a circular area with a radius of 15 meters) into a "monitoring area". Each image within the monitoring area is processed using a camera calibration matrix M, correcting it from the original oblique top-down view to a uniform, distortion-free "top view", ensuring that all images are comparable in the same two-dimensional coordinate system. This coordinate system is based on inverse perspective transformation in the field of computer vision, and its mathematical basis is single-application transformation processing.

[0101] On the corrected top view, image recognition algorithms (such as SIFT, ORB, and other feature point detection algorithms) are used to automatically identify and extract fixed static landmarks, such as parking space line corners and manhole covers on the road. At the same time, the precise position of the target vehicle in the image (through license plate recognition or vehicle feature point locking) in the top view is determined, and the position can be represented by the geometric center point of the vehicle or the center point of the license plate.

[0102] For image P1 taken at time t1, calculate the relative position vector of the target vehicle with respect to N nearby static landmarks. This includes calculating the distance and direction from the vehicle's center point to the three nearest landmarks L1, L2, and L3, forming a "fingerprint" describing the vehicle's position: {d(vehicle, L1), d(vehicle, L2), d(vehicle, L3)}. When image P2 taken at time t2 from the same monitoring area is received, repeat the corresponding steps to calculate a new position "fingerprint" for the target vehicle (confirmed by license plate or vehicle features). Compare the "fingerprints" at times t1 and t2, and set a position tolerance threshold th. pos (e.g., 10 cm), if the difference in "fingerprints" is within the threshold... pos If the time difference between t2 and t1 is greater than the minimum judgment interval (e.g., 1 minute), then it can be determined that the vehicle is stationary during this period.

[0103] In this process, in the face of massive picture data, a distributed storage system is used for storage. After receiving the picture data, the space-time clustering is performed according to the GPS positioning information and the time stamp, and the camera calibration and image correction are called for each picture after clustering, to convert into a unified coordinate system and a unified scale "standard overhead view", to lay a foundation for fusion. The system identifies that a specific geographic location is photographed at different time points, to form a "monitoring area" or "point of interest". For pictures taken at different time points of the same monitoring area, a deep learning model is run on the server side to perform secondary vehicle recognition, license plate recognition and environment feature point extraction, to extract structured semantic information, including vehicle contour, position, license plate number, parking space marking line, static landmark position, and confidence of each information. These semantic information is superimposed on the dynamic digital map, which is not a physical combination of pictures, but a logical fusion of data. It is like an "interactive digital sand table" or "live point map", which contains road geometry, lane lines, numbered parking spaces and other static elements at the bottom. After the pictures sent by different terminals are interpreted as intelligence, the situation is updated on the sand table, which is completely different from the traditional physical combined picture.

[0104] S130, when the vehicle is in a static state, the parking area is divided and identified based on historical parking line information or current image recognition method, and the road panoramic information is identified and recorded to obtain environmental features.

[0105] In this embodiment, the parking area refers to a space surrounded by one or more sets of lines parallel or perpendicular to the road direction, which usually marks the area allowed to park vehicles. Accurate identification of the parking area is crucial for determining whether the vehicle is legally parked in the specified parking space.

[0106] Specifically, in order to improve the accuracy of parking area identification, the system first attempts to use historical parking line information. The historical parking line information is derived from previously captured image data, and by analyzing these data, the mathematical expressions of the lane lines (such as f(x) and g(x)) and their average interval distances (D1 and D2) can be extracted. Using this information, even if the parking lines in the current field of view are not fully acquired due to obstruction or other reasons, the system can still infer the length and width of the parking area based on historical data, and complete the parking lines in the current field of view accordingly.

[0107] Therefore, the parking space line in the image information is used to determine the parking space area in the field of view. The parking space is surrounded by the parking space line, but the parking space line cannot be completely obtained due to problems such as occlusion and lack, and the historical parking space line is used to fill in the current parking space line in the field of view. First, the historical parking space line information collected by the camera is extracted. If the historical parking space line information can be extracted, the mathematical expressions f(x) and g(x) of the horizontal parking space line and the vertical parking space line in the historical image are obtained, and the average interval distances D1 and D2 of the two are recorded. Thus, the length and width of the parking space are D1 and D2. Then, the historical parking space line information is used to divide the parking space line in the current field of view, and the horizontal and vertical lane line recognition results of the current image are combined with the historical results to accurately recognize the parking space information.

[0108] The interval of the horizontal parking space line is relatively fixed, and its position in the subsequent frame is relatively fixed, which can be described by a straight line expression. The position of the vertical parking space line is not fixed and changes with the left and right deviation of the vehicle, so it is described by a straight line expression Ax+By+C=0. The predicted position of the vertical parking space line in the subsequent frame is a range. The image processing algorithm is run in the subsequent frame to recognize the parking space line, and the recognition result is combined with the prediction result: if the parking space line is detected in the subsequent frame, the detected parking space line is the final parking space line; if the detected parking space line is incomplete, the detected parking space line is used as the main part and the predicted parking space line is used as the supplement; if the parking space line is not detected in the subsequent frame, the time t1 is recorded, and if the parking space line is not detected for a period of time, it is judged that the parking space line disappears, and if the parking space line still exists within the time interval, the predicted parking space line is used to supplement the missing part.

[0109] If the historical parking space line information is not available or insufficient, the system relies on the current image recognition technology to directly detect the parking space line from the image. This includes recognizing the position, interval and mathematical expression of the horizontal parking space line and the vertical parking space line. For the case where the recognition result is not complete, the system combines the recognition result of the current frame with the prediction result, for example, when the parking space line is detected in the subsequent frame, the detection result is preferred; if the detection result is incomplete, the historical data is supplemented; if the parking space line is not detected, it is judged whether the parking space line is missing according to the time sequence analysis.

[0110] The environmental features refer to various static and dynamic elements of the road and its surrounding environment in addition to the parking space area. These elements are of great significance for comprehensively understanding the environment in which the vehicle is located and judging the illegal parking behavior.

[0111] Static landmarks: including but not limited to the corner points of the parking space line, the manhole covers on the road surface, the specific marks or cracks on the ground, the roadside fire hydrants, the power poles, etc. These landmarks are important reference points for accurate positioning and description of the vehicle position due to their long-term invariability.

[0112] Traffic signs and road markings: such as sidewalks, fire access, bus lanes, no parking signs, etc., which are the key basis for judging whether the vehicle is illegally parked.

[0113] Vehicle-related information: This includes license plate recognition, vehicle type, color, brand and model identification, as well as precise positioning of vehicle location and orientation. In addition, it also includes parking state judgment, vehicle integrity and component state assessment, etc.

[0114] Lighting and weather conditions: Although these factors are secondary factors, they can provide additional information to help the algorithm model better understand and adapt to image data in different scenarios.

[0115] Specifically, full-scene recognition is performed on road images to obtain panoramic information on the road. The panoramic information includes vehicle type, which is useful for determining the legal parking area of the vehicle; license plate recognition is one of the key information, high-precision license plate recognition technology can uniquely identify the vehicle, which is the basis for tracking the vehicle trajectory and judging its stay time; vehicle color, brand and model identification can provide additional evidence for comparing vehicle features; vehicle location and orientation recognition, parking state judgment, vehicle integrity and component state, road marking line recognition (such as pedestrian walkway, fire access, bus lane, etc.), traffic sign recognition (such as no parking sign, vehicle type restriction, etc.), geographic features and reference objects, lighting and weather conditions, and personnel identification information can be used to supplement the algorithm model information.

[0116] Through the above steps, the system can efficiently and accurately complete the division and recognition of the parking area after the vehicle is confirmed to be stationary, and at the same time capture and record rich environmental feature information. This process not only supports accurate judgment of illegal parking behavior, but also provides strong technical support for urban traffic management. Finally, all this information will be integrated into a structured illegal parking report, including detailed illegal parking types, time and location, confidence score, key evidence chain, etc., and uploaded to the cloud as historical records.

[0117] S140, based on the environment features, parking area and vehicle area, according to the no-parking area rules, parking relationship and time period state, to judge whether the vehicle has illegal parking behavior.

[0118] In this embodiment, according to the priority level corresponding to the no-parking area rules, parking relationship and time period state, based on the environment features, parking area and vehicle area, to judge whether the vehicle has illegal parking behavior.

[0119] The no-parking area rules include checking whether the vehicle is located in a special channel or a specific place where parking is absolutely prohibited to determine the corresponding illegal parking type.

[0120] The parking relationship includes analyzing the relationship between the vehicle area and the parking area, and determining whether the vehicle has a boundary violation illegal parking behavior according to the vehicle area proportion; the time period state includes marking the long-term parked vehicles in combination with the time factor.

[0121] In this embodiment, for each identified vehicle, the system analyzes its presence at different time points: first using the image processing module, then using the parking space processing module for parking area extraction, and finally integrating all information with environmental features to make a judgment. The system triggers the illegal parking judgment engine after confirming that a vehicle has entered a "static" state and the duration exceeds a pre-set threshold (e.g., T still >3 minutes). The engine performs multi-level, multi-dimensional logical judgments in the following priority order:

[0122] First, the vehicle parking position is preliminarily evaluated according to the "absolute prohibition" of the no-parking area rule.

[0123] Special lane violation: If the vehicle area Rc overlaps with the special road marking line area (such as fire access, sidewalk, blind path, non-motor vehicle lane, etc.) identified by the image feature point extraction module, it is directly determined as a special lane violation.

[0124] Specific location violation: If the vehicle is parked within the no-parking range of specific geographical references (such as bus stop signs, gas station signs, fire hydrants, emergency station / fire department gates, etc.), the system will generate a virtual no-parking area based on these references and determine it as a specific location violation.

[0125] No-parking sign violation: When the vehicle Rc is parked in an area or its adjacent area where an explicit "no parking" or "no long-term parking" traffic sign is identified, it is determined as a no-parking sign area violation. In addition, the system can also implement time-based violation judgment combined with the time information on the sign.

[0126] If the vehicle does not trigger any of the above no-parking area rules, the system will further analyze the relationship between the vehicle and the parking space.

[0127] IOT calculation: Define the Intersection over Target (IOT) as an indicator to determine whether the vehicle is within the parking space. Its calculation method is the intersection area of the target and the limited area as a proportion of the target area. Specifically, the proportion of the intersection of the vehicle and the parking space to the vehicle is the Intersection over Target.

[0128] Set Rc as the vehicle area and Rp as the parking space area surrounded by the lane line. Define the Intersection over Target as an indicator to determine whether the vehicle is within the parking space. Its calculation method is the intersection area of the target and the limited area as a proportion of the target area. For parking scenarios, the proportion of the intersection of the vehicle and the parking space to the vehicle is the Intersection over Target.

[0129] The vehicle region and the parking space region are segmented on the image. First, the IOT between the vehicle region and the parking space region is calculated. If the IOT is less than 0, it is considered to be illegal parking. Then, it is identified whether there is a parking marker, such as a tripod, a parking sign, or the like, behind the vehicle. If there is a parking marker, it is determined to be “faulty illegal parking”. If there is no parking marker, it is determined to be “non-faulty illegal parking”. If the IOT is greater than 0, the distance dis between the center point of the vehicle and each parking space is calculated, and all dis are sorted. The parking space closer to the center point of the vehicle is determined to be the parking space of the vehicle. If the direction of the vehicle head is consistent with the driving direction, it is determined to be “normal parking”. If the direction of the vehicle head is inconsistent with the driving direction, it is determined to be “reverse illegal parking”. If the vehicle is parked too far forward or too far backward, blocking the parking space of other parking spaces, it is determined that the parking behavior is “longitudinal boundary violation illegal parking”. If the vehicle is parked too close to the road area, causing a driving safety hazard in the road area, it is determined that the parking behavior is “lateral boundary violation illegal parking”. Wherein R is a limited area, T is a target, S2 is the intersection of the target and the limited area, and S1 is the area without intersection between the target and the limited area. IOT=S2 / (S1+S2).

[0130] The following method is used to determine the type of vehicle boundary violation illegal parking:

[0131] When the vehicle has no intersection with the parking area (IOT=0) and there is a tripod or the like behind the vehicle, it is determined to be faulty illegal parking.

[0132] Similarly, if IOT=0 but there is no marker behind the vehicle, it is determined to be non-faulty illegal parking.

[0133] If IOT>0 and the vehicle head is against the driving direction, it is considered to be reverse illegal parking.

[0134] If IOT>0 but the vehicle is too far forward or backward, invading the space of other parking spaces, it is determined to be longitudinal boundary violation illegal parking.

[0135] If the vehicle invades the road space, it is determined to be lateral boundary violation illegal parking.

[0136] On this basis, the system adds a time dimension to discover special cases such as long-term occupation.

[0137] Suspected zombie car marking: through cross-terminal and cross-period data analysis, if it is found that the same vehicle is stationary at the same location (whether legal or illegal parking space) for more than an extremely long threshold (for example, T still >15 days), and the image features may also include auxiliary judgment information such as vehicle body dust and litter, the vehicle is marked as “suspected zombie car”. Further combined with the illegal parking attribute of the location (such as “zombie car in fire access” or “zombie car in legal parking space”), a zombie car illegal parking report is generated to request relevant departments to handle.

[0138] To improve the accuracy of the results, the system introduces a multi-perspective cross-validation mechanism, which uses the differences between different terminal data sources to correct itself and enhance the results, achieving multi-perspective consistency scoring.

[0139] When independent perspectives from different terminals confirm a parking violation event, the system will increase the final confidence score and include additional evidence in the report. If there is high-confidence contradictory evidence, the system will correct the preliminary results to avoid misjudgment. The terminal data with higher confidence is used as the basis for decision-making to eliminate uncertainty.

[0140] Specifically, based on the image from terminal A at time t1, the system preliminarily judges that a car is "pressing the line and violating parking" and gives an initial confidence C1 (for example, 70%) and a preliminary result.

[0141] The system does not immediately accept this result, but actively searches for data from other terminals within the time window (for example, t1 ± 3 minutes) at that geographic location.

[0142] Cross-validation and confidence correction are performed:

[0143] Case 1: Evidence enhancement: data from terminal B at time t2 is retrieved. After processing, it is clear from B's perspective that the car is pressing the line. Specifically, two independent and different perspectives confirm each other. The system will increase the final confidence C final of this parking violation event to a high level (for example, 95%) and use B's data as additional evidence.

[0144] Case 2: Result correction: data from terminal C at time t3 is retrieved. From C's unobstructed perspective, it is found that the "pressing line" position in terminal A's perspective is actually a dark stain on the road surface, and the vehicle is not pressing the line. If there is high-confidence contradictory evidence, the system will correct the preliminary result and determine that there is no violation, or the confidence will be reduced to a very low level, effectively avoiding a misjudgment.

[0145] Case 3: Ambiguity resolution: terminal A's preliminary judgment confidence C1 is very low (for example, 40%) because its own perspective is partially obstructed and cannot determine whether it is pressing the line. When terminal D's clear data is retrieved, it clearly confirms the pressing line behavior. That is, the data from terminal D with higher confidence is used as the final decision-making basis, eliminating the uncertainty brought by terminal A.

[0146] The optimization step creatively utilizes the natural differences of multi-terminal data sources, creating a self-checking and optimizing closed loop. It changes different terminals from independent informants to mutually supervised verifiers, improving the decision robustness and accuracy of the entire system through cross-validation. The more non-continuous data collected, the higher the accuracy of the final evaluation result. This is something that any single terminal detection method cannot achieve.

[0147] S150, when the vehicle exists a parking violation behavior, subdividing the parking violation type of the vehicle.

[0148] In this embodiment, the parking violation type includes normal parking, fault parking, non-fault parking, reverse parking, transverse boundary parking, and longitudinal boundary parking.

[0149] S160, creating a structured illegal parking report based on the parking violation type, wherein the illegal parking report contains vehicle information, parking violation type, time and location, confidence score, and evidence chain.

[0150] For each confirmed parking violation event, the system generates a detailed structured illegal parking report, covering vehicle identification, parking violation type, parking violation time and location, comprehensive confidence score, key frame pictures, and quantitative indicators, and uploads it to the cloud as a historical record. This method not only makes the parking violation judgment more accurate and comprehensive, but also easy to trace, providing strong technical support for urban traffic management.

[0151] When handling illegal parking events, the system will generate detailed parking violation reports. For each parking violation event, a structured illegal parking report is created, which contains the following key information:

[0152] Vehicle identification: using license plate number as the identification basis.

[0153] Parking violation type: determining the specific parking violation type according to pre-set rules, such as fire access violation, reverse parking, or longitudinal boundary parking, etc.

[0154] Parking violation time and location: including the specific time of parking violation start and end, and accurate GPS coordinate position.

[0155] Confidence score: based on factors such as image clarity, feature point extraction stability, and rule matching certainty, the system gives a comprehensive confidence score to assess the reliability of the judgment result.

[0156] The evidence chain includes:

[0157] Key frame pictures: providing original pictures, corrected pictures, and pictures annotated with relevant information (such as vehicle position, parking violation area boundary, etc.).

[0158] Quantitative metrics: such as specific values ​​recorded by Internet of Things (IoT) devices, the length of time a vehicle remains stationary (T). still wait.

[0159] Environmental evidence includes, but is not limited to, photos of no-parking signs and images of marked areas of fire lanes that are blocked.

[0160] This detailed information is uploaded to the cloud and stored as part of the historical record. In this way, illegal parking detection is not merely a simple set of rules, but a crucial link in the data processing flow and a key node in realizing the value transformation of data. This setup makes illegal parking judgments more accurate and comprehensive, and facilitates future queries and traceability. This comprehensive data collection and analysis method helps improve the efficiency and quality of urban traffic management.

[0161] For example, such as Figure 3 As shown, three terminals (A-taxi, B-bus, C-delivery vehicle) collected images at different times (10:01 AM, 10:03 AM, 10:08 AM) and took pictures of the same silver SUV from different angles (low angle, high angle, side) using the vehicle-mounted camera.

[0162] The cloud platform receives and integrates non-continuous, multi-view image data from the terminal, and performs feature extraction and comparison.

[0163] The images collected by the three terminals are shown in Table 1.

[0164] Table 1. Non-continuous images

[0165]

[0166] By using complementary low, high, and side angles, the problem of occlusion or blurring from a single perspective is compensated (e.g., the license plate is obscured in image A, but clear in image B; image C is clear, providing verification from multiple angles).

[0167] Despite a 7-minute time interval between shots, the consistency between the vehicle (silver SUV) and the static landmarks (manhole cover, lane lines, parking lines) confirms their identity.

[0168] Even if some images are occluded (such as the parking lines in image B and the parking lines in image C), the system can still reconstruct the complete information through cross-view feature association.

[0169] Therefore, the resulting discontinuous images can be used in intelligent traffic monitoring. Through multi-terminal collaboration, the problem of information loss caused by angle, occlusion, and time difference of a single camera can be solved, thereby improving the accuracy of vehicle recognition.

[0170] The obtained non-continuous pictures can be used for abnormal event tracing. If a vehicle is illegally parked or involved in an accident, historical pictures of different terminals can be traced back to splice a complete evidence chain (e.g., combined with a clear license plate and side collision marks).

[0171] The method of the embodiment utilizes heterogeneous data (view angle, time, and difference in definition) of distributed terminals, and realizes high-reliability vehicle identity confirmation and scene restoration through time-space alignment and feature fusion of a cloud platform, and is suitable for intelligent traffic management in a complex urban environment.

[0172] In this example, a silver SUV is photographed by different terminals at different times (10:01, 10:03, and 10:08), and the whole process of gradually forming and updating the “information card” is processed by the cloud platform.

[0173] Picture A taken by a taxi camera is at a low angle and the license plate is blocked. It is identified as a “silver SUV” with a position coordinate (x1, y1). The static landmark is “manhole cover + lane line”. The confidence level is 0.4 (low) because the license plate is blocked and the information is incomplete.

[0174] Picture B taken by a bus at a high angle has a clear license plate. The license plate is successfully identified (updated from “unknown” to “provincial AXOOOX”), and the lane line is improved. The confidence level jumps to 0.8 (high) because the key information (license plate and lane line) is complete.

[0175] Picture C taken by a delivery truck from the side also captures the license plate. It is cross-verified with the previous content. The confidence level is increased to 0.9 (high), and the system is marked as “to be further confirmed”.

[0176] When new content appears, the information card is updated (e.g., license plate, lane line, and blocking state). If the cumulative evidence supports a certain feature (e.g., the license plate “provincial A XOOOX” is verified multiple times), the information is locked. If there is a persistent conflict (e.g., the lane line state is contradictory), the confidence level is reduced or manual review is triggered. When the confidence level is greater than or equal to 0.95 and the key information (license plate, position, and color) is stable, it is marked as “illegal parking confirmation”.

[0177] Low angle (A) + high angle (B) + side (C) jointly solve the blocking and blurring problems. Through multiple observations at different times, errors are gradually corrected (e.g., from “unknown license plate” to “provincial A XOOOX”). The confidence level quantifies the reliability of the information, avoiding single misjudgment (e.g., the missing license plate in picture C does not affect the confirmed license plate number).

[0178] This example simulates the logic of “evidence chain construction” in real traffic monitoring. Through distributed terminals + cloud fusion, fragmented and noisy data is converted into a high-confidence vehicle profile, which is suitable for scenarios such as illegal parking identification and trajectory tracking.

[0179] Additionally, Raw Images A, B, C, etc.: These are the initial image data received by the system. These images can be acquired from a camera or other image capturing device.

[0180] The core processing module contains multiple sub-modules for a series of processing and analysis on the raw images:

[0181] Camera Calibration and Image Rectification: Correct the input images to eliminate distortions caused by camera position or angle, ensuring the accuracy of subsequent processing.

[0182] Vehicle Stationary Judgment: Analyze the vehicle state in the images to determine whether the vehicle is in a stationary state. This step may involve motion detection algorithms to distinguish between moving and stationary objects.

[0183] Parking Spot Processing: Identify and process parking spots in the images, which may include locating parking spots, determining their state (e.g., free or occupied), etc.

[0184] Image Feature Point Extraction: Extract key feature points from the images, such as vehicle outlines, license plates, landmarks, etc., for subsequent matching and recognition.

[0185] Data Analysis and Storage: Analyze the extracted feature points and other relevant information, and store the results for future use.

[0186] After processing by the core processing module, multiple information cards are generated, each containing detailed information after processing:

[0187] Each card corresponds to a processed object (such as a vehicle) and contains object categories (such as SUV), object colors (such as silver), geographic information (such as coordinates (x1, y1)), parking spot information, and comprehensive confidence, etc. For example, information card A describes an SUV with an unknown license plate, silver, located at coordinates (x1, y1), with incomplete parking spot recognition, and a comprehensive confidence of 0.4.

[0188] Finally, based on the processed information cards, a dynamic digital map is constructed and updated:

[0189] Over time, the system continuously receives new image data, processes and analyzes it, and updates the digital map. Each timestamp represents a state update at a certain time point. For example, at timestamp 1 (1001), the system identifies a vehicle with an unknown license plate, silver, located at coordinates (x1, y1), with incomplete parking spot recognition, and a comprehensive confidence of 0.4; at timestamp 2 (1003), the same vehicle is successfully identified with license plate number XXXXX, the parking spot state is updated to have lane lines and parking lines, and the comprehensive confidence is increased to 0.8.

[0190] The whole process shows the construction process from the original image data to the final dynamic digital map, involving image processing, feature extraction, data analysis and other steps, finally realizing the real-time monitoring and management of the scene.

[0191] The above-mentioned fusion multi-terminal non-continuous vehicle illegal parking detection method collects non-continuous pictures taken by different devices at different times and routes, and uses the combination of space-time clustering and correction technology and static landmark extraction to accurately locate the vehicle position, constructs the position fingerprint for cross-period comparison to determine whether the vehicle is stationary; then, multi-view information is fused to form a complete scene description, the parking space area is divided and the environmental features are recorded based on the parking line and image recognition technology, the illegal parking behavior is judged according to the no-parking rule, the parking space relationship and the time period state, the illegal parking type is subdivided and the structured illegal parking report is generated. This way overcomes the problems of single data source, inability to provide comprehensive scene information and difficulty in accurately judging the vehicle state in the prior art, realizes comprehensive and efficient monitoring of different types of illegal parking behavior, and at the same time ensures the accuracy and reliability of the monitoring data.

[0192] Figure 4 is a schematic block diagram of a fusion multi-terminal non-continuous vehicle illegal parking detection system 300 provided by an embodiment of the present application. As shown in Figure 4 corresponding to the above-mentioned fusion multi-terminal non-continuous vehicle illegal parking detection method, the present application also provides a fusion multi-terminal non-continuous vehicle illegal parking detection system 300. The fusion multi-terminal non-continuous vehicle illegal parking detection system 300 includes units for executing the above-mentioned fusion multi-terminal non-continuous vehicle illegal parking detection method, and the system can be configured in a server. Specifically, please refer to Figure 4 , the fusion multi-terminal non-continuous vehicle illegal parking detection system 300 includes an acquisition unit 301, a stationary determination unit 302, a parking space and environment recognition unit 303, an illegal parking behavior judgment unit 304, a type determination unit 305 and a report generation unit 306.

[0193] The acquisition unit 301 is configured to acquire pictures of the same region taken by terminals mounted on different devices at different time periods and on different routes, to obtain a plurality of discontinuous pictures; the static determination unit 302 is configured to perform spatio-temporal clustering and correction on the plurality of discontinuous pictures, to locate a vehicle by using a static landmark extraction technology, to determine a vehicle region, and to construct a location fingerprint for cross-period comparison, to determine whether the vehicle is in a static state; wherein, on the basis of the corrected discontinuous pictures, multi-view information fusion is performed, to fuse the discontinuous pictures of multiple terminals and multiple angles into complete scene description information; the parking space and environment recognition unit 303 is configured to, when the vehicle is in a static state, divide and recognize a parking space region based on historical parking line information or a current image recognition method, and to recognize and record road panoramic information, to obtain environmental features; the illegal parking behavior judgment unit 304 is configured to, based on the environmental features, the parking space region and the vehicle region, judge whether the vehicle has an illegal parking behavior according to a no-parking area rule, a parking space relationship and a time period state; the type determination unit 305 is configured to, when the vehicle has an illegal parking behavior, subdivide the illegal parking type of the vehicle; and the report generation unit 306 is configured to create a structured illegal parking report based on the illegal parking type, wherein the illegal parking report contains vehicle information, illegal parking type, time and location, confidence score and evidence chain.

[0194] In an embodiment, the static determination unit 302 is configured to fuse discontinuous pictures corresponding to terminals of different views on the basis of a dynamic digital map, to locate a vehicle; wherein the dynamic digital map is an interactive digital sand table of the region containing road geometry, lane line, numbered parking space and static facility, for obtaining semantic information by analyzing discontinuous pictures corresponding to terminals of different views, and for updating the situation on the interactive digital sand table in real time with standardized markers.

[0195] In an embodiment, the static determination unit 302 comprises:

[0196] The grouping subunit is configured to group pictures in the plurality of discontinuous pictures that meet a requirement in geographical position distance based on GPS and timestamp information, to obtain a monitoring region; the conversion subunit is configured to convert discontinuous pictures corresponding to the monitoring region into distortion-free overhead views by a camera calibration matrix, to obtain corrected discontinuous pictures; the positioning subunit is configured to locate a vehicle by using static landmarks on the corrected discontinuous pictures, to obtain a vehicle region; the fingerprint construction subunit is configured to calculate a relative position vector of a target vehicle with respect to a static landmark that meets a requirement in the corrected discontinuous pictures at different time points, to obtain vehicle location fingerprints at different time points; and the state determination subunit is configured to determine whether the vehicle is in a static state based on the vehicle location fingerprints at different time points.

[0197] In an embodiment, the positioning subunit comprises:

[0198] a static landmark extraction module configured to identify and extract fixed static landmarks on the corrected non-continuous picture using a feature point detection algorithm, wherein the static landmarks comprise long-term invariant features, including parking line corner points and manhole covers; and a position determination module configured to determine a position of the vehicle in the corrected non-continuous picture based on license plate recognition or vehicle feature points to obtain a vehicle region, wherein the vehicle region comprises position information represented by a vehicle geometric center point or a license plate center point.

[0199] In an embodiment, the state determination subunit is configured to determine that the vehicle is in a static state when a difference between vehicle position fingerprints at different time instants is within a set tolerance range and a time interval exceeds a preset minimum judgment interval.

[0200] In an embodiment, the illegal parking behavior judgment unit 304 is configured to determine whether the vehicle has an illegal parking behavior based on the environment features, the parking area, and the vehicle region according to a priority level corresponding to a no-parking area rule, a parking space relationship, and a time period state.

[0201] It should be noted that a person skilled in the art can clearly understand the specific implementation process of the above-mentioned fusion multi-terminal non-continuous vehicle illegal parking detection system 300 and each unit, which can refer to the corresponding description in the foregoing method embodiments. For the convenience and brevity of description, it will not be repeated here.

[0202] The above-mentioned fusion multi-terminal non-continuous vehicle illegal parking detection system 300 can be implemented in the form of a computer program, which can run on a computer device as shown in the computer device. Figure 5

[0203] Please refer to Figure 5 , Figure 5 is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 500 can be a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0204] Refer to Figure 5 , the computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501, wherein the memory can include a non-volatile storage medium 503 and an internal memory 504.

[0205] ​The nonvolatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions which, when executed, can cause the processor 502 to perform the fusion multi-terminal non-continuity vehicle illegal parking detection method.

[0206] The processor 502 is configured to provide computing and control capabilities to support the operation of the entire computer device 500.

[0207] The memory 504 provides an environment for the computer program 5032 in the nonvolatile storage medium 503 to run, and the computer program 5032, when executed by the processor 502, can cause the processor 502 to perform the fusion multi-terminal non-continuity vehicle illegal parking detection method.

[0208] The network interface 505 is configured to perform network communication with other devices. Those skilled in the art can understand that the network interface 505 can be configured to perform wired or wireless communication with the network. Figure 5 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device 500 to which the scheme of the present application is applied. The specific computer device 500 can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0209] The processor 502 is configured to run the computer program 5032 stored in the memory to implement all steps of the fusion multi-terminal non-continuity vehicle illegal parking detection method.

[0210] It should be understood that, in the embodiments of the present application, the processor 502 can be a central processing unit (CPU), and the processor 502 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), ready programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0211] Those skilled in the art can understand that all or part of the processes in the method of the above embodiments can be completed by instructing the relevant hardware by a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the above method embodiments.

[0212] Therefore, the present application also provides a storage medium. The storage medium can be a computer readable storage medium. The storage medium stores a computer program, wherein the computer program is executed by a processor to make the processor execute all steps of the fusion multi-terminal discontinuity vehicle illegal parking detection method.

[0213] The storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various computer readable storage media that can store program codes.

[0214] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0215] In several embodiments provided by the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed.

[0216] The steps in the method of the embodiments of the present application can be adjusted, combined and deleted in sequence according to actual needs. The units in the system of the embodiments of the present application can be combined, divided and deleted according to actual needs. In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0217] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a storage medium. Based on such an understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application.

[0218] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for detecting illegally parked vehicles across multiple terminals and discontinuous routes, characterized in that: include: To obtain multiple non-continuous images, images of the same area taken by terminals mounted on different devices at different times and along different routes are obtained. Based on GPS and timestamp information, images that meet the geographical distance requirements among the multiple non-continuous images are grouped to obtain the monitoring area; The discontinuous images corresponding to the monitored area are converted into distortion-free top views using a camera calibration matrix to obtain corrected discontinuous images. The feature point detection algorithm is used to identify and extract fixed static landmarks on the corrected discontinuous image, wherein the static landmarks include long-term unchanging features, including parking space line corners and manhole covers; Based on license plate recognition or vehicle feature points, the position of the vehicle is determined in the corrected discontinuous image to obtain a vehicle region, wherein the vehicle region includes position information represented by the vehicle geometric center point or the license plate center point; For the corrected discontinuous image at different times, calculate the relative position vector of the target vehicle with respect to static landmarks at the required distances to obtain the vehicle position fingerprint at different times; The vehicle's position fingerprint at different times determines whether the vehicle is stationary; Among them, multi-view information fusion is performed on the basis of the corrected discontinuous images to integrate discontinuous images from multiple terminals and angles into complete scene description information; When the vehicle is stationary, the parking area is divided and identified based on historical parking line information or the current image recognition method, and the panoramic road information is identified and recorded to obtain environmental features. Based on the environmental characteristics, parking space area, and vehicle area, according to the no-parking zone rules, parking space relationships, and time status, it is determined whether the vehicle has engaged in illegal parking behavior. If the vehicle is illegally parked, then the type of illegal parking is further subdivided. A structured illegal parking report is created based on the aforementioned illegal parking type, wherein the illegal parking report includes vehicle information, illegal parking type, time and location, confidence score, and evidence chain.

2. The method for detecting discontinuous vehicle parking violations using multiple terminals as described in claim 1, characterized in that, The step of fusing multi-view information based on the corrected discontinuous images to integrate discontinuous images from multiple terminals and angles into complete scene description information includes: Based on dynamic digital maps, non-continuous images from terminals with different perspectives are integrated to locate vehicles. The dynamic digital map is an interactive digital sand table that includes road geometry, lane lines, numbered parking spaces, and static facilities in the area. It is used to obtain semantic information by analyzing non-continuous images from terminals with different perspectives and to update the situation on the interactive digital sand table in real time with standardized markers.

3. The method for detecting discontinuous vehicle parking violations using multiple terminals as described in claim 1, characterized in that, The equipment includes different types of vehicles, including taxis, buses, coaches, and delivery vehicles; the installation location and angle of the terminal are not limited.

4. The method for detecting discontinuous vehicle parking violations using multiple terminals as described in claim 1, characterized in that, Determining whether the vehicle is stationary based on vehicle position fingerprints at different times includes: If the difference between the vehicle position fingerprints at different times is within the set tolerance range and the time interval exceeds the preset minimum judgment interval, then the vehicle is determined to be stationary.

5. The method for detecting discontinuous vehicle parking violations using multiple terminals as described in claim 1, characterized in that, The determination of whether a vehicle has engaged in illegal parking based on the environmental characteristics, parking space area, and vehicle area according to no-parking zone rules, parking space relationships, and time period status includes: Based on the no-parking zone rules, the priority level corresponding to the parking space relationship and the time period, and the environmental characteristics, parking space area, and vehicle area, it is determined whether the vehicle has engaged in illegal parking.

6. The method for detecting discontinuous vehicle parking violations using multiple terminals as described in claim 5, characterized in that, The no-parking zone rules include checking whether the vehicle is located in a special lane or specific location where parking is absolutely prohibited, in order to determine the corresponding type of illegal parking.

7. The method for detecting discontinuous vehicle parking violations using multiple terminals as described in claim 5, characterized in that, The parking space relationship includes analyzing the relationship between the vehicle area and the parking space area, and determining whether the vehicle has illegally parked beyond the designated area based on the proportion of the vehicle area; the time period status includes marking vehicles that have been parked for a long time in combination with time factors.

8. An integrated multi-terminal, non-continuous vehicle illegal parking detection system, characterized in that, include: The acquisition unit is used to acquire images of the same area taken by terminals mounted on different devices at different time periods and on different routes, so as to obtain multiple non-continuous images. The static determination unit includes a grouping subunit, a transformation subunit, a positioning subunit, a fingerprint construction subunit, and a state determination subunit; among which, The grouping subunit is used to group images from the multiple non-continuous images whose geographical distances meet the requirements based on GPS and timestamp information, so as to obtain the monitoring area; The conversion subunit is used to convert the discontinuous image corresponding to the monitoring area into a distortion-free top view through the camera calibration matrix, so as to obtain a corrected discontinuous image. The positioning subunit includes: a static landmark extraction module, used to identify and extract fixed static landmarks using a feature point detection algorithm on the corrected discontinuous image, wherein the static landmarks include long-term unchanging features, including parking space line corners and manhole covers; and a position determination module, used to determine the position of the vehicle in the corrected discontinuous image based on license plate recognition or vehicle feature points to obtain a vehicle region, wherein the vehicle region includes position information represented by the vehicle's geometric center point or the license plate's center point. The fingerprint construction subunit is used to calculate the relative position vector of the target vehicle with respect to static landmarks at the required distance from the corrected discontinuous image at different times, so as to obtain the vehicle position fingerprint at different times; the state determination subunit is used to determine whether the vehicle is in a stationary state based on the vehicle position fingerprint at different times. Multi-view information fusion is performed on the corrected discontinuous images to integrate discontinuous images from multiple terminals and angles into complete scene description information; The parking space and environment recognition unit is used to divide and recognize the parking space area based on historical parking line information or the current image recognition method when the vehicle is stationary, and to recognize and record the panoramic road information to obtain environmental features. The illegal parking behavior judgment unit is used to determine whether the vehicle has engaged in illegal parking behavior based on the environmental characteristics, parking space area, and vehicle area according to the no-parking zone rules, parking space relationship, and time period status. A type determination unit is used to further classify the type of illegal parking of a vehicle when the vehicle is found to be illegally parked. The report generation unit is used to create a structured illegal parking report based on the illegal parking type, wherein the illegal parking report includes vehicle information, illegal parking type, time and location, confidence score and evidence chain.

Images