Vehicle illegal parking detection method and system based on dynamic edge calculation

Abstract

The invention discloses a vehicle illegal parking detection method and system based on dynamic edge calculation. The method comprises the following steps: acquiring information acquired by a mobile edge device, and preprocessing; carrying out full-scene identification; tracking the vehicle to determine whether the vehicle is in a stationary state; when the vehicle is in a static state, determining a parking space area, and when historical image data containing parking space lines can be obtained, intelligently predicting and complementing the parking space lines which may be shielded or missing through a mathematical model established by fusing the parking space lines of the parking space in an analysis result of a current frame and historical parking space line data; determining whether the vehicle is illegally parked, and when the vehicle is illegally parked, determining the illegal parking type of the vehicle in combination with the parking marker in the analysis result; and illegal parking information is formed through combination. By implementing the method provided by the invention, not only can the problems of missed shooting, false shooting and insufficient identification precision in the prior art be overcome, but also accurate classification and real-time monitoring of different types of illegal parking behaviors can be realized.

Description

Technical Field

[0001] The present invention relates to a method for detecting illegal parking, and more specifically to a method and system for detecting illegal parking of a vehicle based on dynamic edge computing. Background Art

[0002] In urban traffic management, accurately judging whether a vehicle is illegally parked is crucial to maintaining road order. Traditionally, illegal parking monitoring relies on high-altitude cameras for shooting and analysis. However, this method has several limitations. First, due to the fixed installation position of the high-altitude camera, its shooting angle and time are difficult to ensure coverage of all key scenes, especially when the target is blocked, it is impossible to effectively identify and judge whether the vehicle constitutes illegal parking. Secondly, when the vehicle is traveling at a slower speed, the system may mistakenly judge it as illegal parking, resulting in false alarms; at the same time, static image analysis technology cannot accurately assess the impact of illegal parking on traffic flow, that is, it is impossible to quantify the contribution of illegal parking to road congestion. Furthermore, the effective monitoring area of ​​a single high-altitude camera is relatively small, especially in complex or restricted urban environments, the recognition accuracy is significantly reduced. Moreover, in order to expand the monitoring range and improve the recognition accuracy, some solutions suggest adding multiple cameras for deployment, but this undoubtedly greatly increases the hardware investment and maintenance costs. Finally, mobile monitoring equipment installed on the top of the vehicle can be used to capture long-term illegal parking. Although this method can reduce costs to a certain extent, its monitoring capability is still limited to a single vehicle and can only detect specific types of illegal parking. It cannot fully cover multiple categories and all scenarios of illegal parking.

[0003] Therefore, it is necessary to design a new method that can not only overcome the problems of missed shots, false shots and insufficient recognition accuracy in the existing technology, but also accurately classify and monitor different types of illegal parking behaviors in real time. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the prior art and provide a vehicle illegal parking detection method and system based on dynamic edge computing.

[0005] To achieve the above purpose, the present invention adopts the following technical solution: a vehicle illegal parking detection method based on dynamic edge computing, comprising: Acquire video information or image information collected by the mobile edge device, and perform preprocessing to obtain preprocessed image data; Performing full scene recognition on the preprocessed image data to identify panoramic information on the road to obtain an analysis result, wherein the analysis result includes vehicles, parking spaces, and parking signs; Tracking the vehicle of the analysis result to determine whether the vehicle is in a stationary state; When the vehicle is stationary, the parking area is determined, wherein, when historical image data containing parking space lines can be obtained, the parking space lines in the analysis result of the current frame are integrated with the mathematical model established by the historical parking space line data to intelligently predict and complete the parking space lines that may be blocked or missing; Determine whether the vehicle is illegally parked according to the intersection ratio between the parking space area and the area occupied by the vehicle, and when the vehicle is illegally parked, determine the type of illegal parking of the vehicle in combination with the parking sign in the analysis result; The illegal parking type, vehicle license plate number and illegal parking location of the vehicle are combined to form illegal parking information, and the illegal parking information is sent to the terminal.

[0006] A further technical solution is: the video information or image information collected by the mobile edge device is obtained and preprocessed to obtain preprocessed image data, including: Obtaining video information or image information collected by a mobile edge device; Camera calibration and image correction are performed on the video information or the image information to obtain preprocessed image data.

[0007] A further technical solution is: performing full scene recognition on the pre-processed image data to identify panoramic information on the road to obtain an analysis result, including: The pre-processed image data is subjected to full scene recognition through a target segmentation algorithm to identify panoramic information on the road to obtain an analysis result.

[0008] A further technical solution is: tracking the vehicle of the analysis result to determine whether the vehicle is in a stationary state, including: Using a multi-target tracking algorithm to identify and continuously track the vehicle in the analysis result to obtain the field of view trajectory of the vehicle; Dividing the preprocessed image data into upper and lower regions, randomly selecting an area from the upper region as a stationary reference point, and determining a motion trajectory formed by the stationary reference point during the tracking process to obtain a visual field trajectory of the stationary reference point; During the tracking process, the relative distance between the position of the vehicle in the analysis result and the stationary reference point is calculated to obtain a series of distance data arrays that vary with time; Calculating the similarity between the visual field trajectory of the stationary reference point and the visual field trajectory of the vehicle; Calculating the variance of the distance data array; It is determined whether the vehicle is in a stationary state according to the similarity and the variance.

[0009] A further technical solution is: determining whether the vehicle is in a stationary state according to the similarity and the variance includes: When the similarity is not less than a preset threshold and the variance is not greater than another preset threshold, it is determined that the vehicle is in a stationary state; when the similarity is less than a preset threshold and / or the variance is greater than another preset threshold, it is determined that the vehicle is not in a stationary state.

[0010] A further technical solution is: when the vehicle is stationary, the parking area is determined, wherein when historical image data containing parking space lines can be obtained, a mathematical model established by fusing the parking space lines in the analysis result of the current frame with the historical parking space line data is used to intelligently predict and complete the parking space lines that may be blocked or missing, including: Determine whether historical image data containing parking space lines can be obtained; If historical image data containing parking space lines can be obtained, then the linear equations of the transverse parking space lines and the longitudinal parking space lines in the historical image data, the average spacing distance of the transverse parking space lines, and the average spacing distance of the longitudinal parking space lines are parsed and obtained to obtain a mathematical model of the historical parking space lines; The parking space lines corresponding to the parking spaces in the analysis results are merged using the historical parking space line mathematical model to obtain the final parking space lines, and then obtain the parking space area.

[0011] A further technical solution thereof is: after determining whether the historical image data containing the parking space line can be obtained, it also includes: If the historical image data containing the parking space lines cannot be obtained, the interval distances and straight line equations corresponding to the transverse parking space lines and the longitudinal parking space lines of the parking spaces in the analysis results are recorded to further determine the parking space area.

[0012] A further technical solution is: determining whether the vehicle is illegally parked according to the intersection ratio between the parking space area and the area occupied by the vehicle, and when the vehicle is illegally parked, determining the type of illegal parking of the vehicle in combination with the parking sign in the analysis result, including: Calculating the intersection ratio between the parking space area and the area occupied by the vehicle; Determining whether the intersection ratio is greater than a set threshold; If the intersection ratio is greater than a set threshold, it is determined that the vehicle is illegally parked; Calculating the distance between the center point of the vehicle and each of the parking areas, and determining the parking area closest to the center point of the vehicle as the parking space corresponding to the vehicle, so as to obtain the illegal parking position; The front direction of the vehicle is determined, and the illegal parking type of the vehicle is determined in combination with the parking sign in the analysis result.

[0013] A further technical solution is: the types of illegal parking of the vehicle include illegal parking with fault, illegal parking without fault, illegal parking in the opposite direction, illegal parking beyond the longitudinal boundary and illegal parking beyond the lateral boundary.

[0014] The present invention also provides a vehicle parking violation detection system based on dynamic edge computing, including: An acquisition unit, used to acquire video information or image information collected by the mobile edge device and perform preprocessing to obtain preprocessed image data; A full scene recognition unit, used to perform full scene recognition on the preprocessed image data to identify panoramic information on the road to obtain a parsing result, wherein the parsing result includes vehicles, parking spaces, and parking signs; A tracking unit, used for tracking the vehicle of the analysis result to determine whether the vehicle is in a stationary state; A parking area determination unit, configured to determine a parking area when the vehicle is stationary, wherein when historical image data containing parking space lines can be obtained, a mathematical model established by fusing the parking space lines of the parking spaces in the analysis result of the current frame with the historical parking space line data is used to intelligently predict and complete the parking space lines that may be blocked or missing; a parking violation type determination unit, configured to determine whether the vehicle is illegally parked according to the intersection ratio between the parking space area and the area occupied by the vehicle, and when the vehicle is illegally parked, determine the parking violation type of the vehicle in combination with the parking sign in the analysis result; The information generating unit is used to combine the illegal parking type, the vehicle license plate number and the illegal parking location of the vehicle to form illegal parking information, and send the illegal parking information to the terminal.

[0015] The beneficial effects of the present invention compared with the prior art are as follows: the present invention collects and preprocesses video or image data through mobile edge devices to ensure high-quality input information and reduce the problems of missed shots and wrong shots. The full-scene recognition algorithm is used to accurately identify vehicles, parking spaces and parking signs to improve recognition accuracy. The static state detection and tracking algorithm is used to accurately determine the illegal parking situation, and the intersection ratio of the parking space area and the vehicle position is combined to accurately determine the illegal parking behavior. Finally, the illegal parking type is further classified based on the parking signs, and the illegal parking information is sent to the terminal in real time to achieve accurate monitoring. It can not only overcome the problems of missed shots, wrong shots and insufficient recognition accuracy in the prior art, but also accurately classify and monitor different types of illegal parking behaviors in real time.

[0016] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of a flow chart of a vehicle parking violation detection method based on dynamic edge computing provided by an embodiment of the present invention; Figure 2 A schematic diagram of a camera calibration and image correction process provided by an embodiment of the present invention; Figure 3 A schematic diagram of the effects of camera calibration and image correction provided by an embodiment of the present invention; Figure 4 A schematic diagram of a flow chart of determining whether a vehicle is in a stationary state provided by an embodiment of the present invention; Figure 5 A schematic diagram of a process for processing parking space lines provided in an embodiment of the present invention; Figure 6 A schematic diagram of a calculation process of an intersection ratio provided in an embodiment of the present invention; Figure 7 A schematic diagram of normal parking provided by an embodiment of the present invention; Figure 8 A schematic diagram of faulty illegal parking provided by an embodiment of the present invention; Fig. 9 A schematic diagram of a fault-free illegal parking provided by an embodiment of the present invention; Fig.10 A schematic diagram of reverse illegal parking provided by an embodiment of the present invention; Fig.11 A schematic diagram of lateral cross-border parking violation provided by an embodiment of the present invention; Fig.12 A schematic diagram of longitudinal cross-border parking violation provided by an embodiment of the present invention; Fig.13 A schematic block diagram of a vehicle illegal parking detection system based on dynamic edge computing provided by an embodiment of the present invention; Fig.14 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0021] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0022] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0023] See also Figure 1 , Figure 1 A schematic flow chart of a vehicle illegal parking detection method based on dynamic edge computing provided in an embodiment of the present invention. The vehicle illegal parking detection method based on dynamic edge computing is applied to a server. The server interacts with a mobile edge device for data exchange, wherein the mobile edge device may be a camera installed on a moving vehicle such as a bus, and video or image information is collected through the mobile edge device, and preprocessed to obtain high-quality image data to ensure that missed shots and wrong shots are reduced. Subsequently, a full-scene recognition algorithm is used to identify vehicles, parking spaces, and parking signs on the road, thereby improving recognition accuracy. The stationary state of the vehicle is detected and tracked to accurately determine whether the vehicle is illegally parked. Combined with the intersection of the parking space area and the vehicle position, it is accurately determined whether the vehicle is illegally parked, and the illegal parking type is further classified according to the parking signs. Finally, the illegal parking information (including vehicle type, license plate number, and illegal parking location) is sent to the terminal in real time to achieve accurate classification and real-time monitoring of illegal parking behavior.

[0024] Figure 1 FIG. 1 is a flow chart of a vehicle parking violation detection method based on dynamic edge computing provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110 to S160.

[0025] S110: Obtain video information or image information collected by the mobile edge device, and perform preprocessing to obtain preprocessed image data.

[0026] In this embodiment, the pre-processed image data refers to the image after camera calibration and correction.

[0027] In one embodiment, the above-mentioned step S110 may include steps S111 - S112 .

[0028] S111. Obtain video information or image information collected by the mobile edge device.

[0029] In this embodiment, devices deployed in a mobile edge computing (MEC) environment are used to capture video information in real time. These devices can be cameras installed on vehicles, traffic monitoring facilities, etc., which can provide dynamic image data about the current driving section. In addition to continuous video streams, mobile edge devices can also capture static image information for subsequent more in-depth analysis and processing.

[0030] This step is designed to ensure that the data obtained is the latest and directly reflects the current road conditions. By adopting MEC technology, not only the speed of data transmission is improved, but also the load on the cloud server is reduced, allowing the system to respond and handle emergency situations faster.

[0031] S112: Perform camera calibration and image correction on the video information or the image information to obtain pre-processed image data.

[0032] See also Figures 2 to 3 , using lane lines as the key reference in the calibration process. Because lane lines are regular and stable on the road, they are very suitable as the basis for calibration. According to the pre-set calibration parameters (i.e., calibration matrix M), a geometric transformation is performed on the original image to correct the trapezoidal distortion caused by the camera's oblique downward angle, thereby achieving image rotation.

[0033] A specially designed algorithm is used to eliminate nonlinear distortion caused by the lens, ensuring that all objects in the image have uniform pixel accuracy. Through adjustment, the image originally shot from an oblique perspective is converted into the effect of looking vertically downward from a high altitude. This means that after correction, the lane lines in the image will appear parallel, simulating an ideal high-altitude bird's-eye view.

[0034] Finally, after processing in step S112, pre-processed high-quality image data is obtained. These data have higher accuracy and smoothness, providing a solid foundation for subsequent higher-level tasks (such as vehicle identification, illegal parking judgment, etc.).

[0035] S120, performing full scene recognition on the preprocessed image data to identify panoramic information on the road to obtain an analysis result, wherein the analysis result includes vehicles, parking spaces, and parking signs.

[0036] In this embodiment, the pre-processed image data is subjected to full scene recognition through a target segmentation algorithm to identify panoramic information on the road to obtain an analysis result.

[0037] The high-quality image obtained through the camera calibration and image correction in the aforementioned step S110 is used as input to ensure that all objects in the image have the same pixel accuracy and are presented from a high-altitude bird's-eye view, which is conducive to improving recognition accuracy.

[0038] Using deep learning frameworks or traditional computer vision techniques, such as convolutional neural networks (CNNs), the unique features of different targets in the image are automatically learned and extracted. Based on the extracted features, each pixel in the image is classified to determine the object category to which it belongs; at the same time, the position boundaries of each object are accurately located to form a segmentation mask. The final output contains a list of information about all identified targets, such as the specific location and number of vehicles, the location distribution of available parking spaces, and the presence or absence of parking signs and their coordinates.

[0039] In summary, in this embodiment, by applying the target segmentation algorithm to the pre-processed image data, the panoramic information on the road is effectively recognized, and detailed analysis results are obtained, covering multiple aspects such as vehicles, parking spaces, and parking signs. This not only enhances the intelligence level of the system, but also provides reliable data support for subsequent illegal parking detection.

[0040] S130: Track the vehicle in the analysis result to determine whether the vehicle is stationary.

[0041] In this embodiment, step S130 is to track the vehicle in the analysis result by a moving object monitoring method to determine whether the vehicle is stationary. Specifically, only when the vehicle is confirmed to be stationary will it be further considered whether it constitutes illegal parking.

[0042] In one embodiment, the above-mentioned step S130 may include steps S131 to S136.

[0043] S131, using a multi-target tracking algorithm to identify and continuously track the vehicle in the analysis result to obtain the field of view trajectory of the vehicle.

[0044] In this embodiment, since the video acquisition vehicle is in motion for video acquisition, the trajectory of the vehicle in the field of view is not the actual movement trajectory of the vehicle, and may be referred to as the field of view trajectory hereinafter.

[0045] Using advanced multi-target tracking algorithms (such as SORT, Deep SORT, etc.), all vehicles in the field of view are identified and continuously tracked. For each vehicle, the system generates a unique ID and records its position changes in the video frame sequence to form the field of view track (TC) of the vehicle.

[0046] S132, dividing the preprocessed image data into two upper and lower regions, and randomly selecting an area from the upper region as a stationary reference point, and determining a motion trajectory formed by the stationary reference point during the tracking process to obtain a field of view trajectory of the stationary reference point.

[0047] In this embodiment, the entire field of view is divided into two main areas, the upper area and the lower area. The upper area is used to select a static reference point. A region R1 is randomly selected in the upper area as a static reference point, and this region is tracked as a target. Since the road itself is stationary, the motion trajectory (TR) of the static reference point R1 in the field of view can be used as a static benchmark to measure the motion state of other objects.

[0048] S133. During the tracking process, the relative distance between the position of the vehicle in the analysis result and the stationary reference point is calculated to obtain a series of distance data arrays that vary with time.

[0049] In this embodiment, for each tracked vehicle, the relative distance D1 between the vehicle and the stationary reference point R1 is calculated. As the video acquisition vehicle moves, the value of D1 changes, forming a series of distance data arrays [D1] that change over time.

[0050] S134. Calculate the similarity between the field of view trajectory of the stationary reference point and the field of view trajectory of the vehicle.

[0051] In this embodiment, cosine similarity is used to quantify the similarity between the field of view trajectory TR of R1 and the field of view trajectory TC of the vehicle. This step is intended to evaluate whether the motion patterns of the two are consistent.

[0052] S135. Calculate the variance of the distance data array.

[0053] In this embodiment, the position stability of the vehicle relative to R1 is measured by calculating the variance (Variance, V) of the [D1] array. The smaller the variance, the more stable the position of the vehicle relative to R1; otherwise, it means that the vehicle position changes greatly.

[0054] S136: Determine whether the vehicle is stationary according to the similarity and the variance.

[0055] In this embodiment, when the similarity is not less than a preset threshold and the variance is not greater than another preset threshold, it is determined that the vehicle is in a stationary state; when the similarity is less than the preset threshold and / or the variance is greater than another preset threshold, it is determined that the vehicle is not in a stationary state.

[0056] Specifically, if the similarity S between the visual field trajectory TC of the vehicle and the visual field trajectory TR of the stationary reference point R1 is not less than a preset threshold th2, and at the same time the variance V of the [D1] array is not greater than another preset threshold th1, then it can be inferred that the vehicle is in a stationary state.

[0057] If any of the above conditions is not satisfied (i.e., S < th2 or V > th1), it is considered that the vehicle is moving. At this time, the system can choose to stop further tracking of this vehicle and turn to focus on other possible stationary vehicles.

[0058] Thus, once a vehicle is determined to be in a stationary state, the system will start additional logic to determine whether this vehicle is illegally parked. For the vehicles within the visual field, their visual field trajectories are recorded as TC. If the visual field trajectory TC of the vehicle has a high similarity with the visual field trajectory TR of the stationary reference point R1, and the relative distance D1 between the stationary reference point R1 and the vehicle in the visual field is relatively stable, then it can be determined that the vehicle is in a stationary state. On the contrary, if the similarity between the visual field trajectory TC of the vehicle and the visual field trajectory TR of the stationary reference point R1 is small or the relative distance between the stationary reference point R1 and the vehicle in the visual field fluctuates greatly, then it is considered that the vehicle is in a moving state. Here, cosine is used to describe the similarity S between TR and TC, and the variance V is used to measure the volatility of the relative distance D1. If the vehicle is determined to be in a moving state, there is no need to perform subsequent tracking on this vehicle; if the vehicle is determined to be in a stationary state, further illegal parking judgment is required. Specifically, for the randomly selected area R1 on the road surface, the relative length D1 can be calculated in the vertical direction between it and the observed vehicle in the visual field. As the video - collecting vehicle keeps moving, this relative length will be continuously calculated to form the D1 array. At the same time, as the vehicle moves, R1 will generate its movement trajectory TR within the visual field, and the vehicle will also generate its own movement trajectory TC. The dotted line within Figure 4 is indicated; For the above - mentioned thresholds th1 (variance upper limit) and th2 (similarity lower limit), they are empirical parameters based on actual situations. These parameters can be optimized through simulation tests or historical data analysis to ensure the accuracy and reliability of the moving - state judgment.

[0059] S140. When the vehicle is stationary, determine the parking area. When historical image data containing parking space lines can be obtained, intelligently predict and complete the parking lines that may be blocked or missing by fusing the parking space lines in the analysis result of the current frame with a mathematical model established by historical parking space line data.

[0060] In this embodiment, the parking area refers to the area enclosed by the transverse parking lines and the longitudinal parking lines. Specifically, the parking lines parallel to the road are defined as the longitudinal parking lines; and the parking lines perpendicular to the road are defined as the transverse parking lines.

[0061] In order to determine the parking area within the field of view, this embodiment uses the parking space lines in the image information for recognition. A parking space refers to a specific area surrounded by parking space lines, but in actual scenes, the parking space lines may not be completely obtained due to occlusion or missing. Therefore, the method of filling in the historical parking space lines is used to assist in the recognition of the parking space lines in the current field of view.

[0062] In one embodiment, the above-mentioned step S140 may include steps S141 to S144.

[0063] S141: Determine whether historical image data containing parking space lines can be obtained.

[0064] In this embodiment, an attempt may be made to obtain historical image data with parking space lines. If the acquisition fails, it indicates that the historical image data containing parking space lines cannot be obtained. Otherwise, it is considered that the historical image data containing parking space lines can be obtained.

[0065] S142. If historical image data containing parking space lines can be obtained, parse and obtain the linear equations of the transverse parking space lines and the longitudinal parking space lines in the historical image data, the average spacing distance of the transverse parking space lines, and the average spacing distance of the longitudinal parking space lines to obtain a mathematical model of the historical parking space lines.

[0066] In this embodiment, the historical parking space line information collected by the camera is extracted. If there is available historical parking space line data, it is further analyzed to obtain the linear equations f(x) and g(x) of the horizontal and vertical parking space lines, and the average spacing distances D1 (horizontal) and D2 (vertical) of the two parking space lines are recorded, so as to infer the approximate size of the parking space.

[0067] In this embodiment, the above straight line equation can be expressed as Ax+By+C=0, where A, B, and C are coefficients of the equation, and A and B are not 0 at the same time. When B is not equal to 0, -A / B is the slope, and -C / B is the intercept; when B is equal to 0, the straight line equation is x=-C, the slope is 0, and the straight line is parallel to the x-axis, that is, the horizontal direction.

[0068] S143: using the historical parking space line mathematical model to merge the parking space lines corresponding to the parking spaces in the analysis result to obtain the final parking space lines, and then obtain the parking space area.

[0069] In this embodiment, the above-mentioned mathematical model of historical parking space lines is used to divide the current field of view into parking space lines, and the horizontal and vertical lane line recognition results in the current frame are combined with the historical results, thereby improving the precision and accuracy of the parking space information.

[0070] Specifically, a mathematical model of historical parking lines is established based on the linear equations f(x) and g(x) of the historical parking lines obtained in S142, and the average spacing distances D1 and D2 between the horizontal parking lines and the longitudinal parking lines. The model includes not only static linear equations, but also dynamic characteristics, such as the trend of parking lines changing over time, the impact of vehicles entering and exiting on the position of parking lines, etc.

[0071] Based on the parking lines identified in the current frame, they are matched with the historical parking line model. This step aims to find out which part of the historical model the parking lines that may exist in the current frame correspond to.

[0072] For the parking line that cannot be directly identified from the current frame, the historical parking line model is used for prediction. For example, if the left longitudinal parking line of a parking space is blocked and cannot be detected, but the right longitudinal parking line and other horizontal parking lines can be identified, the approximate position and direction of the left longitudinal parking line can be inferred based on the historical model.

[0073] The parking space lines detected in the current frame are considered the most reliable basis. Therefore, for the detected parts, the parking space lines actually detected are preferred as the final result.

[0074] If some parking lines are not fully detected in the current frame, the detected parking lines will be used as the main ones, supplemented by the results predicted by the historical model to complete these parking lines.

[0075] In extreme cases, when almost no parking lines can be identified in the current frame, or all parking lines are blocked, the historical parking line model can be completely relied upon to reconstruct the parking lines to ensure that the parking areas can still be correctly identified.

[0076] Errors may occur during the fusion process, especially when the historical parking line model is different from the current environment. In order to solve this problem, it is necessary to have a certain degree of adaptive ability to automatically adjust the position and shape of the parking line within a certain range to better adapt to the actual situation.

[0077] Changes over time, such as slight displacements of parking space lines caused by factors such as ground subsidence, should also be considered, and the historical parking space line model should be continuously updated to maintain its timeliness and accuracy.

[0078] After the above fusion processing, a more accurate and complete set of parking space lines is obtained, which not only reflects the actual situation of the current frame, but also inherits the advantages of historical data, so that the specific area of ​​​​each parking space can be clearly defined.

[0079] Specifically, see Figure 5 ,In the subsequent frames, the image is divided into two parts: the lower part shows the actual ,parking space position in the current frame, while the upper part is used to show the ,prediction of the parking space position in the subsequent frames.,This design aims to improve the accuracy and reliability of parking space ,recognition by combining the actual data of the current frame and the ,prediction of future frames.

[0080] For transverse parking lines, since the intervals between them are relatively fixed and transverse parking lines appear within the same interval length, it can be expected that the positions of these transverse parking lines will remain relatively stable in subsequent frames. This means that the positions of transverse parking lines in subsequent frames can be predicted with high accuracy based on the data of the current frame.

[0081] However, the situation of the longitudinal parking space line is different. Because the vehicle may deviate left or right during parking, the position of the longitudinal parking space line on the image is not fixed. To meet this challenge, the linear expression Ax+By+C=0 is used to describe the characteristics of the longitudinal parking space. Considering the possible left and right deviation of the vehicle, the longitudinal parking space line in the subsequent frames has a certain angle deviation range. Specifically, Figure 5 All straight lines between the marks 1 and 2 can be regarded as potential longitudinal parking lines, that is, any straight line within this range can be used as a valid longitudinal parking line.

[0082] In subsequent frames, the image processing algorithm continues to be run to identify parking space lines and combine the actual detected results with the prediction model: When the parking space line is successfully detected in the subsequent frame: at this time, the actually detected parking space line should be preferentially adopted as the final parking space line.

[0083] When the detected parking space lines are incomplete: In this case, the actually detected parking space lines should be used as the main method, and the predicted parking space lines should be used as a supplement to ensure the integrity of the parking area.

[0084] When no parking lines are detected in subsequent frames: the system will record the time point t1 when the parking lines were not detected for the first time. If the parking lines are still not detected within a certain period of time (set time interval) after that, it can be judged that the parking lines have indeed disappeared. However, if the parking lines reappear within this set time interval, it can be considered that the parking lines are only temporarily missing, and the missing part can still be supplemented by the previous prediction results.

[0085] In summary, this method not only takes into account the possibility of parking space lines changing over time, but also provides a flexible and reliable mechanism to deal with the uncertainty in the parking space line recognition process, thereby improving the robustness and accuracy of the overall system. This method can effectively cope with various changes in the vehicle parking process while ensuring the accuracy of parking space recognition.

[0086] S144: If the historical image data containing the parking space lines cannot be obtained, the interval distances and straight line equations corresponding to the horizontal parking space lines and the vertical parking space lines of the parking spaces in the analysis results are recorded to further determine the parking space area.

[0087] In this embodiment, if there is no historical parking space information or the historical information is insufficient for parking space line recognition of the current frame, the parking space line recognition is directly performed on the current frame. Once the parking space line is successfully recognized, the average spacing distance between the two parking space lines and the corresponding straight line equation are recorded.

[0088] In this embodiment, the average interval between the two parking space lines and the corresponding straight line equation can be used to infer the length and width of the parking space area.

[0089] S150. Determine whether the vehicle is illegally parked based on the intersection ratio between the parking area and the area occupied by the vehicle, and when the vehicle is illegally parked, determine the type of illegal parking of the vehicle in combination with the parking signs in the analysis result.

[0090] In this embodiment, the types of illegal parking of the vehicle include illegal parking with fault, illegal parking without fault, illegal parking in the wrong direction, illegal parking beyond the longitudinal boundary and illegal parking beyond the lateral boundary.

[0091] The intersection ratio between the parking area and the area occupied by the vehicle (i.e., the in-target ratio, IOT) is used to determine whether the vehicle has parked illegally. After confirming the illegal parking, the specific type of illegal parking is further determined in combination with the parking signs in the analysis results. The types of illegal parking include but are not limited to illegal parking with faults, illegal parking without faults, illegal parking in the opposite direction, illegal parking beyond the longitudinal boundary, and illegal parking beyond the lateral boundary.

[0092] Specifically, the definition of illegal parking types is based on a comprehensive judgment of factors such as the vehicle's parking location, direction, and whether there are specific landmarks.

[0093] See also Figures 7 to 12 , presenting a schematic diagram of normal parking and various types of illegal parking.

[0094] Faulty parking: When the vehicle has no intersection with the parking area, that is, the IOT (inter-target ratio) between the vehicle and the parking area is equal to 0, and there is a parking sign such as a tripod behind the vehicle, the vehicle is considered to be "faulty parking". This usually means that the vehicle is forced to park in a non-parking space due to a malfunction or other emergency.

[0095] No fault parking: Similarly, if the vehicle has no intersection with the parking area (IOT=0), but there is no sign behind the vehicle, it is considered "No fault parking". This means that the vehicle is parked illegally in a non-parking area without a valid reason.

[0096] Reverse parking: When a vehicle is partially or completely in the parking area (IOT>0), but the direction of the vehicle head is opposite to the prescribed driving direction, this parking method is defined as "reverse parking". This not only affects the normal entry and exit of other vehicles, but may also pose a safety hazard.

[0097] Longitudinal cross-boundary parking: If a vehicle is parked in a parking space with the front of the vehicle in the correct direction, but it is parked too far forward or too far backward, thus encroaching on the space of an adjacent parking space and affecting the normal parking of other vehicles, this behavior is considered "longitudinal cross-boundary parking".

[0098] Illegal parking beyond the boundary: When a vehicle is parked too close to one side of the lane, resulting in the occupation of road space and possibly obstructing the passage of pedestrians or other vehicles, this type of illegal parking is called "illegal parking beyond the boundary". It also poses a traffic safety hazard.

[0099] In one embodiment, the above-mentioned step S150 may include steps S151 to S156.

[0100] S151. Calculate the intersection ratio between the parking space area and the area occupied by the vehicle.

[0101] In this example, see Figure 6 First, we need to calculate the intersection ratio between the parking area Rp and the area occupied by the vehicle Rc. Specifically, we need to calculate the ratio of the intersection area of ​​the two to the total area of ​​the area Rc occupied by the vehicle, that is, IOT=S2 / (S1+S2), where S2 represents the intersection of the target and the restricted area, that is, the intersection of the position of the vehicle and the parking area, and S1 represents the part of the target that exceeds the restricted area, that is, the part of the area where the vehicle is located that exceeds the parking area.

[0102] S152: Determine whether the intersection ratio is greater than a set threshold.

[0103] In this embodiment, it is evaluated whether the calculated intersection ratio exceeds a preset threshold. This threshold is used to distinguish between normal parking and illegal parking. If the intersection ratio is not greater than the threshold, step S156 is executed; otherwise, the next step is entered.

[0104] S153: If the intersection ratio is greater than a set threshold, it is determined that the vehicle is illegally parked.

[0105] In this embodiment, when the intersection ratio is greater than the set threshold, it means that the vehicle is parked in violation of regulations and there is illegal parking. At this time, further analysis is required to determine the specific type of illegal parking. In this embodiment, the threshold is set to 0.

[0106] S154, calculating the distance between the center point of the vehicle and each of the parking areas, and determining the parking area closest to the center point of the vehicle as the parking space corresponding to the vehicle, so as to obtain the illegal parking position.

[0107] In this embodiment, for the case that has been determined to be illegal parking, the distance (dis) from the center point of the vehicle to each parking area is calculated, and the parking area closest to the center point of the vehicle is regarded as the parking space corresponding to the vehicle. This step helps to accurately identify the specific location of illegal parking.

[0108] S155. Determine the front direction of the vehicle, and determine the illegal parking type of the vehicle in combination with the parking sign in the analysis result.

[0109] In this embodiment, once the illegal parking position is determined, it is necessary to determine the vehicle's front direction and check whether there is a parking sign such as a tripod behind the vehicle. The vehicle's front direction can be obtained from the initial parsing process, that is, the parsing result also includes the vehicle's front direction.

[0110] If the vehicle does not intersect with the parking area and there is a parking sign behind the vehicle, it will be considered as "illegal parking with fault"; When the vehicle and the parking area do not have any intersection, that is, the IOT value of the vehicle and the parking area is equal to 0, and there is no sign behind the vehicle, it is judged as "no fault illegal parking"; If the vehicle and the parking area intersect, that is, the IOT value of the vehicle and the parking area is greater than 0, but the vehicle's front direction is opposite to the driving direction, it is judged as "reverse illegal parking"; If the vehicle and the parking area intersect, that is, the IOT value of the vehicle and the parking area is greater than 0, and the front of the vehicle is in the direction of travel, but the vehicle is parked too far forward or too far back in the parking space, encroaching on the space of other parking spaces, it is judged as "longitudinal cross-border illegal parking"; When the vehicle and the parking area intersect, that is, the IOT value of the vehicle and the parking area is greater than 0, and the front of the vehicle is in the driving direction in the parking space, but the vehicle is parked too close to the lane area in the parking space, encroaching on the road space and causing a safety hazard, it is judged as "lateral cross-border illegal parking."

[0111] S156: If the intersection ratio is not greater than a set threshold, it is determined that the vehicle is not illegally parked.

[0112] S160: Combining the illegal parking type, the vehicle license plate number and the illegal parking location of the vehicle to form illegal parking information, and sending the illegal parking information to the terminal.

[0113] In this embodiment, a high-definition camera combined with OCR (optical character recognition) technology can be used to automatically read and record the vehicle's license plate number. For situations that are difficult to identify through automated means, manual assistance can be used for confirmation. With the help of GPS, GIS (geographic information system) or other positioning services, the specific location coordinates of the vehicle are accurately located, and its positional relationship relative to reference objects such as roads and buildings is described. In addition, the specific location of the illegal parking can also be marked on an electronic map.

[0114] After the terminal receives the illegal parking information, it should take corresponding measures according to the actual situation, such as notifying the owner to move the vehicle, issuing a ticket, providing guidance on nearby legal parking spaces, etc. At the same time, it should keep an eye on the status of the illegally parked vehicle until the problem is resolved.

[0115] The method of this embodiment processes video data through edge computing devices, has high intelligence and accuracy, and reduces manual intervention. Refined illegal parking classification is adopted to ensure comprehensive identification of various types of illegally parked vehicles. The method supports full-scene and multi-angle identification without pre-defining the identification area, and can efficiently handle various situations on the road.

[0116] The above-mentioned vehicle illegal parking detection method based on dynamic edge computing collects and preprocesses video or image data through mobile edge devices to ensure high-quality input information and reduce the problems of missed shots and wrong shots. The full-scene recognition algorithm is used to accurately identify vehicles, parking spaces and parking signs to improve recognition accuracy. The static state detection and tracking algorithm is used to accurately judge the illegal parking situation, and the intersection ratio of the parking space area and the vehicle position is combined to accurately determine the illegal parking behavior. Finally, the illegal parking type is further classified based on the parking signs, and the illegal parking information is sent to the terminal in real time to achieve accurate monitoring. It can not only overcome the problems of missed shots, wrong shots and insufficient recognition accuracy in the existing technology, but also accurately classify and monitor different types of illegal parking behaviors in real time.

[0117] Fig.13 is a schematic block diagram of a vehicle parking violation detection system 300 based on dynamic edge computing provided by an embodiment of the present invention. Fig.13 As shown, corresponding to the above vehicle illegal parking detection method based on dynamic edge computing, the present invention also provides a vehicle illegal parking detection system 300 based on dynamic edge computing. The vehicle illegal parking detection system 300 based on dynamic edge computing includes a unit for executing the above vehicle illegal parking detection method based on dynamic edge computing, and the system can be configured in a server. Specifically, please refer to Fig.13 The vehicle illegal parking detection system 300 based on dynamic edge computing includes an acquisition unit 301, a full-scene recognition unit 302, a tracking unit 303, a parking area determination unit 304, a illegal parking type determination unit 305 and an information generation unit 306.

[0118] The acquisition unit 301 is used to acquire video information or image information collected by the mobile edge device and perform preprocessing to obtain preprocessed image data; the full scene recognition unit 302 is used to perform full scene recognition on the preprocessed image data to identify the panoramic information on the road to obtain a parsing result, wherein the parsing result includes a vehicle, a parking space, and a parking sign; the tracking unit 303 is used to track the vehicle in the parsing result to determine whether the vehicle is in a stationary state; the parking area determination unit 304 is used to determine the parking area when the vehicle is in a stationary state, wherein the parking area can be obtained. When the historical image data of parking space lines are obtained, a mathematical model is established by fusing the parking space lines in the parsing result of the current frame with the historical parking space line data to intelligently predict and complete the parking space lines that may be blocked or missing; a parking violation type determination unit 305 is used to determine whether the vehicle is illegally parked according to the intersection ratio of the parking space area and the area occupied by the vehicle, and when the vehicle is illegally parked, the illegal parking type of the vehicle is determined in combination with the parking signs in the parsing result; an information generation unit 306 is used to combine the illegal parking type of the vehicle, the vehicle license plate number and the illegal parking position to form illegal parking information, and send the illegal parking information to the terminal.

[0119] In one embodiment, the acquisition unit 301 includes: The information acquisition subunit is used to acquire video information or image information collected by the mobile edge device; the preprocessing subunit is used to perform camera calibration and image correction on the video information or the image information to obtain preprocessed image data.

[0120] In one embodiment, the full scene recognition unit 302 is used to perform full scene recognition on the preprocessed image data through a target segmentation algorithm to recognize panoramic information on the road to obtain a parsing result.

[0121] In one embodiment, the tracking unit 303 includes: a multi-target recognition and tracking subunit, for identifying and continuously tracking the vehicle in the analysis result by using a multi-target tracking algorithm, so as to obtain the field of view trajectory of the vehicle; a reference point determination subunit, for dividing the preprocessed image data into upper and lower areas, and randomly selecting an area from the upper area as a stationary reference point, and determining the motion trajectory formed by the stationary reference point during the tracking process, so as to obtain the field of view trajectory of the stationary reference point; a distance array formation subunit, for calculating the relative distance between the position of the vehicle in the analysis result and the stationary reference point during the tracking process, so as to obtain a series of distance data arrays that change with time; a similarity calculation subunit, for calculating the similarity between the field of view trajectory of the stationary reference point and the field of view trajectory of the vehicle; a variance calculation subunit, for calculating the variance of the distance data array; a determination subunit, for determining whether the vehicle is in a stationary state according to the similarity and the variance.

[0122] In one embodiment, the determination subunit is used to determine that the vehicle is in a stationary state when the similarity is not less than a preset threshold and the variance is not greater than another preset threshold; when the similarity is less than a preset threshold and / or the variance is greater than another preset threshold, determine that the vehicle is not in a stationary state.

[0123] In one embodiment, the parking area determination unit 304 includes: The judging subunit is used to judge whether the historical image data containing the parking space lines can be obtained; the parsing subunit is used to parse and obtain the linear equations of the horizontal parking space lines and the longitudinal parking space lines in the historical image data, the average spacing distance of the horizontal parking space lines, and the average spacing distance of the longitudinal parking space lines if the historical image data containing the parking space lines can be obtained, so as to obtain the mathematical model of the historical parking space lines; the final parking space line determination subunit is used to fuse the parking space lines corresponding to the parking spaces in the parsing results using the mathematical model of the historical parking space lines to obtain the final parking space lines, and then obtain the parking space area. The recording subunit is used to record the spacing distances and linear equations corresponding to the horizontal parking space lines and the longitudinal parking space lines of the parking spaces in the parsing results, and then determine the parking space area if the historical image data containing the parking space lines cannot be obtained.

[0124] In one embodiment, the parking violation type determination unit 305 includes: An intersection ratio calculation subunit is used to calculate the intersection ratio between the parking area and the area occupied by the vehicle; a ratio judgment subunit is used to judge whether the intersection ratio is greater than a set threshold; an illegal parking determination subunit is used to determine that the vehicle is illegally parked if the intersection ratio is greater than the set threshold; a position determination subunit is used to calculate the distance between the center point of the vehicle and each of the parking areas, and determine the parking area closest to the center point of the vehicle as the parking space corresponding to the vehicle to obtain the illegal parking position; a type determination subunit is used to determine the front direction of the vehicle, and determine the illegal parking type of the vehicle in combination with the parking sign in the analysis result.

[0125] It should be noted that technical personnel in the relevant field can clearly understand that the specific implementation process of the above-mentioned vehicle illegal parking detection system 300 based on dynamic edge computing and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and conciseness of the description, it will not be repeated here.

[0126] The above-mentioned vehicle parking violation detection system 300 based on dynamic edge computing can be implemented in the form of a computer program. The computer program can be used in Fig.14 Runs on the computer device shown.

[0127] See also Fig.14 , Fig.14 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0128] See also Fig.14 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0129] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute a vehicle illegal parking detection method based on dynamic edge computing.

[0130] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0131] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a vehicle illegal parking detection method based on dynamic edge computing.

[0132] The network interface 505 is used to communicate with other devices over the network. Fig.14 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0133] The processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps: The video information or image information collected by the mobile edge device is obtained, and preprocessed to obtain preprocessed image data; the preprocessed image data is subjected to full scene recognition to identify panoramic information on the road to obtain a parsing result, wherein the parsing result includes a vehicle, a parking space, and a parking sign; the vehicle in the parsing result is tracked to determine whether the vehicle is in a stationary state; when the vehicle is in a stationary state, the parking area is determined, wherein when historical image data containing parking space lines can be obtained, a mathematical model established by fusing the parking space lines of the parking space in the parsing result of the current frame with the historical parking space line data is used to intelligently predict and complete the parking space lines that may be blocked or missing; whether the vehicle is illegally parked is determined according to the intersection ratio of the parking space area and the area occupied by the vehicle, and when the vehicle is illegally parked, the type of illegal parking of the vehicle is determined in combination with the parking sign in the parsing result; the illegal parking type of the vehicle, the vehicle license plate number, and the illegal parking position are combined to form illegal parking information, and the illegal parking information is sent to the terminal.

[0134] Among them, the types of illegal parking of the vehicle include illegal parking with fault, illegal parking without fault, illegal parking in the opposite direction, illegal parking beyond the longitudinal boundary and illegal parking beyond the lateral boundary.

[0135] In one embodiment, when the processor 502 implements the step of acquiring video information or image information collected by the mobile edge device and performing preprocessing to obtain preprocessed image data, the processor 502 specifically implements the following steps: Acquire video information or image information collected by a mobile edge device; perform camera calibration and image correction on the video information or the image information to obtain preprocessed image data.

[0136] In one embodiment, when the processor 502 implements the step of performing full scene recognition on the pre-processed image data to identify panoramic information on the road to obtain a parsing result, the processor 502 specifically implements the following steps: The pre-processed image data is subjected to full scene recognition through a target segmentation algorithm to identify panoramic information on the road to obtain an analysis result.

[0137] In one embodiment, when the processor 502 implements the step of tracking the vehicle of the analysis result to determine whether the vehicle is in a stationary state, the processor 502 specifically implements the following steps: A multi-target tracking algorithm is used to identify and continuously track the vehicle in the analysis result to obtain the field of view trajectory of the vehicle; the pre-processed image data is divided into upper and lower areas, and an area is randomly selected from the upper area as a stationary reference point, and the motion trajectory formed by the stationary reference point during the tracking process is determined to obtain the field of view trajectory of the stationary reference point; during the tracking process, the relative distance between the position of the vehicle in the analysis result and the stationary reference point is calculated to obtain a series of distance data arrays that change with time; the similarity between the field of view trajectory of the stationary reference point and the field of view trajectory of the vehicle is calculated; the variance of the distance data array is calculated; and whether the vehicle is in a stationary state is determined according to the similarity and the variance.

[0138] In one embodiment, when the processor 502 implements the step of determining whether the vehicle is in a stationary state according to the similarity and the variance, the processor 502 specifically implements the following steps: When the similarity is not less than a preset threshold and the variance is not greater than another preset threshold, it is determined that the vehicle is in a stationary state; when the similarity is less than a preset threshold and / or the variance is greater than another preset threshold, it is determined that the vehicle is not in a stationary state.

[0139] In one embodiment, the processor 502 implements the step of determining the parking space area when the vehicle is in a stationary state, wherein, when historical image data containing parking space lines can be obtained, intelligently predicting and completing the parking space lines that may be blocked or missing by fusing the parking space lines in the analysis result of the current frame with a mathematical model established by the historical parking space line data, specifically implementing the following steps: Determine whether historical image data containing parking lines can be obtained; if historical image data containing parking lines can be obtained, parse and obtain the linear equations of the transverse parking lines and the longitudinal parking lines in the historical image data, the average spacing distance of the transverse parking lines, and the average spacing distance of the longitudinal parking lines to obtain a mathematical model of historical parking lines; use the mathematical model of historical parking lines to fuse the parking lines corresponding to the parking spaces in the analysis results to obtain the final parking lines, and then obtain the parking area.

[0140] In one embodiment, after implementing the step of determining whether historical image data containing parking space lines can be obtained, the processor 502 further implements the following steps: If the historical image data containing the parking space lines cannot be obtained, the interval distances and straight line equations corresponding to the transverse parking space lines and the longitudinal parking space lines of the parking spaces in the analysis results are recorded to further determine the parking space area.

[0141] In one embodiment, the processor 502 implements the steps of determining whether the vehicle is illegally parked according to the intersection ratio between the parking space area and the area occupied by the vehicle, and determining the illegal parking type of the vehicle in combination with the parking sign in the analysis result when the vehicle is illegally parked, specifically implementing the following steps: Calculate the intersection ratio between the parking area and the area occupied by the vehicle; determine whether the intersection ratio is greater than a set threshold; if the intersection ratio is greater than the set threshold, determine that the vehicle is illegally parked; calculate the distance between the center point of the vehicle and each of the parking areas, and determine the parking area closest to the center point of the vehicle as the parking space corresponding to the vehicle to obtain the illegal parking position; determine the front direction of the vehicle, and determine the type of illegal parking of the vehicle in combination with the parking signs in the analysis results.

[0142] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0143] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0144] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps: The video information or image information collected by the mobile edge device is obtained, and preprocessed to obtain preprocessed image data; the preprocessed image data is subjected to full scene recognition to identify panoramic information on the road to obtain a parsing result, wherein the parsing result includes a vehicle, a parking space, and a parking sign; the vehicle in the parsing result is tracked to determine whether the vehicle is in a stationary state; when the vehicle is in a stationary state, the parking area is determined, wherein when historical image data containing parking space lines can be obtained, a mathematical model established by fusing the parking space lines of the parking space in the parsing result of the current frame with the historical parking space line data is used to intelligently predict and complete the parking space lines that may be blocked or missing; whether the vehicle is illegally parked is determined according to the intersection ratio of the parking space area and the area occupied by the vehicle, and when the vehicle is illegally parked, the type of illegal parking of the vehicle is determined in combination with the parking sign in the parsing result; the illegal parking type of the vehicle, the vehicle license plate number, and the illegal parking position are combined to form illegal parking information, and the illegal parking information is sent to the terminal.

[0145] Among them, the types of illegal parking of the vehicle include illegal parking with fault, illegal parking without fault, illegal parking in the opposite direction, illegal parking beyond the longitudinal boundary and illegal parking beyond the lateral boundary.

[0146] In one embodiment, when the processor executes the computer program to implement the step of acquiring video information or image information collected by the mobile edge device and performing preprocessing to obtain preprocessed image data, the processor specifically implements the following steps: Acquire video information or image information collected by a mobile edge device; perform camera calibration and image correction on the video information or the image information to obtain preprocessed image data.

[0147] In one embodiment, when the processor executes the computer program to implement the step of performing full scene recognition on the pre-processed image data to identify panoramic information on the road to obtain a parsing result, the processor specifically implements the following steps: The pre-processed image data is subjected to full scene recognition through a target segmentation algorithm to identify panoramic information on the road to obtain an analysis result.

[0148] In one embodiment, when the processor executes the computer program to implement the step of tracking the vehicle in the analysis result to determine whether the vehicle is in a stationary state, the processor specifically implements the following steps: A multi-target tracking algorithm is used to identify and continuously track the vehicle in the analysis result to obtain the field of view trajectory of the vehicle; the pre-processed image data is divided into upper and lower areas, and an area is randomly selected from the upper area as a stationary reference point, and the motion trajectory formed by the stationary reference point during the tracking process is determined to obtain the field of view trajectory of the stationary reference point; during the tracking process, the relative distance between the position of the vehicle in the analysis result and the stationary reference point is calculated to obtain a series of distance data arrays that change with time; the similarity between the field of view trajectory of the stationary reference point and the field of view trajectory of the vehicle is calculated; the variance of the distance data array is calculated; and whether the vehicle is in a stationary state is determined according to the similarity and the variance.

[0149] In one embodiment, when the processor executes the computer program to implement the step of determining whether the vehicle is in a stationary state according to the similarity and the variance, the processor specifically implements the following steps: When the similarity is not less than a preset threshold and the variance is not greater than another preset threshold, it is determined that the vehicle is in a stationary state; when the similarity is less than a preset threshold and / or the variance is greater than another preset threshold, it is determined that the vehicle is not in a stationary state.

[0150] In one embodiment, the processor implements the step of determining a parking space area when the vehicle is stationary when executing the computer program, wherein when historical image data containing parking space lines can be obtained, intelligently predicting and completing the parking space lines that may be blocked or missing by fusing the parking space lines in the analysis result of the current frame with a mathematical model established by the historical parking space line data, specifically implementing the following steps: Determine whether historical image data containing parking lines can be obtained; if historical image data containing parking lines can be obtained, parse and obtain the linear equations of the transverse parking lines and the longitudinal parking lines in the historical image data, the average spacing distance of the transverse parking lines, and the average spacing distance of the longitudinal parking lines to obtain a mathematical model of historical parking lines; use the mathematical model of historical parking lines to fuse the parking lines corresponding to the parking spaces in the analysis results to obtain the final parking lines, and then obtain the parking area.

[0151] In one embodiment, after executing the computer program to implement the step of determining whether historical image data containing parking space lines can be acquired, the processor further implements the following steps: If the historical image data containing the parking space lines cannot be obtained, the interval distances and straight line equations corresponding to the transverse parking space lines and the longitudinal parking space lines of the parking spaces in the analysis results are recorded to further determine the parking space area.

[0152] In one embodiment, when the processor executes the computer program to implement the step of determining whether the vehicle is illegally parked according to the intersection ratio between the parking space area and the area occupied by the vehicle, and when the vehicle is illegally parked, determining the type of illegal parking of the vehicle in combination with the parking sign in the analysis result, the processor specifically implements the following steps: Calculate the intersection ratio between the parking area and the area occupied by the vehicle; determine whether the intersection ratio is greater than a set threshold; if the intersection ratio is greater than the set threshold, determine that the vehicle is illegally parked; calculate the distance between the center point of the vehicle and each of the parking areas, and determine the parking area closest to the center point of the vehicle as the parking space corresponding to the vehicle to obtain the illegal parking position; determine the front direction of the vehicle, and determine the type of illegal parking of the vehicle in combination with the parking signs in the analysis results.

[0153] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0154] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0155] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0156] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the system of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0157] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0158] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A vehicle illegal parking detection method based on dynamic edge computing, characterized in that: include: Acquire video information or image information collected by the mobile edge device, and perform preprocessing to obtain preprocessed image data; Performing full scene recognition on the preprocessed image data to identify panoramic information on the road to obtain an analysis result, wherein the analysis result includes vehicles, parking spaces, and parking signs; Tracking the vehicle of the analysis result to determine whether the vehicle is in a stationary state; When the vehicle is stationary, the parking area is determined, wherein, when historical image data containing parking space lines can be obtained, the parking space lines in the analysis result of the current frame are integrated with the mathematical model established by the historical parking space line data to intelligently predict and complete the parking space lines that may be blocked or missing; Determine whether the vehicle is illegally parked according to the intersection ratio between the parking space area and the area occupied by the vehicle, and when the vehicle is illegally parked, determine the type of illegal parking of the vehicle in combination with the parking sign in the analysis result; The illegal parking type, vehicle license plate number and illegal parking location of the vehicle are combined to form illegal parking information, and the illegal parking information is sent to the terminal.

2. The vehicle parking violation detection method based on dynamic edge computing according to claim 1 is characterized in that: The acquiring of video information or image information collected by the mobile edge device and preprocessing to obtain preprocessed image data includes: Obtaining video information or image information collected by a mobile edge device; Camera calibration and image correction are performed on the video information or the image information to obtain preprocessed image data.

3. The vehicle illegal parking detection method based on dynamic edge computing according to claim 1 is characterized in that: The performing full scene recognition on the pre-processed image data to identify panoramic information on the road to obtain a parsing result includes: The pre-processed image data is subjected to full scene recognition through a target segmentation algorithm to identify panoramic information on the road to obtain an analysis result.

4. The vehicle parking violation detection method based on dynamic edge computing according to claim 1 is characterized in that: Tracking the vehicle of the analysis result to determine whether the vehicle is in a stationary state includes: Using a multi-target tracking algorithm to identify and continuously track the vehicle in the analysis result to obtain the field of view trajectory of the vehicle; Dividing the preprocessed image data into upper and lower regions, randomly selecting an area from the upper region as a stationary reference point, and determining a motion trajectory formed by the stationary reference point during the tracking process to obtain a visual field trajectory of the stationary reference point; During the tracking process, the relative distance between the position of the vehicle in the analysis result and the stationary reference point is calculated to obtain a series of distance data arrays that vary with time; Calculating the similarity between the visual field trajectory of the stationary reference point and the visual field trajectory of the vehicle; Calculating the variance of the distance data array; It is determined whether the vehicle is in a stationary state according to the similarity and the variance.

5. The vehicle parking violation detection method based on dynamic edge computing according to claim 4 is characterized in that: The determining whether the vehicle is in a stationary state according to the similarity and the variance includes: When the similarity is not less than a preset threshold and the variance is not greater than another preset threshold, it is determined that the vehicle is in a stationary state; when the similarity is less than a preset threshold and / or the variance is greater than another preset threshold, it is determined that the vehicle is not in a stationary state.

6. The vehicle parking violation detection method based on dynamic edge computing according to claim 1 is characterized in that: When the vehicle is stationary, the parking area is determined. When historical image data containing parking space lines can be obtained, a mathematical model established by fusing the parking space lines in the analysis result of the current frame with the historical parking space line data is used to intelligently predict and complete the parking space lines that may be blocked or missing, including: Determine whether historical image data containing parking space lines can be obtained; If historical image data containing parking space lines can be obtained, then the linear equations of the transverse parking space lines and the longitudinal parking space lines in the historical image data, the average spacing distance of the transverse parking space lines, and the average spacing distance of the longitudinal parking space lines are parsed and obtained to obtain a mathematical model of the historical parking space lines; The parking space lines corresponding to the parking spaces in the analysis results are merged using the historical parking space line mathematical model to obtain the final parking space lines, and then obtain the parking space area.

7. The vehicle parking violation detection method based on dynamic edge computing according to claim 6 is characterized in that: After determining whether the historical image data containing the parking space line can be obtained, the method further includes: If the historical image data containing the parking space lines cannot be obtained, the interval distances and straight line equations corresponding to the transverse parking space lines and the longitudinal parking space lines of the parking spaces in the analysis results are recorded to further determine the parking space area.

8. The vehicle illegal parking detection method based on dynamic edge computing according to claim 1 is characterized in that: The determining whether the vehicle is illegally parked according to the intersection ratio between the parking space area and the area occupied by the vehicle, and when the vehicle is illegally parked, determining the type of illegal parking of the vehicle in combination with the parking sign in the analysis result, includes: Calculating the intersection ratio between the parking space area and the area occupied by the vehicle; Determining whether the intersection ratio is greater than a set threshold; If the intersection ratio is greater than a set threshold, it is determined that the vehicle is illegally parked; Calculating the distance between the center point of the vehicle and each of the parking areas, and determining the parking area closest to the center point of the vehicle as the parking space corresponding to the vehicle, so as to obtain the illegal parking position; The front direction of the vehicle is determined, and the illegal parking type of the vehicle is determined in combination with the parking sign in the analysis result.

9. The vehicle parking violation detection method based on dynamic edge computing according to claim 8 is characterized in that: The types of illegal parking of the vehicle include illegal parking with fault, illegal parking without fault, illegal parking in the opposite direction, illegal parking beyond the longitudinal boundary and illegal parking beyond the lateral boundary.

10. The vehicle illegal parking detection system based on dynamic edge computing is characterized by: include: An acquisition unit, used to acquire video information or image information collected by the mobile edge device and perform preprocessing to obtain preprocessed image data; A full scene recognition unit, used to perform full scene recognition on the preprocessed image data to identify panoramic information on the road to obtain a parsing result, wherein the parsing result includes vehicles, parking spaces, and parking signs; A tracking unit, used for tracking the vehicle of the analysis result to determine whether the vehicle is in a stationary state; A parking area determination unit, configured to determine a parking area when the vehicle is stationary, wherein when historical image data containing parking space lines can be obtained, a mathematical model established by fusing the parking space lines of the parking spaces in the analysis result of the current frame with the historical parking space line data is used to intelligently predict and complete the parking space lines that may be blocked or missing; a parking violation type determination unit, configured to determine whether the vehicle is illegally parked according to the intersection ratio between the parking space area and the area occupied by the vehicle, and when the vehicle is illegally parked, determine the parking violation type of the vehicle in combination with the parking sign in the analysis result; The information generating unit is used to combine the illegal parking type, the vehicle license plate number and the illegal parking location of the vehicle to form illegal parking information, and send the illegal parking information to the terminal.

Images