A method for detecting parking line crossing in a service area based on high-position video

Abstract

The present invention proposes a method for detecting parking line crossing in service areas based on high-position monitoring videos, aiming at the possible parking line crossing phenomena in service areas, assisting staff in supervising parking spaces and improving work efficiency. First, using the object detection algorithm, search for and frame the parking vehicle targets in the video frames, and then count the edge contour features of the vehicle targets to judge the vehicle orientation; then, according to the coordinate transformation relationship obtained by camera calibration and the vehicle orientation, deduce and fit the geodetic coordinates of the vehicle, so as to obtain the size information of the vehicle. Finally, based on the size information of the vehicle, locate the vehicle, combine with the prior calibrated parking space information, calculate the coincidence degree between the vehicle and the parking space, and give the possibility of the vehicle parking across the line. This method is implemented using ordinary monitoring cameras, which is friendly to both new construction and renovation projects. The accuracy and real-time performance of each module are relatively high, and it has strong engineering practicability.

Description

Technical Field

[0001] The present invention relates to the technology for detecting parking line crossing in service areas, and particularly to a method for detecting parking line crossing in service areas based on high-position video. Background Art

[0002] High-speed service areas are one of the important facilities in cities. With the increase in the number of vehicle users, the number of high-speed service areas and the number of parking spaces they set up are gradually increasing. The workload and supervision difficulty of the management side also increase accordingly. Irregular parking phenomena such as parking across the line and their solutions have attracted much attention from the management side.

[0003] People's parking thinking is often from easy to difficult. When there are many idle parking spaces, parking is relatively random and the parking difficulty is relatively low. However, as the parking spaces are gradually occupied, the number of available parking spaces for later vehicles becomes fewer and fewer, and the buffer zones available for them to park also become fewer and fewer. In order to keep a certain distance from the vehicle in front and avoid rubbing, later vehicles will choose to drive into the parking space at a greater safe distance from the vehicle in front. By analogy, once the vehicle in front parks across the line, especially when crossing the line left and right, the possibility of the later vehicle parking across the line will increase greatly. In this vicious cycle, on the one hand, it will lead to more vehicles parking across the line, increasing the parking difficulty and greatly increasing the possibility of rubbing incidents. On the other hand, the vehicle in front occupying the parking space of the later vehicle will also cause a certain degree of waste of parking spaces to some extent. The parking line crossing detection function can immediately inform the relevant supervision personnel for on-site scheduling when detecting the situation of parking across the line, which helps to improve the parking management level of the service area, ensure safety and avoid risks.

[0004] For open-air parking lots, parking line crossing detection is usually achieved by means of two types of hardware: (1) Installing ranging sensors on the parking spaces and judging whether the vehicle crosses the line according to the sensed distance. However, for service areas with dozens to hundreds of parking spaces, extensive use of sensors is very inconvenient for construction and later maintenance; (2) Relying on surveillance cameras for management is a currently widely used method. It uses target detection technology to search for targets in real time, analyzes the matching relationship between the vehicle and the parking space, and then judges the situation of the vehicle crossing the line.

[0005] The video-based method can be further divided into two categories: laser camera type and ordinary surveillance camera type. The former is more likely to obtain the scene depth of the target, which is beneficial for positioning and describing the shape of the object. The disadvantage is that it is expensive. For ordinary surveillance cameras, more feature extraction work needs to be done on the video to obtain information such as the vehicle's attitude and world distance for judging parking line crossing. However, its cost is much lower than the former, and more existing projects and newly built projects tend to use this type of surveillance camera. Therefore, researching the method for detecting line crossing based on ordinary surveillance cameras has certain engineering significance. Summary of the Invention

[0006] The object of the present invention is to provide a method for detecting parking line pressing in a service area based on high-position video.

[0007] The technical solution for achieving the object of the present invention is: a method for detecting parking line pressing in a service area based on high-position video, including the following steps:

[0008] Step 1: Calibrate the monitoring camera to obtain the conversion relationship between the image coordinates and the earth coordinates;

[0009] Step 2: Detect vehicle targets, read the monitoring video stream, detect parking targets in real time, output the vehicle type category, and at the same time give the image coordinate information of the parking target and the size of the bounding box of the parking target;

[0010] Step 3: Judge the vehicle orientation, select the target vehicle area according to the vehicle bounding box, perform Gaussian filtering, edge detection, and line detection on this area, extract the contour features of the vehicle, and judge the vehicle orientation;

[0011] Step 4: Fit the vehicle size, based on the conversion relationship between the image coordinates and the earth coordinates, deduce the 3D detection box of the vehicle from the 2D detection box obtained in Step 2, and estimate and optimize the vehicle parameters;

[0012] Step 5: Detect the situation of the vehicle pressing the line. After determining the vehicle size, re-determine the earth coordinates of the vehicle, determine the overlapping area between the bottom contour rectangle of the vehicle and the contour of the parking space, and determine the probability that the vehicle is in the state of pressing the parking line according to the line pressing probability function.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0014] (1) The technical solution of the present invention only uses an ordinary monitoring camera to achieve the function of detecting line pressing, and the implementation cost is relatively low;

[0015] (2) The technical solution of the present invention provides a function for assisting in judging whether the vehicle presses the line, reduces the work difficulty of parking management personnel, and improves the parking management level;

[0016] (3) The technical solution of the present invention can standardize the parking methods of drivers and passengers in a timely manner, and ensure parking safety and parking efficiency;

[0017] The following further describes the present invention in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of the steps of the method for detecting parking line pressing in a service area based on high-position video of the present invention.

[0019] Figure 2 It is a flow chart of the steps for judging the vehicle orientation of the present invention.

[0020] Figure 3 Flow chart of steps for vehicle size fitting according to the present invention

[0021] Figure 4 Schematic diagram of camera calibration according to the present invention

[0022] Figure 5 Schematic diagram for derivation of vehicle orientation and earth coordinates according to the present invention

[0023] Figure 6 Schematic diagram of line - crossing probability function according to the present invention

[0024] Figures 7 to 8 Effect diagram of line - crossing detection in the embodiment according to the present invention Detailed implementation manners

[0025] A method for detecting line - crossing of parking in service area based on high - altitude video, comprising the following steps:

[0026] Step 1: Calibrate the monitoring camera to obtain the conversion relationship between image coordinates and earth coordinates. Specifically:

[0027] When there are no or few vehicles in the monitoring area, take samples of the parking space area. The sampled pictures should be able to completely show the shapes of each parking line in the area, including the four vertices of each parking space area. The camera is set on the lamp post, and the posture of the camera is fixed when taking pictures;

[0028] Set the origin of the camera coordinate system as O c , and its coordinate axes are X c , Y c , Z c ; The origin of the earth coordinate system is O w , located at the bottom of the high pole, and its coordinate axes are X w , Y w , Z w ;

[0029] Let the corresponding points of the four points A, B, C, D in the parking space area on the image of the camera imaging plane be a, b, c, d, where O o is the principal point of the imaging plane image, the vanishing point v1 is the intersection point of the line segment da and the line segment cb, and the vanishing point v2 is the intersection point of the line segment cd and the line segment ba. The vanishing point coordinates are respectively: v1(v x1 , v y1 ), v2(v x2 , v y2 ), β is the pitch angle, γ is the angle between the optical axis projection line and the parking line, and 0° ≤ γ < 90°, and the installation height of the camera is h;

[0030] The world coordinates corresponding to the vanishing points are (-tanγ, 1, 0), (1, tanγ, 0), and substitute them into the following formula:

[0031]

[0032] It can be obtained that:

[0033]

[0034] where k is the proportionality coefficient;

[0035] Then the coordinate conversion formula relationship between the geodetic coordinates and the image coordinates (x pi , y pi ) is:

[0036]

[0037] β = tan -1 (-v y1 / f)

[0038] γ = tan -1 (-v x1 cosβ / f).

[0039] Step 2: Detect the vehicle target, read the surveillance video stream, detect the parking target in real time, output the vehicle type category, and at the same time give the image coordinate information of the parking target and the bounding box size of the parking target, specifically:

[0040] Step 2-1: Classify the vehicle target detection model, including cars, SUVs, large and small buses, hazardous chemical vehicles, and extra-long vehicles;

[0041] Select a single-stage target detection method to ensure a frame processing rate in milliseconds; the training set used for model training balances the target quantities of various vehicle types to ensure that the model has a high recognition rate for various vehicle types

[0042] Step 2-2: Input the video collected by the surveillance camera, and use the vehicle target detection model to output the vehicle type category of the parking target in the video frame, the vehicle center coordinates P(x p , y p ), the width w p of the target area, and the height h p .

[0043] Step 3: Determine the vehicle orientation, select the target vehicle area based on the vehicle bounding box, perform Gaussian filtering, edge detection, and line detection on this area, extract the contour features of the vehicle, and based on the image coordinate system, divide it into two vehicle orientations according to whether the slope of the vehicle orientation is greater than 0. The specific steps are as follows:

[0044] Step 3-1: Denoise the original monitoring image to avoid misjudging noise points as pseudo-edges during the post-processing. During the denoising process, the size of the convolution kernel should not be set too large to avoid filtering out useful features;

[0045] Step 3-2: Perform canny edge detection on the target area to extract the edge features of the vehicle target;

[0046] Step 3-3: Screen the results of the edge detection, extract the straight edges as the direction vectors of the vehicle, set thresholds according to the size of the vehicle area, which are respectively used to constrain the length and search step of the straight edges, and remove small and isolated edge segments, finally obtaining the set of direction vectors of the vehicle;

[0047] Step 3-4: Perform a histogram statistics on the direction vectors of the vehicle. Its distribution pattern also shows a relatively significant normal distribution. Since the length of the side contour of the vehicle body is longer than that of the front and rear side contours and can better represent the orientation of the vehicle, the mean value of its normal distribution is considered to be the direction represented by the side of the vehicle body, thereby determining the orientation of the vehicle.

[0048] Step 4: Fit the vehicle size. Based on the conversion relationship between the image coordinates and the earth coordinates, deduce the 3D detection box of the vehicle from the 2D detection box obtained in Step 2, and estimate and optimize the vehicle parameters. Specifically:

[0049] Step 4-1: Use the conversion matrix obtained in Step 1 to convert the coordinates P(x p , y p ) of the 2D detection box to the earth coordinate system W(x w , y w , z w ). Select the corresponding coordinate derivation method according to the vehicle orientation to obtain its 8 spatial coordinate points. The specific coordinate derivation formula is shown in Table 1, where l, j, and h respectively represent the length, width, and height of the vehicle;

[0050] Table 1

[0051]

[0052] Step 4-2: Adopt the LM algorithm, set the initial size values according to the vehicle type, and correct the length, width, and height of the vehicle target through error calculation and regression fitting of a large number of observed values. It is necessary to select an appropriate number of observed values for fitting to balance the accuracy of the estimation and the time of parameter estimation;

[0053] The error objective function used in the fitting process is as follows:

[0054]

[0055] Where is the diagonal distance of the 2D detection box, and ‖q0,q6‖ is the 3D diagonal distance of the vehicle in the earth coordinate system;

[0056] Find the minimum value of the error objective function, gradually correct the length, width, and height of the vehicle target, and reduce the morphological difference between the 2D detection box and the 3D detection box until the error and parameters converge.

[0057] Step 5: Detect the situation of the vehicle pressing the line. After determining the vehicle size, re-determine the earth coordinates of the vehicle, determine the overlapping area between the rectangular bottom contour of the vehicle and the contour of the parking space, and determine the probability that the vehicle is in the state of pressing the line according to the line-pressing probability function, specifically:

[0058] Step 5-1: Determine the intersection area s between the four vertex coordinates of the vehicle bottom obtained in Step 3 and the four vertices of the parking space;

[0059] Step 5-2: Judge whether the vehicle presses the line according to the intersection area s between the vehicle and the parking space. The change relationship between the intersection area s and the line-pressing probability p is:

[0060]

[0061] where s ∈ [0, S], S represents the upper limit value of the occupied area of the corresponding vehicle type, and n is used to control the inner swing amplitude of the function, that is, the range allowing a small error.

[0062] Due to the randomness and unexplainability of neural network prediction, there are inevitable regression errors in the 2D detection box of object detection. Also, due to the lack of sufficient environmental references in camera calibration, it ultimately leads to errors in the 3D detection box. This function can allow certain errors in each pre-processing process, make a more reasonable judgment on the possibility of the vehicle pressing the line, and avoid misjudgment.

[0063] Therefore, when judging whether the line is pressed according to the intersection area s between the vehicle and the parking space, within a certain range where s is small, the possibility of pressing the line increases slowly with the increase of s, but as s increases and exceeds a certain range, the possibility of pressing the line will increase rapidly.

[0064] A service area parking line-pressing detection system based on high-position video includes the following modules:

[0065] Vehicle target detection module: used to detect parking targets in real time, output the vehicle type category, and at the same time give the image coordinate information of the parking target and the size of the bounding box of the parking target;

[0066] Vehicle orientation judgment module: used to extract the contour features of the vehicle and judge the orientation of the vehicle;

[0067] Vehicle size fitting module: used to estimate and optimize vehicle parameters;

[0068] The line - pressing judgment module: used to detect the situation of the vehicle pressing the line.

[0069] A computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0070] Step 1: Calibrate the monitoring camera to obtain the conversion relationship between the image coordinates and the earth coordinates;

[0071] Step 2: Detect the vehicle target, read the monitoring video stream, detect the parking target in real - time, output the vehicle type category, and at the same time give the image coordinate information of the parking target and the size of the bounding box of the parking target;

[0072] Step 3: Judge the vehicle orientation. Select the target vehicle area according to the vehicle bounding box, perform Gaussian filtering, edge detection, and line detection on this area, extract the contour features of the vehicle, and judge the vehicle orientation;

[0073] Step 4: Fit the vehicle size. Based on the conversion relationship between the image coordinates and the earth coordinates, deduce the 3D detection box of the vehicle from the 2D detection box obtained in Step 2, and estimate and optimize the vehicle parameters;

[0074] Step 5: Detect the situation of the vehicle pressing the line. After determining the vehicle size, re - determine the earth coordinates of the vehicle, determine the overlapping area between the bottom - contour rectangle of the vehicle and the contour of the parking space, and determine the probability that the vehicle is in the state of pressing the line according to the line - pressing probability function.

[0075] A computer - storable medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0076] Step 1: Calibrate the monitoring camera to obtain the conversion relationship between the image coordinates and the earth coordinates;

[0077] Step 2: Detect the vehicle target, read the monitoring video stream, detect the parking target in real - time, output the vehicle type category, and at the same time give the image coordinate information of the parking target and the size of the bounding box of the parking target;

[0078] Step 3: Judge the vehicle orientation. Select the target vehicle area according to the vehicle bounding box, perform Gaussian filtering, edge detection, and line detection on this area, extract the contour features of the vehicle, and judge the vehicle orientation;

[0079] Step 4: Fit the vehicle size. Based on the conversion relationship between the image coordinates and the earth coordinates, deduce the 3D detection box of the vehicle from the 2D detection box obtained in Step 2, and estimate and optimize the vehicle parameters;

[0080] Step 5: Detect the situation of the vehicle crossing the line. After determining the vehicle size, re-determine the vehicle's geodetic coordinates, determine the overlapping area between the rectangle of the vehicle bottom contour and the parking space contour, and determine the probability of the vehicle being in a state of crossing the line while parking according to the line-crossing probability function.

[0081] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0082] Embodiment

[0083] In this embodiment, the cameras are installed on the intelligent light poles, and the intelligent light poles are installed in the outdoor parking lot of the service area. Each light pole is equipped with 2-4 cameras according to the coverage range of the layout points. In order to meet the design principles of the intensification of the service area and the lighting range, etc., the monitoring angles of the cameras are mostly in the oblique illumination perspective (the heading angle range is less than 45°). After the cameras are installed, their positions are not changed, and each camera is responsible for a certain fixed parking area.

[0084] Combined with Figure 1 , a method for detecting vehicle line crossing in a service area based on high-position video includes the following steps:

[0085] Step 1: Combine Figure 4 , calibrate the monitoring cameras to obtain the conversion relationship between the image coordinates and the geodetic coordinates, specifically:

[0086] When there are no vehicles or few vehicles in the monitoring area, take samples of the parking space area. The sampled pictures should be able to completely display the shapes of the parking space lines in the area, including the four vertices of each parking space area. The cameras are installed on the light poles, and the posture of the cameras is fixed when taking pictures.

[0087] Set the origin of the camera coordinate system as O c , and its coordinate axes are X c , Y c , Z c ; The origin of the geodetic coordinate system is O w , located at the bottom of the high pole, and its coordinate axes are X w , Y w , Z w ;

[0088] Let the corresponding points of the four points A, B, C, and D in the parking space area on the image of the camera imaging plane be a, b, c, and d, where O o is the principal point of the imaging plane image, the vanishing point v1 is the intersection point of the line segment da and the line segment cb, and the vanishing point v2 is the intersection point of the line segment cd and the line segment ba. The vanishing point coordinates are respectively: v1(v x1 , v y1 ), v2(v x2 , v y2) where β is the pitch angle, γ is the angle between the projection line of the optical axis and the parking space line, and 0° ≤ γ < 90°, and h is the installation height of the camera;

[0089] The world coordinates corresponding to the vanishing point are (-tanγ, 1, 0) and (1, tanγ, 0), substituting them into the following formula:

[0090]

[0091] We can get:

[0092]

[0093] where k is the proportionality coefficient;

[0094] Then the coordinate transformation formula relationship between the earth coordinates and the image coordinates (x pi , y pi ) is:

[0095]

[0096] β = tan -1 (-v y1 / f)

[0097] γ = tan -1 (-v x1 cosβ / f).

[0098] Step 2: Select the YOLOv5 algorithm to detect vehicle targets, read the surveillance video stream, detect parking targets in real time, output the vehicle type category, and at the same time give the image coordinate information of the parking target and the bounding box size of the parking target, specifically:

[0099] Step 2-1: Classify the vehicle target detection model, including cars, SUVs, large and small buses, hazardous chemical vehicles, and extra-long vehicles;

[0100] Select a single-stage target detection method to ensure a frame processing rate of milliseconds; the training set used in model training balances the target numbers of various vehicle types to ensure that the model has a high recognition rate for various vehicle types;

[0101] In this embodiment, in addition to using the public dataset UA-DETRAC, the dataset also uses the surveillance picture images self-organized in the service area scenario;

[0102] The graphics card used in the environment of this embodiment is RTX2070-8G, with a memory of 16G and a video resolution of 1280*720. Under this condition, the single-frame detection rate of YOLOv5 is about 33ms

[0103] Step 2-2: Input the video collected by the monitoring camera, and use the vehicle target detection model to output the vehicle type category of the parking target in the video frame, the vehicle center coordinates P(x p , y p ), the width w p of the target area, and the height h p .

[0104] Step 3: Combine Figure 2 , judge the vehicle orientation, select the target vehicle area according to the vehicle bounding box, perform Gaussian filtering, edge detection, and line detection on this area, extract the contour features of the vehicle, and divide it into two vehicle orientations based on whether the slope of the vehicle orientation is greater than 0 with the image coordinate system as the reference. The specific steps are as follows:

[0105] Step 3-1: Perform noise reduction processing on the original monitoring screen to avoid misjudging noise points as pseudo-edges in the post-processing process. During the noise reduction process, the size of the convolution kernel should not be set too large to avoid filtering out useful features;

[0106] In this embodiment, Gaussian blur is used to filter noise. After repeated experiments, it is more appropriate to set the length and width of the convolution kernel to 3. Before and after filtering, the accuracy of orientation judgment for 1000 targets in the measurement set has increased from 91.6% to 94.4%;

[0107] Step 3-2: Perform canny edge detection on the target area to extract the edge features of the vehicle target. In this embodiment, the upper and lower thresholds of canny edge detection are set to 30 and 100 respectively;

[0108] Step 3-3: Screen the results of edge detection, extract the straight edges as the direction vectors of the vehicle, set thresholds according to the size of the vehicle area, which are used to constrain the length and search step of the straight edges respectively, and remove small and isolated edge segments to finally obtain the direction vector set of the vehicle;

[0109] In this embodiment, the minimum length and search step of the straight line are parameter-optimized and set to 4 and 7 respectively to remove small and isolated edge segments;

[0110] Experiments prove that the direction vector set of the vehicle obtained by this method can improve the accuracy of orientation judgment to 97.2%;

[0111] Step 3-4: Perform histogram statistics on the direction vectors of the vehicle. Its distribution law also shows a relatively significant normal distribution. Since the length of the side contour of the vehicle body is longer than that of the front and rear sides of the vehicle, it can better represent the orientation of the vehicle. Therefore, the mean value of its normal distribution is considered to be the direction represented by the side of the vehicle body, thereby determining the orientation of the vehicle.

[0112] Judging the vehicle orientation according to this method, its accuracy rate exceeds 97%, and the average processing time for a single target is about 3.1 ms. The orientation judging module has high accuracy and real-time performance.

[0113] Step 4: Fit the vehicle size. Based on the conversion relationship between the image coordinates and the earth coordinates, deduce the 3D detection frame of the vehicle from the 2D detection frame obtained in Step 2, estimate and optimize the vehicle parameters. In this embodiment, the length and width of the parking space are 3 m and 6 m respectively, and the height h of the camera is about 12 m. Adjust the attitude to determine its parking space monitoring area. The specific steps are as follows:

[0114] Step 4-1: Use the conversion matrix obtained in Step 1 to convert the coordinates P(x p , y p ) of the 2D detection frame to the earth coordinate system W(x w , y w , z w ). Select the corresponding coordinate deduction method according to the vehicle orientation to obtain its 8 spatial coordinate points, as Figure 5 shown. The specific coordinate deduction formula is shown in Table 1, where l, j, and h represent the length, width, and height of the vehicle respectively;

[0115] Table 1

[0116]

[0117] Step 4-2: Adopt the LM algorithm. Set the initial size values according to the vehicle type. Through the error calculation and regression fitting of a large number of observed values, correct the length, width, and height of the vehicle target. It is necessary to select appropriate numbers of observed values for fitting to balance the accuracy of estimation and the time of parameter estimation;

[0118] In this embodiment, for example, set the initial values of the car size to 5 m, 2.5 m, and 2.5 m. Observe the same target 50 times, that is, 50 frames of images. If the frame rate is 25, correct the length, width, and height of the vehicle target once every 2 seconds;

[0119] The error objective function used in the fitting process is as follows:

[0120]

[0121] where is the diagonal distance of the 2D detection frame, and ‖q0, q6‖ is the 3D diagonal distance of the vehicle in the earth coordinate system;

[0122] Find the minimum value of the error objective function, gradually correct the length, width, and height of the vehicle target, and reduce the morphological difference between the 2D detection frame and the 3D detection frame until the error and parameters converge.

[0123] Step 5: Detect whether the vehicle crosses the line. After determining the vehicle size, re-determine the vehicle's geodetic coordinates, determine the overlapping area between the rectangle of the vehicle bottom contour and the parking space contour, and determine the probability that the vehicle is in a state of crossing the line while parking according to the line-crossing probability function, specifically as follows:

[0124] Step 5-1: Determine the intersection area s between the four vertex coordinates of the vehicle bottom obtained in Step 3 and the four vertices of the parking space. In this embodiment, the Sutherland-Hodgman algorithm is used to obtain the intersection area s.

[0125] Step 5-2: Determine whether the vehicle crosses the line according to the intersection area s between the vehicle and the parking space. The variation relationship between the intersection area s and the line-crossing probability p is as follows:

[0126]

[0127] where s ∈ [0, S], S represents the upper limit value of the occupied area corresponding to the vehicle model, and n is used to control the inner swing amplitude of the function, that is, the range allowing a small error. The schematic diagram is as Figure 6 shown;

[0128] Due to the randomness and unexplainability of neural network prediction, there are inevitable regression errors in the target detection 2D detection frame. Also, due to the lack of sufficient environmental references in camera calibration, it ultimately leads to errors in the 3D detection frame. This function can allow certain errors in each pre-processing process, make a more reasonable judgment on the possibility of the vehicle crossing the line, and avoid misjudgment.

[0129] Therefore, when judging whether the line is crossed according to the intersection area s between the vehicle and the parking space, within a certain range where s is small, the possibility of crossing the line increases slowly with the increase of s. However, as s increases and exceeds a certain range, the possibility of crossing the line will increase rapidly.

[0130] In this embodiment, n is set to 2, W 小车 is 2.5, L 小车 is 5, S is set to 12.5, and some values of the line-crossing probability function are shown in Table 2.

[0131] Table 2 Reference values of line-crossing probability and overlapping area

[0132] Area 0 1 2 3 4 5 6 Probability 1 0.85 0.71 0.58 0.46 0.36 0.27 Area 7 8 9 10 11 12 12.5 Probability 0.19 0.12 0.08 0.04 0.01 0.001 0

[0133] In this embodiment, the running speeds of the target detection, orientation judgment, and size fitting modules in each frame are 31.2 ms, 3.1 ms, and 7.8 ms respectively, and the total consumption is 42.1 ms, which has good real-time performance.

[0134] As Figure 7 , Figure 8As shown, these are 2 frames intercepted from the surveillance video in the service area, respectively showing the algorithm detection effects when there is no vehicle pressing the line and when there is a vehicle pressing the line.

[0135] The surveillance camera monitors 7 complete parking spaces on the left. There are 4 vehicles in stable parking states in the figure. The red and green dot arrays in the lower right corner represent the states of the parking spaces from top to bottom (red indicates occupied, green indicates vacant), and the English words in the lower right corner indicate the current numbers of vacant, occupied, and line-pressing parking spaces.

[0136] After several frames, the algorithm has conducted sufficient observational sampling on the current scene and starts to fit the 3D dimensions of the vehicle, thus transforming into the detection effect diagram as Figure 8 shown. In this diagram, auxiliary prompt states of line-pressing alarms appear in the 3 parking spaces at the bottom. At this time, the supervisors can conduct more rigorous inspections on the key parking spaces to confirm whether they are in the state of a vehicle pressing the line.

[0137] It is not difficult to find from the figure that the front part of the first vehicle at the bottom may slightly press the line, and for the second vehicle, due to parking too high, there is a high possibility of pressing the upper parking line. The third vehicle may also be in a line-pressing situation because it is parked too high.

[0138] Two consecutive frames can lead to the following conclusions: (1) In the case of occlusion, it is difficult to judge line-pressing, and the final judgment needs to combine people's actual experience and the actual on-site situation. However, the line-pressing detection function gives the possibility of this situation for the target vehicle, reducing the time for the supervisors to check numerous vehicles and playing an auxiliary role in screening; (2) Figure 8 The video frame shown is located Figure 7 after the video frame shown. Both vehicles are in the state of pressing the line and are parked too high, reflecting the relatively common driving and parking thinking and the cause of the line-pressing state, that is, the irregular parking of the front vehicle may bring about the irregular parking of the rear vehicle. If the line-pressing detection function can be used in time to assist the on-site personnel to park more regularly, it may be possible to avoid the irregular parking of later vehicles, playing a role in strengthening the management of parking spaces and avoiding accidents.

Claims

1. A method for detecting parking line crossing in a service area based on high-position video, characterized in that, It includes the following steps: Step 1: Calibrate the surveillance camera to obtain the conversion relationship between the image coordinates and the geodetic coordinates: When there are no cars or few cars in the monitored area, take pictures and samples of the parking space area. The sampled pictures should be able to completely display the shapes of each parking space line in the area, including the four vertices of each parking space area. The camera is set on the lamp post, and the posture of the camera is fixed when taking pictures and samples; Set the origin of the geodetic coordinate system to , which is located at the bottom of the high pole, and its coordinate axes are respectively , , ; Let the corresponding points of the four points A, B, C, and D in the parking space area on the camera imaging plane image be a, b, c, and d, where is the principal point of the imaging plane image, and the vanishing point is the intersection point of the line segment da and the line segment cb, and the vanishing point is the intersection point of the line segment cd and the line segment ba. The vanishing point coordinates are respectively: ( , ), ( , ), is the pitch angle, is the angle between the optical axis projection line and the parking space line, and 0° ≤ <90°, and the camera installation height is h; Then the coordinate conversion formula relationship between the geodetic coordinates and the image coordinates ( , ) is as follows: ； ； ； ； where k is the proportionality coefficient; Step 2: Detect vehicle targets, read the surveillance video stream, detect parking targets in real time, output the vehicle type category, and at the same time give the image coordinate information of the parking target and the size of the bounding box of the parking target; Step 3: Judge the vehicle orientation. Select the target vehicle area according to the vehicle bounding box, perform Gaussian filtering, edge detection, and line detection on this area, extract the contour features of the vehicle, and judge the vehicle orientation; Step 4: Fit the vehicle size. Based on the conversion relationship between the image coordinates and the geodetic coordinates, deduce the 3D detection box of the vehicle from the 2D detection box obtained in Step 2, and estimate and optimize the vehicle parameters: Step 4-1: Using the transformation matrix obtained in Step 1, transform the coordinates P( ) of the 2D detection box to the geodetic coordinate system as Q( ). Select the corresponding coordinate derivation method according to the vehicle orientation to obtain its 8 spatial coordinate points. The specific coordinate derivation formula is as follows, where represent the length, width, and height of the vehicle respectively; When the slope is less than 0, the 8 spatial coordinate points are respectively: ： ， ： ， ： ， ： ， ： ， ： ， ： ， ： ； When the slope is greater than 0, the 8 spatial coordinate points are respectively: ： ， ： ， ： ， ： ， ： ， ： ， ： ， ： ； Step 4-2: Adopt the LM algorithm, set the initial size value according to the vehicle type, and correct the length, width, and height of the vehicle target through the error calculation and regression fitting of a large number of observed values. The error objective function used in the fitting process is as follows: ； Among them is the diagonal distance of the two-dimensional detection box, is the 3D diagonal distance of the vehicle in the earth coordinate system; Find the minimum value of the error objective function, gradually correct the length, width, and height of the vehicle target, and reduce the morphological difference between the two-dimensional detection box and the three-dimensional detection box until the error and parameters converge; Step 5: Detect the vehicle's line-crossing situation. After determining the vehicle size, re-determine the geodetic coordinates of the vehicle, determine the overlapping area between the bottom contour rectangle of the vehicle and the contour of the parking space, and determine the probability that the vehicle is in a parking line-crossing state according to the line-crossing probability function; Step 5-1: Determine the intersection area between the four vertex coordinates of the vehicle bottom obtained in Step 3 and the four vertices of the parking space ; Step 5-2: Based on the intersection area between the vehicle and the parking space Determine whether the vehicle crosses the line and the intersection area The probability of pressing the line The change relationship is: ； Among them, , represents the upper limit value of the occupied area of the corresponding vehicle model, which is used to control the inner swing amplitude of the function.

2. The method for detecting parking line crossing in a service area based on high-position video according to claim 1, characterized in that, The detection of vehicle targets in Step 2 specifically includes the following steps: Step 2-1: Classify the vehicle target detection model, including cars, SUVs, large and small buses, hazardous chemical vehicles, and extra-long vehicles, and select a single-stage target detection method; Step 2-2: Input the video collected by the monitoring camera, and use the vehicle target detection model to output the vehicle type category of the parking target in the video frame, the vehicle center coordinate P( ), the width of the target area , and the height .

3. The method for detecting parking line crossing in a service area based on high-position video according to claim 1, characterized in that, The judgment of vehicle orientation in Step 3 is based on the image coordinate system and is divided into two vehicle orientations according to whether the slope of the vehicle orientation is greater than 0. Specifically, it includes the following steps: Step 3-1: Denoise the original surveillance picture; Step 3-2: Perform canny edge detection on the target area to extract the edge features of the vehicle target; Step 3-3: Screen the results of the edge detection, extract the straight edges as the direction vectors of the vehicle, set thresholds according to the size of the vehicle area, respectively used to constrain the length and search step of the straight edges, and remove small and isolated edge segments to finally obtain the direction vector set of the vehicle; Step 3-4: Make a histogram statistics of the direction vectors of the vehicle, and consider the mean value of its normal distribution as the direction represented by the side of the vehicle body, so as to determine the vehicle orientation.

4. A service area parking line crossing detection system based on high-position video, which is used to execute the method described in claim 1, and is characterized in that It includes the following modules: Vehicle target detection module: used to detect parking targets in real time, output the vehicle type category, and at the same time give the image coordinate information of the parking target and the size of the bounding box of the parking target; Vehicle orientation determination module: used to extract the contour features of the vehicle and determine the vehicle's orientation; Vehicle size fitting module: used to estimate and optimize vehicle parameters; Line crossing determination module: used to detect the situation of the vehicle crossing the line.

5. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1-3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method described in any one of claims 1-3.

Images