A method and system for monitoring vehicle violations based on urban monitoring poles

Abstract

The present application relates to the technical field of vehicle violation monitoring, and discloses a vehicle violation monitoring method and system based on a city monitoring pole, which comprises the following steps: a mapping relationship between image coordinates and world coordinates is established by a stop line and a feature point of a pedestrian crossing, a polar coordinate system with an entry corner point of the pedestrian crossing as a pole point is constructed, and an angular sector area is divided, and a detection zone is formed by limiting a height range. A time sequence of the first appearance of a pedestrian in the detection zone is extracted, a time difference value of adjacent angular sectors is calculated to obtain an angular gradient peak index and a maximum peak angular sector position. When the index exceeds a threshold value, a forensic judgment time window is generated according to the maximum peak angular sector; otherwise, a backup time window is used. The position of the front end of a vehicle relative to the stop line and the pedestrian occupancy rate of the entry area of the pedestrian crossing are detected in the time window and the angular neighborhood, and if the pedestrian occupancy rate reaches a threshold value and the vehicle position crosses the line, a violation is output, otherwise, no violation is output.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle violation monitoring, more particularly, it relates to a vehicle violation monitoring method and system based on urban monitoring poles. BACKGROUND

[0002] The existing urban traffic violation monitoring system mainly relies on the camera fixedly installed on the monitoring pole to automatically collect and analyze the vehicle and signal state at the intersection. Such a system can realize the automatic identification of red light running, line crossing and other violations through fixed frame rate and preset detection area in the conventional traffic scene. However, the school surrounding road section has obvious time and behavior differences: during the school and school dismissal period, the flow and vehicle flow are densely interwoven in a short time, the vehicles often approach the intersection or stop at the curb to pick up students at a very low speed, and the pedestrian groups often swarm into the zebra crossing when the signal switches or the traffic interval. Unlike ordinary intersections, this type of area has multiple levels of dynamic interference, including the obstruction formed by the temporarily parked vehicles of parents, the interference of school bus parking signals, and the temporary violation of the gestures of school protection personnel. Thus, the appearance, obstruction and disappearance of pedestrians and vehicles in the monitoring picture show high frequency and unpredictable nature.

[0003] Under the existing image forensics mechanism, the camera performs image collection and analysis at a fixed time interval, and its determination logic is usually based on the static geometric relationship among the vehicle, the stop line and the signal light. This method relies on the smooth change between consecutive frames, and when the picture appears suddenly due to obstruction, the algorithm cannot immediately perceive the real scene change. Especially at the school gate, the temporarily parked vehicles and the opening and passenger drop-off behavior form a random obstruction corridor, and the child pedestrians often suddenly appear in a very narrow angle area of the monitoring field of view. Since the camera sampling period and the time of sudden appearance often do not match, the system may miss the vehicle line crossing moment or misjudge the vehicle courtesy state, causing the violation forensics evidence chain to break. Such misjudgment is less common at ordinary intersections, but it is very common during the swarming release stage at the school gate, which has been a long-standing technical bottleneck in traffic regulation and law enforcement.

[0004] The reason for the above problem lies in the suddenness and discontinuity of the traffic scene around the school. The parent vehicles frequently start and stop near the stop line, forming a temporary visual barrier, which causes the spatial relationship between pedestrians and vehicles in the monitoring picture to change nonlinearly over time. At the same time, the child pedestrians are low in height and slow in pace, and they only need a very short time to be completely obscured to first visible in the picture, which often occurs at the critical moment when the vehicle slowly probes or prepares to pass. The traditional monitoring system lacks the dynamic response capability to such sudden visibility phenomenon, and cannot adjust the image sampling density or timing analysis window in real time according to the obstruction change, ultimately causing the forensics picture to deviate from the key time point of the actual event. SUMMARY

[0005] The application provides a vehicle violation monitoring method and system based on a city monitoring pole, and solves the technical problem of how to accurately capture and determine whether a vehicle has a violation behavior such as line crossing or not yielding in the case of sudden blocking of a signalized intersection around a school and sudden appearance of pedestrians, so as to avoid missing of key frames or distortion of determination time.

[0006] In a first aspect, a vehicle violation monitoring method based on a city monitoring pole comprises:

[0007] Collecting a target image;

[0008] Establishing a mapping relationship between image coordinates and world coordinates based on feature points of a stop line and a pedestrian crossing, constructing a polar coordinate system with an entry corner point of the pedestrian crossing as a pole point and dividing angle fan regions, and limiting a detection zone of a preset height range in each angle fan region;

[0009] Obtaining a time sequence of first appearance of pedestrians in the detection zone of each angle fan region, calculating a time difference value sequence between adjacent angle fan regions, obtaining an angular gradient peak index and an angle fan position corresponding to a maximum peak based on the time difference value sequence;

[0010] When the angular gradient peak index is greater than a preset angular gradient peak index threshold, determining an angular neighborhood at the angle fan position corresponding to the maximum peak, and generating a forensic determination time window in combination with the first appearance time and the peak index; when the angular gradient peak index is less than or equal to the preset angular gradient peak index threshold, a preset backup determination time window is used;

[0011] In the forensic determination time window or the backup determination time window and the angular neighborhood, a position parameter of a front end of a vehicle driving towards the pedestrian crossing relative to the stop line and a pedestrian occupation parameter of an entry region of the pedestrian crossing are detected; when the pedestrian occupation parameter meets a preset occupation condition and the vehicle position parameter meets a preset line crossing condition, a violation is output; otherwise, no violation is output.

[0012] In a second aspect, a vehicle violation monitoring system based on a city monitoring pole is applied to any one of the vehicle violation monitoring methods based on a city monitoring pole, and comprises:

[0013] A data acquisition module acquires a target image;

[0014] A data processing module establishes a mapping relationship between image coordinates and world coordinates based on feature points of a stop line and a pedestrian crossing, constructs a polar coordinate system with an entry corner point of the pedestrian crossing as a pole point and divides angle fan regions, and limits a detection zone of a preset height range in each angle fan region;

[0015] The data screening module acquires a time sequence of the first appearance of the pedestrian in the detection zone of each angular sector region, calculates a time difference value sequence between adjacent angular sector regions, obtains an angular gradient peak index and an angular sector position corresponding to a maximum peak based on the time difference value sequence;

[0016] The determination time window selection module determines an angular neighborhood at the angular sector position corresponding to the maximum peak when the angular gradient peak index is greater than a preset angular gradient peak index threshold, and generates a forensic determination time window in combination with the first appearance time and the peak index; when the angular gradient peak index is less than or equal to the preset angular gradient peak index threshold, a preset backup determination time window is used;

[0017] The violation determination module detects a position parameter of the front end of the vehicle driving towards the pedestrian crossing relative to the stop line and a pedestrian occupancy parameter of the entrance area of the pedestrian crossing in the forensic determination time window or the backup determination time window and the angular neighborhood; when the pedestrian occupancy parameter meets a preset occupancy condition and the vehicle position parameter meets a preset line-crossing condition, a violation is output; otherwise, a non-violation is output.

[0018] The present application has the beneficial effects that by introducing the first appearance time angular gradient peak index and combining the time and space distribution modeling of the polar angular sector partition, the spatial direction and time mutation information of the pedestrian rushing into the pedestrian crossing can be captured in the high dynamic interference scene of the signalized intersection around the school, so as to adaptively generate a forensic determination time window, without relying on a fixed time delay or external signal control logic to realize dynamic self-calibration of the violation triggering time. The present application takes the monitoring image as the input, and can still stably separate the pedestrian rushing behavior and the vehicle line-crossing behavior under the conditions of complex illumination, pedestrian gathering and non-motor vehicle mixed driving, significantly reduces the misjudgment rate and the missed detection rate, and improves the real-time performance, accuracy and intelligent adaptive level of the violation detection system based on the city monitoring pole in the multi-target interference scene. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a flowchart of a vehicle violation monitoring method based on a city monitoring pole of the present application;

[0020] Figure 2 is a module diagram of a vehicle violation monitoring method based on a city monitoring pole of the present application. DETAILED DESCRIPTION

[0021] The subject matter described herein will now be discussed with reference to example implementations. It should be understood that the implementations discussed are merely exemplary of the subject matter described herein and a variety of modifications can be made to the elements discussed without departing from the scope of the present description. Various examples can omit, substitute, or add various procedures or components as appropriate, and the methods described can also be unable to follow the order discussed, where appropriate. Additionally, the features described with respect to some examples can be combined in other examples.

[0022] Embodiment One: As shown in the figure, a vehicle violation monitoring method based on urban monitoring pole, comprising: Figure 1

[0023] Collecting target images;

[0024] Establishing mapping relationship between image coordinates and world coordinates based on feature points of stop line and pedestrian crossing, constructing polar coordinate system with entry corner point of pedestrian crossing as pole point and dividing angle sector area, limiting detection zone of preset height range in each angle sector area;

[0025] Obtaining time sequence of first appearance of pedestrians in detection zone of each angle sector area, calculating time difference value sequence between adjacent angle sector areas, obtaining angle gradient peak index and angle sector position corresponding to maximum peak based on time difference value sequence;

[0026] When angle gradient peak index is greater than preset angle gradient peak index threshold, determining angle neighborhood with angle sector position corresponding to maximum peak, and generating evidence collection determination time window in combination with first appearance time and peak index; when angle gradient peak index is less than or equal to preset angle gradient peak index threshold, adopting preset standby determination time window;

[0027] In evidence collection determination time window or standby determination time window and angle neighborhood, detecting position parameter of front end of vehicle driving towards pedestrian crossing relative to stop line and pedestrian occupation parameter of entry area of pedestrian crossing; when pedestrian occupation parameter meets preset occupation condition and vehicle position parameter meets preset crossing line condition, outputting violation; otherwise, outputting no violation.

[0028] In an embodiment of the present application, based on feature points of stop line and pedestrian crossing, mapping relationship between image coordinates and world coordinates is established, polar coordinate system with entry corner point of pedestrian crossing as pole point is constructed and angle sector area is divided, and detection zone of preset height range is limited in each angle sector area, comprising:

[0029] Collecting four pairs of corresponding points of stop line and pedestrian crossing; each pair of corresponding points includes an image coordinate and a world coordinate, the image coordinate is a coordinate on the target image, and the world coordinate is a coordinate on the road plane; mapping relationship between image coordinates and world coordinates is established through plane homography matrix; ​

[0030] Select the entrance angle point of the pedestrian crossing, obtain the image coordinates of the entrance angle point of the pedestrian crossing, convert the image coordinates into corresponding world coordinates through the plane homography matrix, and construct a polar coordinate system with the world coordinates as the polar point; in the polar coordinate system, the polar radius of any point is the world distance from the point to the polar point, and the polar angle of any point is the world azimuth of the point relative to the polar point;

[0031] Set the angle lower bound, angle upper bound, angle step, and angle fan number of the polar coordinate system, divide the interval of the polar angle according to the angle lower bound, angle upper bound, and angle step into angle fan regions equal to the angle fan number, and each angle fan region is a continuous angle range;

[0032] Set the lower boundary pixel value and the upper boundary pixel value in the longitudinal pixel direction of the target image, and define a detection zone in each angle fan region, which is a pixel set that is in the angle range of the angle fan region and between the lower boundary pixel value and the upper boundary pixel value in the longitudinal pixel direction.

[0033] Each pair of corresponding points is composed of the pixel coordinates (image coordinates) of the feature points in the target image and the physical coordinates (world coordinates) of the points on the actual road plane; four pairs of corresponding points are selected from the points with obvious features on the stop line and the pedestrian crossing (such as the endpoints of the stop line, the endpoints of the entrance of the pedestrian crossing, the endpoints of the exit of the pedestrian crossing, and any four pairs of non-collinear points), to ensure accurate coordinate mapping.

[0034] The plane homography matrix is a 3x3 matrix, which is used to convert the pixel coordinates on the image plane into the world coordinates on the road plane, or vice versa; specifically:

[0035] First, select four pairs of non-collinear corresponding points, and let the image coordinates be (u1, v1), (u2, v2), (u3, v3), and (u4, v4), and the corresponding world coordinates be (X1, Y1), (X2, Y2), (X3, Y3), and (X4, Y4);

[0036] Second, for each pair of corresponding points, two equations are listed: u multiplied by (the third row first column element of the matrix x X + the third row second column element of the matrix x Y + the third row third column element of the matrix) is equal to (the first row first column element of the matrix x X + the first row second column element of the matrix x Y + the first row third column element of the matrix), and v multiplied by (the third row first column element of the matrix x X + the third row second column element of the matrix x Y + the third row third column element of the matrix) is equal to (the second row first column element of the matrix x X + the second row second column element of the matrix x Y + the second row third column element of the matrix);

[0037] Third, let the third row third column element of the matrix be 1 (to eliminate the scale ambiguity), and obtain eight equations, and solve the remaining eight matrix elements by using the least squares method to obtain the plane homography matrix.

[0038] Pedestrian crossing entrance corner point is the two end points of the pedestrian crossing near the side of the oncoming vehicle direction (optionally one of them, usually the end point near the monitoring rod side), the image coordinates are obtained by clicking the corner point in the target image, and the world coordinates are converted by the plane homography matrix.

[0039] The polar point of the polar coordinate system is the world coordinate of the pedestrian crossing entrance corner point; the polar radius is the actual physical distance from any point to the polar point (unit: meters); the polar angle is the angle between the line connecting any point and the polar point and the reference line pointing to the front of the oncoming vehicle direction (unit: degrees, clockwise is positive).

[0040] The angle lower bound is the minimum angle of the polar angle interval (usually set to -60 degrees, corresponding to the left side of the oncoming vehicle direction), the angle upper bound is the maximum angle of the polar angle interval (usually set to 60 degrees, corresponding to the right side of the oncoming vehicle direction), the angle step is the angle interval of adjacent angle sectors, and the angle sector number is the total number of angle sectors to be divided; specifically:

[0041] First, the angle lower bound and the angle upper bound are set according to the actual width of the intersection to ensure that all possible areas near the pedestrian crossing in the oncoming vehicle direction are covered (generally -60 degrees to 60 degrees);

[0042] Second, the angle step is equal to (angle upper bound minus angle lower bound) divided by (angle sector number minus 1), and the angle sector number is usually set to 12 (step is 10 degrees) to ensure uniform coverage of the angle sectors;

[0043] Third, angle sector region division: the first angle sector ranges from the angle lower bound to (angle lower bound + angle step), the second from (angle lower bound + angle step) to (angle lower bound + 2 × angle step), and so on until the angle upper bound.

[0044] The longitudinal pixel direction is the vertical direction of the target image (from top to bottom or from bottom to top); the lower boundary pixel value and the upper boundary pixel value are the pixel coordinates in this direction, which are used to define the longitudinal range of the detection zone, corresponding to the child head and shoulder height range (0.6 meters to 1.4 meters) in the world coordinates; specifically:

[0045] First, convert the 0.6 meters and 1.4 meters heights (based on the road plane) in the world coordinates to longitudinal pixel coordinates in the target image through the plane homography matrix;

[0046] Second, set the pixel coordinates corresponding to the 0.6 meters height obtained by conversion as the lower boundary pixel value of the longitudinal pixels, and set the pixel coordinates corresponding to the 1.4 meters height as the upper boundary pixel value of the longitudinal pixels.

[0047] The detection zone is a collection of pixels in a specific area of the target image, which needs to meet two conditions:

[0048] I. The polar angle corresponding to the pixel is within the angle range of the angle sector,

[0049] II. The longitudinal pixel coordinate of the pixel is between the lower boundary pixel value and the upper boundary pixel value, which is used for focus detection of children and pedestrians.

[0050] In an embodiment of the present application, the time sequence of the first appearance of pedestrians in the detection zone of each angle sector region is obtained, and the time difference sequence between adjacent angle sector regions is calculated, including:

[0051] determining a timestamp set of the video, the timestamp set being a frame time set that is monotonically increasing in image acquisition order;

[0052] For the detection zone corresponding to the mth angle sector region, at each timestamp in the timestamp set, a pedestrian pixel set composed of pixels determined as pedestrians in the detection zone is determined;

[0053] A pedestrian pixel number threshold is set, and for the pedestrian pixel set at each timestamp, the number of pixels in the pedestrian pixel set is counted; if the number of pixels is greater than or equal to the pedestrian pixel number threshold, the appearance indicator of the mth angle sector region at the timestamp is set to 1; if the number of pixels is less than the pedestrian pixel number threshold, the appearance indicator is set to 0;

[0054] From all the timestamps corresponding to the appearance indicator of the mth angle sector region being 1, the smallest timestamp is selected as the first appearance time of pedestrians in the mth angle sector region; the first appearance times of pedestrians in multiple angle sector regions together constitute the time sequence of the first appearance of pedestrians;

[0055] The difference between the first appearance times of pedestrians in adjacent angle sector regions is calculated, specifically: the first appearance time of pedestrians in the latter angle sector region minus the first appearance time of pedestrians in the former angle sector region; the difference values of multiple adjacent angle sector regions together constitute the time difference sequence between adjacent angle sector regions; the difference value is calculated only when there are first appearance times of pedestrians in adjacent two angle sector regions.

[0056] The timestamp set is a set composed of the acquisition times corresponding to each frame of image in the video, arranged in order of frame acquisition, with time values increasing in turn, reflecting the shooting time of each frame of image; specifically:

[0057] First, determine the frame rate of the monitoring camera (usually take 25 frames per second or 30 frames per second, set according to device parameters);

[0058] Second, the first frame timestamp is set to 0 seconds, the second frame is (1 divided by frame rate) seconds, the third frame is (2 divided by frame rate) seconds, and so on, the nth frame timestamp is (n minus 1) divided by frame rate, and all frame timestamps are arranged in order to form a timestamp set.

[0059] The pedestrian pixel set is a set of all pixels determined as belonging to a pedestrian by a pedestrian detection algorithm within the mth corner fan detection zone in a frame (corresponding to a timestamp); in particular:

[0060] First, background subtraction is used (compare the pixels in the current frame detection zone with the corresponding pixels in the background frame without pedestrians, and if the difference is greater than a preset gray threshold (usually 30, adjusted according to light conditions), the pixel is preliminarily determined as a foreground pixel);

[0061] Second, morphological filtering is performed on the foreground pixels (noise regions with an area less than 5 pixels are removed);

[0062] Third, the filtered foreground pixels are determined as pedestrian pixels to form a pedestrian pixel set.

[0063] The pedestrian pixel number threshold is a pixel number standard for determining whether there are pedestrians in the detection zone, used to distinguish between the presence and absence of pedestrians; in particular:

[0064] First, calculate the total number of pixels in the mth corner fan detection zone (the number of horizontal pixels multiplied by the number of vertical pixels);

[0065] Second, set the pedestrian pixel number threshold to 5% of the total number of pixels in the detection zone (if the total number of pixels in the detection zone is 2000, the threshold is 100, which can be adjusted according to the minimum imaging size of a child pedestrian, which is not less than 30 pixels).

[0066] The occurrence indicator is a binary variable representing whether the mth corner fan detection zone has pedestrians at a certain timestamp, with 1 indicating the presence of pedestrians and 0 indicating the absence of pedestrians.

[0067] The first appearance time of a pedestrian is the time corresponding to the frame in which a pedestrian is first detected in the mth corner fan detection zone, i.e., the earliest time at which a pedestrian appears in the corner fan; in particular:

[0068] First, collect all timestamps with an occurrence indicator of 1 in the mth corner fan region to form a timestamp subset;

[0069] Second, if the subset is empty (no pedestrians appear), mark the corner fan as having no first appearance time; if the subset is not empty, select the smallest timestamp value in the subset as the first appearance time of a pedestrian in the corner fan.

[0070] The time series is a sequence of pedestrian first appearance times for each corner fan arranged in order of corner fan number (from 1 to the total number of corner fans); if a corner fan has no first appearance time, mark the position as no data.

[0071] The difference value is the calculation result of the first occurrence time of two adjacent corner fans (such as the first and second corner fans, the second and third corner fans), and reflects the time difference of the adjacent corner fans in detecting pedestrians; specifically: first, determine the adjacent corner fan pair (the i-th and i+1-th corner fans, i is from 1 to the total number of corner fans minus 1); second, only when both corner fans have a first occurrence time, calculate the difference value: the difference value is equal to the first occurrence time of the i+1-th corner fan minus the first occurrence time of the i-th corner fan; if any corner fan has no first occurrence time, the difference value of the pair is not calculated.

[0072] The time difference value sequence is a sequence composed of all valid difference values (calculated difference values) arranged in the order of adjacent corner fan pairs (from the first-2 pair to the (total number of corner fans-1)-total number of corner fans pair).

[0073] In an embodiment of the present application, the angular gradient peak index and the corner fan position corresponding to the maximum peak are obtained based on the time difference value sequence, comprising:

[0074] The absolute value of each time difference value in the time difference value sequence between adjacent corner fan regions is taken respectively to obtain the absolute amount of angular difference corresponding to each adjacent corner fan region;

[0075] The arithmetic mean of all absolute amounts of angular difference, the arithmetic mean of the absolute amount, the number of absolute amounts of angular difference participating in the calculation is equal to the total number of corner fan regions minus one;

[0076] The maximum absolute amount of angular difference is selected from all absolute amounts of angular difference, and the maximum absolute amount of angular difference is divided by the arithmetic mean of all absolute amounts of angular difference to obtain the angular gradient peak index;

[0077] The count variable of the adjacent corner fan region corresponding to the maximum absolute amount of angular difference is determined, and the count variable is the corner fan position corresponding to the maximum peak.

[0078] The absolute amount of angular difference is the absolute value of the time difference value of the adjacent corner fan, which is used to eliminate the direction influence brought by the positive and negative of the difference value and only keep the size of the time difference; specifically: for each valid difference value in the time difference value sequence, the absolute value is taken, and the result is the absolute amount of angular difference corresponding to the adjacent corner fan pair.

[0079] The absolute amount mean is the average level of all absolute amounts of angular difference, which reflects the overall trend of the time difference of the first occurrence of pedestrians in adjacent corner fans; specifically:

[0080] First, the effective number of absolute amounts of angular difference (only including the calculated absolute amount, if a pair of adjacent corner fans has no difference value, it is excluded, and the effective number is at most equal to the total number of corner fan regions minus one) is counted;

[0081] Second step, calculate the sum of all effective absolute values of angular difference; third step, the absolute value mean is equal to the sum divided by the effective number.

[0082] The maximum absolute value of angular difference is the maximum value among all effective absolute values of angular difference, reflecting the extreme case of the adjacent angle fan pedestrian first appearance time difference; Specifically: compare all effective absolute values of angular difference, select the maximum value, which is the maximum absolute value of angular difference.

[0083] The angular gradient peak index is an index for measuring the difference between the extreme value of the adjacent angle fan time difference and the overall average value, the larger the value, the more prominent the time difference of the adjacent angle fan (corresponding to the sudden appearance of pedestrians caused by occlusion); Specifically:

[0084] First step, calculate the maximum absolute value of angular difference and the absolute value mean according to the method described above;

[0085] Second step, if the absolute value mean is zero (all adjacent angle fan time differences are zero), the angular gradient peak index is set to 1 (no peak); Otherwise, the angular gradient peak index is equal to the maximum absolute value of angular difference divided by the absolute value mean.

[0086] The count variable of the adjacent angle fan region is the number of adjacent angle fan pairs, used to locate the specific adjacent angle fan corresponding to the maximum absolute value; Specifically:

[0087] From the first adjacent angle fan pair (the first angle fan and the second angle fan), it is numbered in order as 1, the second adjacent angle fan pair (the second angle fan and the third angle fan) is numbered as 2, and so on, the count variable of the kth adjacent angle fan pair (the kth angle fan and the k+1th angle fan) is k, until the last adjacent angle fan pair.

[0088] The angle fan position corresponding to the maximum peak is the count variable value of the adjacent angle fan pair to which the maximum absolute value of angular difference belongs, used to locate the angle fan region with the most prominent pedestrian first appearance time difference.

[0089] In an embodiment of the present application, when the angular gradient peak index is greater than the preset angular gradient peak index threshold, the angle neighborhood is determined by the angle fan position corresponding to the maximum peak, and the first appearance time and the peak index are combined to generate a forensic judgment time window, including:

[0090] Set the angle neighborhood half-width parameter to determine the number of angle fan regions;

[0091] Define a first angle neighborhood set for the first appearance time, the angle fan number in the first angle neighborhood set satisfies greater than or equal to the angle fan position corresponding to the maximum peak minus the angle neighborhood half-width parameter, less than or equal to the angle fan position corresponding to the maximum peak plus the angle neighborhood half-width parameter plus one, and the angle fan number is between one and the number of angle fan regions;

[0092] define a second angular neighborhood set for the absolute value of the angular difference, the angular sector number in the second angular neighborhood set satisfies greater than or equal to the angular sector position corresponding to the maximum peak minus the angular neighborhood half-width parameter, less than or equal to the angular sector position corresponding to the maximum peak plus the angular neighborhood half-width parameter, and the angular sector number is between one and the number of angular sector regions minus one;

[0093] obtain the first occurrence time of each angular sector region and the absolute value of the angular difference of each adjacent angular sector region; calculate the arithmetic mean of all absolute values of the angular difference in the angular neighborhood set for the absolute value of the angular difference, to obtain the neighborhood average absolute value of the difference, the number of absolute values of the angular difference participating in the calculation is equal to the number of elements in the angular neighborhood set;

[0094] select the minimum first occurrence time from all first occurrence times of the angular sector regions in the angular neighborhood set for the first occurrence time, to obtain the neighborhood first occurrence time;

[0095] set a time advance coefficient and a time retention coefficient;

[0096] multiply the angular gradient peak index by the time advance coefficient, and then multiply it by the neighborhood average absolute value of the difference, to obtain the time advance amount;

[0097] multiply the angular gradient peak index by the time retention coefficient, to obtain the time retention amount;

[0098] take the neighborhood first occurrence time as the reference, subtract the time advance amount from the neighborhood first occurrence time to obtain the starting time of the evidence determination time window, add the time retention amount to the neighborhood first occurrence time to obtain the end time of the evidence determination time window, and the starting time to the end time constitutes the evidence determination time window.

[0099] The angular neighborhood half-width parameter is an integer parameter for controlling the coverage range of the angular neighborhood, which is used to limit the number of angular sectors that need to be included in the calculation around the angular sector corresponding to the maximum peak, to avoid interference caused by too large neighborhood or insufficient information caused by too small neighborhood; Specifically: the angular neighborhood half-width parameter is set according to the number of angular sector regions, if the number of angular sector regions is 12 (commonly set in claim 2), the half-width parameter is set to 1 or 2; take 2 for serious occlusion scenes (such as double-row parking), and take 1 for lighter occlusion scenes, to ensure that the neighborhood covers 3-5 angular sectors (half-width 1 covers 3, half-width 2 covers 5).

[0100] The first angular neighborhood set is a set of angular sector numbers participating in the calculation of the neighborhood first occurrence time, focusing on the first occurrence time of the angular sector around the maximum peak; Specifically:

[0101] First, calculate the lower limit of the number = the angular sector position corresponding to the maximum peak minus the angular neighborhood half-width parameter, and the upper limit of the number = the angular sector position corresponding to the maximum peak plus the angular neighborhood half-width parameter + 1;

[0102] Second step, if the lower limit is less than 1, take 1, if the upper limit is greater than the number of angular sector area, take the number of angular sector area;

[0103] Third step, collect all integer angular sector numbers from the lower limit to the upper limit (including the boundary) to form the first angular neighborhood set.

[0104] The second angular neighborhood set is the set of adjacent angular sector pair numbers participating in the calculation of the absolute value of the average difference of the neighborhood (the upper limit of the adjacent angular sector pair number is the number of angular sector areas minus one); Specifically:

[0105] First step, calculate the lower limit of the number = the angular sector position corresponding to the maximum peak - the angular neighborhood half-width parameter, the upper limit of the number = the angular sector position corresponding to the maximum peak + the angular neighborhood half-width parameter;

[0106] Second step, if the lower limit is less than 1, take 1, if the upper limit is greater than (the number of angular sector areas - 1), take (the number of angular sector areas - 1);

[0107] Third step, collect all integer adjacent angular sector pair numbers from the lower limit to the upper limit (including the boundary) to form the second angular neighborhood set.

[0108] The absolute value of the average difference of the neighborhood is the average value of all effective angular difference absolute values in the second angular neighborhood set, reflecting the average level of the time difference of adjacent angular sectors in the neighborhood; Specifically:

[0109] First step, extract the angular difference absolute value corresponding to each number in the second angular neighborhood set (only keep the effective absolute value that has been calculated, exclude the number without difference);

[0110] Second step, calculate the sum of these effective absolute values;

[0111] Third step, the absolute value of the average difference of the neighborhood = the sum ÷ the number of effective absolute values (i.e. the number of effective elements in the second angular neighborhood set).

[0112] The neighborhood first appearance time is the earliest time of the pedestrian appearing in all angular sectors with first appearance time in the first angular neighborhood set, which is taken as the reference time of the evidence window; Specifically:

[0113] First step, extract the first appearance time corresponding to each angular sector in the first angular neighborhood set (exclude angular sectors without first appearance time);

[0114] Second step, if the extraction result is empty (no angular sector has first appearance time), take the video starting time (0 seconds) as the neighborhood first appearance time; Otherwise, select the smallest time value in the extraction result, which is the neighborhood first appearance time.

[0115] The time advance coefficient is a coefficient for controlling the length of the forward extension of the evidence window, and is used to cover the vehicle overline key frame before the pedestrian appears; specifically, the time advance coefficient is in the range of 0.5 to 1.0, 1.0 is taken for the school morning peak (severe occlusion, dense pedestrians), 0.5 is taken for the noon flat peak (light occlusion, few pedestrians), and the advance amount is ensured to cover the vehicle overline preparation phase.

[0116] The time maintenance coefficient is a coefficient for controlling the length of the backward extension of the evidence window, and is used to cover the vehicle continuous overline frame after the pedestrian appears; specifically, the time maintenance coefficient is in the range of 0.3 to 0.8, 0.8 is taken for the school evening peak (vehicle slow, overline lasts long), and 0.3 is taken for the flat peak, and the maintenance amount is ensured to cover the whole process of vehicle overline.

[0117] The time advance amount is the length of the forward extension of the starting time of the evidence window relative to the neighbor first appearance time, the greater the peak index (the heavier the occlusion), the greater the neighbor average difference (the more obvious the time difference), and the greater the advance amount; specifically, the time advance amount = time advance coefficient x angular gradient peak index x neighbor average difference absolute amount; if the calculation result is greater than 5 seconds (to avoid too long window), take 5 seconds.

[0118] The time maintenance amount is the length of the backward extension of the ending time of the evidence window relative to the neighbor first appearance time, the greater the peak index (the heavier the occlusion), the greater the maintenance amount; specifically, the time maintenance amount = time maintenance coefficient x angular gradient peak index; if the calculation result is greater than 3 seconds (to avoid window redundancy), take 3 seconds.

[0119] The starting time is the start time of the evidence window, which needs to be ensured not to be earlier than the video start time; specifically, the starting time = neighbor first appearance time - time advance amount; if the starting time is less than 0 seconds (video start time), take 0 seconds.

[0120] The ending time is the end time of the evidence window, which needs to be ensured not to be later than the video end time; specifically, the ending time = neighbor first appearance time + time maintenance amount; if the ending time is greater than the total length of the video, take the total length of the video.

[0121] The evidence determination time window is a time interval focusing on the key period of pedestrian appearance, and only in this interval is the vehicle overline and pedestrian occupation detected, which improves the evidence accuracy.

[0122] In an embodiment of the present application, when the angular gradient peak index is less than or equal to a preset angular gradient peak index threshold, a preset standby determination time window is adopted, which comprises:

[0123] Obtaining the first appearance time of all angular sector regions and the number of angular sector regions, selecting the smallest first appearance time from all angular sector region first appearance times as a standby reference time;

[0124] Set the standby time advance and standby time keeping amount;

[0125] Subtract the standby time advance from the standby reference time to obtain the start time of the standby determination time window, and add the standby time keeping amount to the standby reference time to obtain the end time of the standby determination time window, and the start time to the end time constitute the standby determination time window.

[0126] The standby reference time is the time when pedestrians are detected earliest in all angle sectors, which is used as the reference time of the standby determination time window, and is suitable for scenes without obvious occlusion (angular gradient peak index ≤ threshold value); Specifically:

[0127] First, extract the effective time data with the first occurrence time in all angle sectors (exclude angle sectors without the first occurrence time);

[0128] Second, if the effective time data is empty (no angle sector detects pedestrians), the standby reference time is 10% of the total video duration (e.g. 6 seconds for a 60-second video); Otherwise, select the smallest value in the effective time data, which is the standby reference time.

[0129] The standby time advance is a fixed time length that the standby window start time extends forward relative to the standby reference time, which is used to cover the vehicle crossing frames before the appearance of pedestrians in the non-occlusion scene; Specifically: the standby time advance is set according to the time period, 1 second for the school flat peak period (no concentrated flow), and 2 seconds for the school peak transition period (a small amount of pedestrians); If the calculated advance is less than 0 seconds, take 0 seconds.

[0130] The standby time keeping amount is a fixed time length that the standby window end time extends backward relative to the standby reference time, which is used to cover the vehicle crossing frames after the appearance of pedestrians in the non-occlusion scene; Specifically: the standby time keeping amount is set according to the time period, 1 second for the school flat peak period, and 2 seconds for the school peak transition period; If the calculated keeping amount is greater than 3 seconds (to avoid window redundancy), take 3 seconds.

[0131] The start time is the start time of the standby window, which needs to be ensured not to be earlier than the video start time; Specifically: start time = standby reference time - standby time advance; If the start time is less than 0 seconds (video start time), take 0 seconds.

[0132] The end time is the end time of the standby window, which needs to be ensured not to be later than the video end time; Specifically: end time = standby reference time + standby time keeping amount; If the end time is greater than the total video duration, take the total video duration.

[0133] The standby determination time window is the evidence interval in the scene without obvious occlusion (angular gradient peak index ≤ threshold value), which ensures to cover the key frames before and after the appearance of pedestrians through fixed advance / keeping amount, which is different from the dynamic evidence window in the occlusion scene.

[0134] In an embodiment of the present application, within the forensic decision time window or the backup decision time window and the angular neighborhood, the position parameter of the front end of the vehicle driving towards the pedestrian crossing relative to the stop line is detected, and the pedestrian occupancy parameter of the pedestrian crossing entrance area is detected, including:

[0135] When the angular gradient peak index is greater than the preset angular gradient peak index threshold, the effective time set is the forensic decision time window;

[0136] When the angular gradient peak index is less than or equal to the preset angular gradient peak index threshold, the effective time set is the backup decision time window;

[0137] The detection is only performed in the angular neighborhood in space;

[0138] The straight line parameter of the stop line in the world coordinates is determined, which is used to describe the world geometry of the stop line; at each time point in the effective time set, a set of world coordinate points of the vehicle contour located in the angular neighborhood is obtained; a linear calculation formula is constructed based on the straight line parameter of the stop line, and the point with the minimum result of the linear calculation formula is selected from the set of world coordinate points of the vehicle contour as the vehicle front end point;

[0139] The directional distance from the vehicle front end point to the stop line is calculated, which is the position parameter of the vehicle front end relative to the stop line;

[0140] The pedestrian crossing entrance area is set as a polygon area in the world coordinates, which defines the range of the pedestrian crossing entrance; at each time point in the effective time set, a set of world coordinate points of the pedestrian located in the angular neighborhood is obtained; the area of the overlapping part of the set of world coordinate points of the pedestrian and the polygon area of the pedestrian crossing entrance area is calculated, and the pedestrian occupancy parameter of the pedestrian crossing entrance area is obtained by dividing the total area of the polygon area of the pedestrian crossing entrance area by the area of the overlapping part.

[0141] The effective time set is the time interval actually used to detect the vehicle and pedestrian parameters, which is essentially the forensic decision time window or the backup decision time window, which is switched according to the size of the angular gradient peak index.

[0142] The straight line parameter of the stop line in the world coordinates is a mathematical parameter representing the position and direction of the stop line, which is usually represented by the coefficients (A, B, C) in the form of Ax+By+C=0, and is used to calculate the distance from the vehicle to the stop line; specifically:

[0143] First, two end points on the stop line (such as the left end point and the right end point of the stop line) are selected in the world coordinates (the world coordinates of the left end point and the right end point are obtained by on-site measurement, the left end point is (X1, Y1), and the right end point is (X2, Y2));

[0144] Second step, calculate the straight line parameters: A equals (Y2-Y1), B equals (X1-X2), C equals (X2Y1-X1Y2);

[0145] Third step, standardize A, B, C (such as making A 2 +B 2 =1), to get the straight line parameters of the stop line in the world coordinates.

[0146] Each time in the set of valid time refers to the timestamp corresponding to all video frames in the valid time interval, which is selected frame by frame according to the camera frame rate (such as 25 frames per second, then every 0.04 seconds a time).

[0147] The set of world coordinate points of the vehicle contour is a set composed of world coordinates of all points of the vehicle contour edge in the corner neighborhood at a certain time, which is used to screen the vehicle front end point; Specifically:

[0148] First step, in the current frame image, identify the vehicle in the corner neighborhood through the vehicle detection algorithm (such as YOLO algorithm), and obtain the set of image coordinate points of the vehicle contour;

[0149] Second step, convert each image coordinate point into the corresponding world coordinate point through the plane homography matrix;

[0150] Third step, collect all the converted world coordinate points to form the set of world coordinate points of the vehicle contour.

[0151] The linear calculation formula is an expression for measuring the distance of the vehicle contour point to the stop line, and the smaller the result is, the closer the point is to the stop line, which is used to locate the vehicle front end; Specifically:

[0152] Given the straight line parameters of the stop line as A, B, and C, and the world coordinate point of the vehicle contour as (X, Y), the linear calculation formula is the absolute value of (A×X+B×Y+C) divided by the square root of (A 2 +B 2 ); The essence of this formula is the formula for calculating the perpendicular distance from a point to a straight line.

[0153] The vehicle front end point is the point on the vehicle contour closest to the stop line, representing the actual position of the vehicle front end; Specifically:

[0154] First step, for each point in the set of world coordinate points of the vehicle contour, substitute it into the linear calculation formula to calculate the distance;

[0155] Second step, compare the distance results of all points, and select the point with the smallest distance value, which is the vehicle front end point; If there are multiple points with the same minimum distance, take the point closest to the side of the pedestrian crossing.

[0156] The direction distance is a distance value with positive and negative signs, which is used to determine whether the vehicle has crossed the stop line (positive for not crossing the line, negative for crossing the line), i.e., the vehicle position parameter; specifically:

[0157] First, obtain the stop line straight line parameters A, B, and C, and the world coordinates (X, Y) of the front end point of the vehicle;

[0158] Second, calculate the direction distance: (A×X+B×Y+C) divided by the square root of (A 2 +B 2 );

[0159] Third, if the result is positive, it means that the vehicle is in front of the stop line (not crossing the line); if it is negative, it means that the vehicle is behind the stop line (crossing the line).

[0160] The crosswalk entrance area is a polygon (usually a rectangle) in the world coordinates that defines the starting area of pedestrians entering the crosswalk, which is used to calculate the pedestrian occupancy ratio; specifically:

[0161] First, measure the world coordinates of the four vertices of the crosswalk entrance in the field (such as the left front end (Xa, Ya), the left rear end (Xb, Yb), the right rear end (Xc, Yc), and the right front end (Xd, Yd) in clockwise or counterclockwise order);

[0162] Second, connect the four vertex coordinates in order to form a rectangular polygon area, which is the crosswalk entrance area.

[0163] The set of world coordinate points of the pedestrian is a set of world coordinates of all points of the pedestrian's body in the corner neighborhood at a certain moment, which is used to calculate the overlapping area between the pedestrian and the entrance area; specifically:

[0164] First, in the current frame image, identify the pedestrians in the corner neighborhood through a pedestrian detection algorithm (such as the SSD algorithm) to obtain the set of image coordinate points of the pedestrian contour;

[0165] Second, convert each image coordinate point to the corresponding world coordinate point through a plane homography matrix;

[0166] Third, collect all the converted world coordinate points to form the set of world coordinate points of the pedestrian.

[0167] The area of the overlapping part is the area of the region where the set of world coordinate points of the pedestrian falls within the crosswalk entrance polygon area, reflecting the occupancy range of the pedestrian in the entrance area; specifically:

[0168] First, fit the set of world coordinate points of the pedestrian into a polygon (such as through a convex hull algorithm to connect the outermost coordinate points to form a convex polygon of the pedestrian);

[0169] Second, the intersection area of the pedestrian convex polygon and the crosswalk entrance polygon area is calculated by using the scan line algorithm.

[0170] Third, the area of the intersection area is calculated, which is the area of the overlapping part.

[0171] The total area is the area of the crosswalk entrance polygon area itself, which is used to calculate the occupation ratio. Specifically, if the entrance area is a rectangle (four vertices (Xa, Ya), (Xb, Yb), (Xc, Yc), (Xd, Yd)), the total area is equal to the length multiplied by the width of the rectangle; the length is the absolute value of (Xb-Xa) (or the absolute value of (Xd-Xc)), and the width is the absolute value of (Ya-Yd) (or the absolute value of (Yb-Yc)). If it is other polygon, the shoelace formula is used: the sum of (Xi*Yi+1-Xi+1*Yi) of each vertex (Xi, Yi) and the next vertex (Xi+1, Yi+1) is calculated in turn, and the absolute value is taken and divided by 2 to obtain the total area.

[0172] The pedestrian occupation parameter is a value representing the proportion of the crosswalk entrance area occupied by pedestrians, ranging from 0 to 1, and the larger the value, the more serious the occupation. Specifically, the pedestrian occupation parameter = the area of the overlapping part ÷ the total area of the crosswalk entrance area; if the total area is 0 (an abnormal situation), the occupation parameter is set to 0.

[0173] In an embodiment of the present application, when the pedestrian occupation parameter meets the preset occupation condition and the vehicle position parameter meets the preset crossing line condition, the violation is output; otherwise, the non-violation is output, including:

[0174] Determine the effective time set, when the angular gradient peak index is greater than the preset angular gradient peak index threshold, the effective time set is the evidence determination time window; when the angular gradient peak index is less than or equal to the preset angular gradient peak index threshold, the effective time set is the standby determination time window;

[0175] Set the preset occupation threshold and the preset crossing line tolerance, the preset occupation threshold is the lower limit of the area proportion occupied by pedestrians in the crosswalk entrance area, and the preset crossing line tolerance is the non-negative distance tolerance from the front end of the vehicle to the stop line;

[0176] At each time in the effective time set, the pedestrian occupation parameter and the vehicle position parameter at that time are obtained; it is judged whether each time meets the first condition and the second condition at the same time; all times that meet the two conditions at the same time are collected to form a determination set; the number of times in the determination set is counted; wherein the first condition is that the pedestrian occupation parameter is greater than or equal to the preset occupation threshold, and the second condition is that the vehicle position parameter is less than or equal to the negative preset crossing line tolerance;

[0177] When the number of times in the determination set is greater than or equal to one, the violation is determined.

[0178] When the number of moments in the set is determined to be equal to zero, it is determined that there is no violation.

[0179] The preset occupancy threshold is an area ratio standard for determining whether there is a pedestrian at the entrance of the pedestrian crossing. If the ratio meets the standard, it is determined that the pedestrian occupies effectively. Specifically, the preset occupancy threshold is set according to the scene. The peak period of school (child pedestrian intensive) is set to 0.2 (i.e. 20%), and the off-peak period (sporadic pedestrians) is set to 0.1 (i.e. 10%). If the proportion of children in the scene exceeds 80%, it can be additionally reduced by 5% (such as 0.15 for peak), which adapts to the characteristics of smaller children.

[0180] The preset overline tolerance is the maximum distance (non-negative) that allows the vehicle to slightly cross the line but not be determined as a violation, which is used to exclude parking errors. Specifically, the preset overline tolerance is set according to the road type. The city branch (school surrounding is mostly branch) is set to 0.3 meters, and the city main road is set to 0.2 meters. If there is interference such as water accumulation or snow accumulation near the stop line, it can be temporarily increased by 0.1 meters (maximum not more than 0.4 meters) to avoid misjudgment.

[0181] Each moment in the effective time set is a timestamp corresponding to all video frames in the effective time interval, which is selected frame by frame according to the camera frame rate (such as 30 frames per second, then every 1 / 30 second a moment).

[0182] The first condition is the determination standard of pedestrian occupancy. When the pedestrian occupancy parameter at this moment reaches or exceeds the preset occupancy threshold, it is determined that there is a pedestrian at the entrance of the pedestrian crossing.

[0183] The second condition is the determination standard of vehicle overline. The negative preset overline tolerance indicates that the vehicle overline distance exceeds the tolerance (such as a tolerance of 0.3 meters, a negative tolerance of -0.3 meters, and a position parameter ≤ -0.3 meters, which means overline exceeds 0.3 meters).

[0184] The determination set is a set of moments that simultaneously satisfy the effective pedestrian occupancy and the effective vehicle overline in the effective time set, which is used to count the number of frames related to violations. Specifically:

[0185] First, check whether each moment in the effective time set simultaneously satisfies the first condition and the second condition frame by frame.

[0186] Second, collect the moments (timestamps) that satisfy the double conditions one by one to form the determination set. If there is no moment that satisfies, the determination set is an empty set.

[0187] The number of moments is the total number of moments contained in the determination set, which reflects the number of frames of the violation behavior. Specifically:

[0188] First, if the determination set is an empty set, the number of moments is 0.

[0189] Second step, if the judgment set is not empty, the number of time points in the set is counted (i.e. the number of video frames satisfying the double conditions), denoted as the number of time points.

[0190] Embodiment two: as shown, a vehicle violation monitoring system based on urban monitoring poles, applied in any one of the vehicle violation monitoring methods based on urban monitoring poles, comprising: Figure 2

[0191] A data acquisition module acquires target images.

[0192] A data processing module establishes a mapping relationship between image coordinates and world coordinates based on the feature points of the stop line and the pedestrian crossing, constructs a polar coordinate system with the entry corner point of the pedestrian crossing as the pole point and divides the angular sector area, and limits the detection zone of the preset height range in each angular sector area.

[0193] A data filtering module acquires the time sequence of the first appearance of pedestrians in the detection zone of each angular sector area, calculates the time difference value sequence between adjacent angular sector areas, and obtains the angular gradient peak index and the angular sector position corresponding to the maximum peak based on the time difference value sequence.

[0194] A determination time window selection module determines the angular neighborhood at the angular sector position corresponding to the maximum peak when the angular gradient peak index is greater than the preset angular gradient peak index threshold, and generates a forensic determination time window in combination with the first appearance time and the peak index; when the angular gradient peak index is less than or equal to the preset angular gradient peak index threshold, a preset backup determination time window is used.

[0195] A violation determination module detects the position parameter of the front end of the vehicle driving towards the pedestrian crossing relative to the stop line and the pedestrian occupancy parameter of the entry area of the pedestrian crossing in the forensic determination time window or the backup determination time window and the angular neighborhood; when the pedestrian occupancy parameter meets the preset occupancy condition and the vehicle position parameter meets the preset line-crossing condition, a violation is output; otherwise, no violation is output.

[0196] The above describes the embodiments of the present embodiment, but the present embodiment is not limited to the specific embodiments described above, which are only illustrative and not limiting, and those skilled in the art can make many forms under the inspiration of the present embodiment, which are all within the protection scope of the present embodiment.​

Claims

1. A method for monitoring vehicle violations based on a city monitoring pole, characterized by, The method comprises the following steps: acquiring a target image; establishing a mapping relationship between image coordinates and world coordinates based on feature points of a stop line and a pedestrian crossing, constructing a polar coordinate system with an entry corner point of the pedestrian crossing as a pole point, and dividing angle fan regions, and limiting a detection zone with a preset height range in each angle fan region; acquiring a time sequence of the first appearance of pedestrians in the detection zone of each angle fan region, calculating a time difference value sequence between adjacent angle fan regions, and obtaining an angular gradient peak index and an angle fan position corresponding to a maximum peak based on the time difference value sequence; when the angular gradient peak index is greater than a preset angular gradient peak index threshold, determining an angular neighborhood at the angle fan position corresponding to the maximum peak, and generating a forensic judgment time window in combination with the first appearance time and the peak index; when the angular gradient peak index is less than or equal to the preset angular gradient peak index threshold, a preset backup judgment time window is used; in the forensic judgment time window or the backup judgment time window and the angular neighborhood, detecting a position parameter of the front end of a vehicle driving towards the pedestrian crossing relative to the stop line and a pedestrian occupancy parameter of an entry region of the pedestrian crossing; when the pedestrian occupancy parameter meets a preset occupancy condition and the vehicle position parameter meets a preset line-crossing condition, outputting a violation; otherwise, outputting no violation. The method for establishing a mapping relationship between image coordinates and world coordinates based on feature points of a stop line and a pedestrian crossing, constructing a polar coordinate system with an entry corner point of the pedestrian crossing as a pole point, and dividing angle fan regions, and limiting a detection zone with a preset height range in each angle fan region comprises the following steps:

2. The method of claim 1, wherein the method further comprises: acquiring four pairs of corresponding points of a stop line and a pedestrian crossing; each pair of corresponding points comprises an image coordinate and a world coordinate; the image coordinate is a coordinate on a target image, and the world coordinate is a coordinate on a road plane; a mapping relationship between the image coordinates and the world coordinates is established through a plane homography matrix; selecting an entry corner point of the pedestrian crossing, acquiring an image coordinate of the entry corner point, converting the image coordinate into a corresponding world coordinate through the plane homography matrix, and constructing a polar coordinate system with the world coordinate as a pole point; in the polar coordinate system, the polar distance of any point is the world distance of the point to the pole point, and the polar angle of any point is the world azimuth angle of the point relative to the pole point; setting a lower angle limit, an upper angle limit, an angle step, and an angle fan number of the polar coordinate system, dividing the interval of the polar angle into angle fan regions equal to the angle fan number according to the lower angle limit, the upper angle limit, and the angle step, and defining each angle fan region as a continuous angle range; setting a lower boundary pixel value and an upper boundary pixel value in the longitudinal pixel direction of the target image, and defining a detection zone in each angle fan region; the detection zone is a pixel set that is in the angle range of the angle fan region and between the lower boundary pixel value and the upper boundary pixel value in the longitudinal pixel direction. acquiring a time sequence of the first appearance of pedestrians in the detection zone of each angle fan region, and calculating a time difference value sequence between adjacent angle fan regions, comprising the following steps:

3. The method of claim 2, wherein the method further comprises: determining a timestamp set of a video; the timestamp set is a frame time set that is monotonically increasing in image acquisition order; for a detection zone corresponding to the mth angle fan region, determining a pedestrian pixel set composed of pixels in the detection zone that are determined to be pedestrians at each timestamp in the timestamp set. ​ A threshold of the number of pedestrian pixels is set, and the number of pixels in the set of pedestrian pixels at each timestamp is counted. If the number of pixels is greater than or equal to the threshold of the number of pedestrian pixels, the appearance indicator of the mth corner fan region at the timestamp is set to 1. If the number of pixels is less than the threshold of the number of pedestrian pixels, the appearance indicator is set to 0. From all the timestamps corresponding to the appearance indicator of 1 of the mth corner fan region, the smallest timestamp is selected as the first appearance time of the mth corner fan region. The first appearance times of the plurality of corner fan regions jointly constitute a time sequence of the first appearance of the pedestrian. The difference between the first appearance times of adjacent corner fan regions is calculated, specifically: the first appearance time of the next corner fan region minus the first appearance time of the previous corner fan region. The differences of the plurality of adjacent corner fan regions jointly constitute a time difference sequence between adjacent corner fan regions. The difference is calculated only when both adjacent corner fan regions have the first appearance time of the pedestrian.

4. The method of claim 3, wherein the method further comprises: Based on the time difference sequence, the angular gradient peak index and the angular fan position corresponding to the maximum peak are obtained, including: The absolute value of each time difference in the time difference sequence between adjacent corner fan regions is taken to obtain the absolute amount of angular difference corresponding to each adjacent corner fan region. The arithmetic mean of all absolute amounts of angular difference is calculated, and the number of absolute amounts of angular difference participating in the calculation is equal to the total number of corner fan regions minus one. The maximum absolute amount of angular difference is selected from all absolute amounts of angular difference, and the maximum absolute amount of angular difference is divided by the arithmetic mean of all absolute amounts of angular difference to obtain the angular gradient peak index. The count variable of the adjacent corner fan region corresponding to the maximum absolute amount of angular difference is determined, and the count variable is the angular fan position corresponding to the maximum peak.

5. The method of claim 4, wherein the method further comprises: When the angular gradient peak index is greater than a preset angular gradient peak index threshold, the angular neighborhood is determined based on the angular fan position corresponding to the maximum peak, and a forensic judgment time window is generated based on the first appearance time and the peak index, including: An angular neighborhood half-width parameter is set to determine the number of corner fan regions. A first angular neighborhood set for the first appearance time is defined, and the corner fan number in the first angular neighborhood set satisfies greater than or equal to the angular fan position corresponding to the maximum peak minus the angular neighborhood half-width parameter, less than or equal to the angular fan position corresponding to the maximum peak plus the angular neighborhood half-width parameter plus one, and the corner fan number is between one and the number of corner fan regions. A second angular neighborhood set for the absolute amount of angular difference is defined, and the corner fan number in the second angular neighborhood set satisfies greater than or equal to the angular fan position corresponding to the maximum peak minus the angular neighborhood half-width parameter, less than or equal to the angular fan position corresponding to the maximum peak plus the angular neighborhood half-width parameter, and the corner fan number is between one and the number of corner fan regions minus one. The first appearance time of each corner fan region and the absolute amount of angular difference of each adjacent corner fan region are obtained. The arithmetic mean of all absolute amounts of angular difference in the angular neighborhood set for the absolute amount of angular difference is calculated to obtain the average absolute amount of neighborhood difference, and the number of absolute amounts of angular difference participating in the calculation is equal to the number of elements in the angular neighborhood set. selecting a minimum first occurrence time from the first occurrence times of all the angular sector regions in the angular neighborhood set for the first occurrence time, to obtain a neighborhood first occurrence time; setting a time advance coefficient and a time retention coefficient; multiplying the angular gradient peak index by the time advance coefficient, and then multiplying the result by the neighborhood average difference absolute value, to obtain a time advance amount; multiplying the angular gradient peak index by the time retention coefficient, to obtain a time retention amount; taking the neighborhood first occurrence time as a reference, subtracting the time advance amount from the neighborhood first occurrence time to obtain a start time of the evidence determination time window, and adding the time retention amount to the neighborhood first occurrence time to obtain an end time of the evidence determination time window, so that the start time to the end time form the evidence determination time window.

6. The method of claim 5, wherein the method further comprises: when the angular gradient peak index is less than or equal to a preset angular gradient peak index threshold value, a preset backup determination time window is adopted, including: obtaining first occurrence times of all the angular sector regions and a number of the angular sector regions, and selecting a minimum first occurrence time from the first occurrence times of all the angular sector regions as a backup reference time; setting a backup time advance amount and a backup time retention amount; subtracting the backup time advance amount from the backup reference time to obtain a start time of the backup determination time window, and adding the backup time retention amount to the backup reference time to obtain an end time of the backup determination time window, so that the start time to the end time form the backup determination time window.

7. The method of claim 6, wherein the method further comprises: in the evidence determination time window or the backup determination time window and in the angular neighborhood, detecting a position parameter of a front end of a vehicle driving towards the pedestrian crossing relative to the stop line, and a pedestrian occupancy parameter of an entrance area of the pedestrian crossing, including: determining an effective time set, when the angular gradient peak index is greater than a preset angular gradient peak index threshold value, the effective time set is the evidence determination time window; when the angular gradient peak index is less than or equal to a preset angular gradient peak index threshold value, the effective time set is the backup determination time window; detecting only in the angular neighborhood in space; determining a straight line parameter of the stop line in the world coordinates, the straight line parameter being used to describe a world geometric shape of the stop line; at each time in the effective time set, obtaining a world coordinate point set of a vehicle contour located in the angular neighborhood; constructing a linear calculation formula based on the straight line parameter of the stop line, and selecting a point with a minimum result of the linear calculation formula from the world coordinate point set of the vehicle contour as a vehicle front end point; calculating a directional distance from the vehicle front end point to the stop line, and the directional distance is the position parameter of the vehicle front end relative to the stop line; setting the entrance area of the pedestrian crossing as a polygon area in the world coordinates, the polygon area defining a range of the pedestrian crossing entrance; at each time in the effective time set, obtaining a world coordinate point set of a pedestrian located in the angular neighborhood; calculating an area of an overlapping part of the world coordinate point set of the pedestrian and the polygon area of the entrance area of the pedestrian crossing, and dividing the area of the overlapping part by a total area of the polygon area of the entrance area of the pedestrian crossing, to obtain the pedestrian occupancy parameter of the entrance area of the pedestrian crossing.

8. The method of claim 7, wherein the method further comprises: when the pedestrian occupancy parameter meets a preset occupancy condition and the vehicle position parameter meets a preset crossing condition, outputting a violation; otherwise, outputting no violation, including: Determine an effective time set, when the angular gradient spike index is greater than a preset angular gradient spike index threshold, the effective time set is a forensic determination time window; when the angular gradient spike index is less than or equal to the preset angular gradient spike index threshold, the effective time set is a backup determination time window; Set a preset occupancy threshold and a preset cross-line tolerance, the preset occupancy threshold is the lower limit of the area proportion of the pedestrian occupancy in the entrance area of the pedestrian crossing, and the preset cross-line tolerance is a non-negative distance tolerance from the front end of the vehicle to the stop line; In each time in the effective time set, obtain the pedestrian occupancy parameter and the vehicle position parameter at the time; determine whether each time meets the first condition and the second condition at the same time; collect all times that meet the two conditions at the same time to form a determination set; and count the number of times in the determination set; wherein the first condition is that the pedestrian occupancy parameter is greater than or equal to the preset occupancy threshold, and the second condition is that the vehicle position parameter is less than or equal to the negative preset cross-line tolerance; When the number of times in the determination set is greater than or equal to one, it is determined that the violation; When the number of times in the determination set is equal to zero, it is determined that there is no violation.

9. A vehicle violation monitoring system based on urban monitoring pole, applied to the vehicle violation monitoring method based on urban monitoring pole in any one of claims 1-8, characterized in that, Comprise: A data acquisition module acquires a target image; A data processing module establishes a mapping relationship between image coordinates and world coordinates based on feature points of a stop line and a pedestrian crossing, constructs a polar coordinate system with an entrance corner point of the pedestrian crossing as a pole point, and divides angular sector regions, and limits a detection zone of a preset height range in each angular sector region; A data filtering module acquires a time sequence of the first appearance of pedestrians in the detection zone of each angular sector region, calculates a time difference value sequence between adjacent angular sector regions, and obtains an angular gradient spike index and an angular sector position corresponding to a maximum spike based on the time difference value sequence; A determination time window selection module, when the angular gradient spike index is greater than a preset angular gradient spike index threshold, determines an angular neighborhood at the angular sector position corresponding to the maximum spike, and generates a forensic determination time window in combination with the first appearance time and the spike index; when the angular gradient spike index is less than or equal to the preset angular gradient spike index threshold, a preset backup determination time window is used; A violation determination module detects the position parameter of the front end of the vehicle relative to the stop line and the pedestrian occupancy parameter of the entrance area of the pedestrian crossing in the forensic determination time window or the backup determination time window and the angular neighborhood; when the pedestrian occupancy parameter meets the preset occupancy condition and the vehicle position parameter meets the preset cross-line condition, output the violation; Otherwise, output no violation.

Images