A high-speed toll station non-motor vehicle and pedestrian high-speed intelligent early warning system

Abstract

The present application relates to the technical field of road warning, in particular to a kind of intelligent early warning system for non-motor vehicle and pedestrian on high-speed at high-speed toll station, comprising: the image of toll station is associated with road to be divided into multiple subspaces in topological connected order, and non-motor vehicle and pedestrian in each subspace are regarded as target object;Monitoring the moving track of target object in subspace, according to the track range involved in target object in subspace, deducing the intention label of moving track in each subspace, the intention label deduced is combined with moving track to be the track map of each target object;Judge the track vector under the track fitting of any target object, the track vector after fitting is oriented with space position, and the intention vector of each target object is set;The intention vector of target object is cross-identified, and the interaction point of the intention vector of each target object and the average error of moving track are obtained to output early warning result;The accuracy and efficiency of pedestrian intention recognition are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of road warning, in particular to an intelligent warning system for non-motor vehicles and pedestrians on a highway at a highway toll station. BACKGROUND

[0002] Among the participants in road traffic, in addition to vehicles, there are also pedestrians, non-motor vehicles and other vulnerable traffic participants. These vulnerable traffic participants, especially pedestrians, have greater freedom of movement and are less constrained, and are more likely to have traffic accidents. Once an accident occurs, these vulnerable traffic participants often suffer more serious injuries due to the lack of protective measures.

[0003] For example, Chinese Patent Publication No. CN114639245A discloses a vehicle and pedestrian collision warning method and device, which includes: constructing a vehicle entity model of a vehicle and a pedestrian entity model of a pedestrian; obtaining a vehicle heading angle of the vehicle, a pedestrian heading angle of the pedestrian, a vehicle speed of the vehicle and a pedestrian speed of the pedestrian in a northeast celestial coordinate system; determining at least one collision critical point according to the vehicle entity model and the pedestrian entity model, and determining a collision critical point coordinate value of the collision critical point according to the vehicle heading angle, the pedestrian heading angle, the vehicle speed and the pedestrian speed; and performing collision warning on the vehicle and the pedestrian according to the collision critical point coordinate value and a preset warning judgment rule.

[0004] For example, Chinese Patent Publication No. CN114267181A discloses a pedestrian warning method and device for urban road intersections and electronic equipment. The present application takes the speed of the pedestrian, the speed of the vehicle, the area of the pedestrian and the state of the driver as the basis for judgment to warn the risk of the pedestrian passing through the intersection, thereby outputting different collision warning levels to issue a collision warning prompt to the driver of the vehicle, and combining the light indicator at the intersection, the voice warning device and the zebra crossing warning light to issue a warning prompt to the pedestrian.

[0005] In the prior art, the predicted collision point coordinates are obtained by the circular coordinates of the pedestrian entity and the rectangular coordinates of the vehicle entity, and collision analysis is performed according to the speed components; and the pedestrian warning is realized by detecting the pixel distance between the pedestrian and the vehicle; however, in the prior art, the moving behavior intention of the vulnerable traffic participant is ignored, so that after identifying the predicted collision point and the pixel distance, timely warning cannot be performed according to this distance; in dangerous scenarios such as highway toll stations, the moving intention and trajectory of the pedestrian need to be analyzed to determine the direction of the pedestrian's intention, and a warning strategy needs to be set to prevent the vulnerable traffic participant from performing dangerous behaviors such as driving on the highway. SUMMARY

[0006] In order to solve the above technical problems, the technical scheme adopted by the present application is: a non-motor vehicle and pedestrian on high-speed intelligent early warning system of high-speed toll station, comprising: a target acquisition module, used for associating the toll station image with the road to be divided into multiple subspaces in topological connected order, and regarding the non-motor vehicles and pedestrians in each subspace as target objects.

[0007] A trajectory extraction module is used for monitoring the moving trajectory of the target objects in the subspace, deriving the intention label of the moving trajectory in each subspace according to the trajectory range involved in the subspace, and combining the derived intention label and the moving trajectory into a trajectory map of each target object.

[0008] A trajectory fitting module is used for judging the trajectory vector of any target object under trajectory fitting according to the intention label of the trajectory map, and setting the intention vector of each target object in the spatial position direction of the fitted trajectory vector.

[0009] An early warning recognition module is used for cross-recognizing the intention vector of the target object, obtaining the interaction point and the average error of the moving trajectory of the intention vector of each target object under cross analysis, and obtaining the output early warning result.

[0010] The beneficial effects of the present application are as follows: first, the present application divides the toll station image associated with the road into multiple subspaces in topological connected order, regards the non-motor vehicles and pedestrians in each subspace as target objects, monitors the moving trajectory of the target objects in the subspace, derives the intention label according to the trajectory range, combines into a trajectory map, then judges the trajectory vector according to the intention label of the trajectory map, sets the intention vector, finally cross-recognizes the intention vector, obtains the early warning result according to the interaction point and the average error, can accurately obtain the related information of the non-motor vehicles and pedestrians around the toll station, and warns through the moving intention of the pedestrians, reduces the safety risk caused by the non-motor vehicles and pedestrians on high-speed.

[0011] Second, the present application dynamically segments the non-motor vehicles and pedestrians in each subspace according to the topological relationship of each subspace in a continuous time axis, sets verification anchor points including time anchor points marked by the time axis, position anchor points associated with the position, and speed anchor points of the moving speed, determines the target objects in the cross-verification mode of the verification anchor points, implants the trajectory extraction mechanism in the obtained target objects, and classifies the target objects with consistent trajectories according to the subspace type. It can more accurately screen out target objects, provide accurate basis for subsequent trajectory extraction and analysis, and reduce the problem of moving recognition error caused by leaf shielding and flying birds.

[0012] Thirdly, the application maps the moving track to all involved subspaces, respectively processes single subspaces and multiple subspaces, determines the moving track of the target object in different subspaces, accurately deduces the intention label, fully reflects the moving intention of the target object, and provides sufficient data information for constructing the track map.

[0013] Fourthly, the application extracts the intention label in the track map, divides the moving track associated with the intention label at least once, forms multiple divided local moving tracks, applies disturbance to the local moving track, linearly fits the track vector, performs vector dot product between the track vector and the potential intention direction, extracts the intention vector, improves the accuracy of the analysis on the moving intention of the target object, detects the intersection point of the intention vector, constructs the trigger condition according to the subspaces to which the intersection point belongs and the average error between the moving track and the fitted moving track in the track map, and takes the pre-warning level corresponding to the trigger condition as the pre-warning result of the current output, which can accurately identify the potential dangerous situation, gives the corresponding pre-warning level according to the trigger condition, realizes timely and effective pre-warning, and reduces the safety risk. BRIEF DESCRIPTION OF DRAWINGS

[0014] The application will be further described below in combination with the drawings and embodiments.

[0015] Figure 1 It is a system framework diagram of the intelligent pre-warning system for non-motor vehicles and pedestrians on the expressway at the expressway toll station.

[0016] Figure 2 It is a flowchart of the target acquisition module of the intelligent pre-warning system for non-motor vehicles and pedestrians on the expressway at the expressway toll station.

[0017] Figure 3 It is a flowchart of the track extraction module of the intelligent pre-warning system for non-motor vehicles and pedestrians on the expressway at the expressway toll station.

[0018] Figure 4 It is a flowchart of the track fitting module of the intelligent pre-warning system for non-motor vehicles and pedestrians on the expressway at the expressway toll station.

[0019] Figure 5 It is a flowchart of the pre-warning identification module of the intelligent pre-warning system for non-motor vehicles and pedestrians on the expressway at the expressway toll station. DETAILED DESCRIPTION

[0020] The embodiments of the application are described in detail below. The embodiments described below are exemplary and are only used to explain the application, and cannot be understood as a limitation on the application. If the specific technology or condition is not indicated in the embodiments, the technology or condition described in the literature in the art or according to the product instruction is used.

[0021] Referring to Figure 1 A high-speed toll station non-motor vehicle and pedestrian high-speed intelligent early warning system, comprising: a target acquisition module, a trajectory extraction module, a trajectory fitting module and a warning recognition module; wherein the output end of the target acquisition module is connected with the trajectory extraction module, the output end of the trajectory extraction module is connected with the trajectory fitting module, and the output end of the trajectory fitting module is connected with the warning recognition module.

[0022] The target acquisition module is used to divide the toll station image associated with the road into multiple subspaces in topological connected order, and non-motor vehicles and pedestrians in each subspace are regarded as target objects.

[0023] The trajectory extraction module is used to monitor the moving trajectory of the target object in the subspace, derive the intention label of the moving trajectory in each subspace according to the trajectory range of the target object involved in the subspace, and combine the derived intention label and the moving trajectory into a trajectory map of each target object.

[0024] The trajectory fitting module is used to judge the trajectory vector of any target object under trajectory fitting according to the intention label of the trajectory map, and set the intention vector of each target object by guiding the spatial position of the fitted trajectory vector.

[0025] The warning recognition module is used to cross-identify the intention vector of the target object, and obtain the output warning result according to the interaction point of the intention vector of each target object and the average error of the moving trajectory under cross-analysis.

[0026] The above-mentioned subspace includes a toll station square space, a toll station downstream space and a toll station upstream space, the surrounding environment of the toll station is divided into multiple space regions according to the position of the toll station and the entrance and exit ramps, and the current region state of the toll station is described. Form multiple subspace regions under multiple categories such as warning area (square outside the toll station, surrounding auxiliary road), dangerous area (toll road, ETC lane, highway ramp), disposal area (toll station booth) and judge the moving trajectory, average speed and other parameters in each divided subspace. The divided subspace will divide multiple regional boundaries according to the actual shape of the toll road, ETC lane, toll station booth, square outside the toll station, etc., and extend the regions corresponding to these lanes to multiple positions in front and behind the toll station.

[0027] By arranging multiple groups of cameras, radars and laser radars and other devices at the toll station, the images of pedestrians and non-motor vehicles at multiple positions of the toll station are extracted, and image recognition algorithms such as YOLOv7 are used to recognize the trajectories of the current pedestrians and non-motor vehicles to complete subsequent processing.

[0028] The subspace division is based on GB5768.3-2009 "Road Traffic Signs and Markings". The toll plaza space is within 30m in front of and 20m behind the toll booth (physical coordinates X∈[-20, 30]m, Y∈[-10, 10]m, with the toll booth as the origin); the downstream space is 20-100m behind the toll booth (Y∈[-100, -20]m); the upstream space is 30-100m in front of the toll booth (Y∈[30, 100]m). The topological connectivity order is determined by the vehicle traffic direction of upstream space → plaza space → downstream space.

[0029] like Figure 2 As shown, the implementation of the target acquisition module includes: dynamically segmenting non-motorized vehicles and pedestrians in each subspace according to the topological relationship of each subspace using a continuous time axis, and setting verification anchor points for the dynamically segmented non-motorized vehicles and pedestrians, including time anchor points marked on the time axis, location anchor points associated with the location, and speed anchor points for the movement speed.

[0030] The verification anchors are verified sequentially using a cross-validation method, with time anchors, location anchors, and speed anchors being used as target objects. The non-motorized vehicles and pedestrians corresponding to the verified verification anchors are then used as target objects.

[0031] A trajectory extraction mechanism is embedded in the acquired target object. Target objects with consistent trajectories are classified according to subspace type, and the classified target objects are used as the output target objects.

[0032] The subspace type represents the warning zone, danger zone, and handling zone as described above, to complete the division of the area where the current toll station is located.

[0033] When classifying, the target objects under the three anchor points of time, position, and velocity are used as the main content for identification. The target images with consistent trajectories are identified. These represent multiple target objects extracted through the time anchor points, position anchor points, and velocity anchor points within a unit time period. At least two of these target objects have consistent trajectories. The target objects with consistent trajectories will be used as the output target objects to prevent identification errors due to a single dimension.

[0034] Preferably, target objects with trajectory consistency ≥ 90% are classified according to subspace type, and the classified target objects are used as the final output target objects. In this case, trajectory consistency is achieved by comparing the movement trajectory to which each verified anchor point belongs, comparing these movement trajectories one by one. Trajectory consistency represents the percentage value of the trajectory being consistent after comparison. This percentage is calculated by comparing the proportion of consistent coordinates of each trajectory point on the movement trajectory.

[0035] Preferably, when the above setting verifies the anchor point, the implementation manner comprises: sequentially setting time anchor points for the target objects in each subspace according to the time sequence of appearance, and setting position anchor points based on the moving path length of the target objects in each time period, and taking the average speed when setting the position anchor points as the speed anchor point of the corresponding target object.

[0036] When the verification anchor point is cross-verified, the three anchor points are verified step by step, and the implementation manner comprises: performing time continuity checking on the target objects based on the time anchor point, and for the toll station image of each subspace, if the current toll station image does not detect the target object in the continuous frame recognition, the toll station image of the adjacent subspace is called to update the time anchor point of the target object as the verified time anchor point. At this time, it is indicated that there is a frame of undetected pedestrians and non-motor vehicles, which emphasizes that a frame of disappearance appears in the image obtained in a subspace, which represents that the current pedestrian is far away from the toll station or the pedestrian moves to other subspaces. These pedestrians and non-motor vehicles belong to the currently mainly recognized objects, and prevent them from forcibly passing through the toll station.

[0037] When it is recognized that there is a frame of undetected corresponding non-motor vehicle or pedestrian in the continuous frame recognition, the toll station image of the adjacent subspace is called to find the time point corresponding to the pedestrian or non-motor vehicle, and after updating the time point to the currently labeled time anchor point, the verification of the time anchor point is completed.

[0038] Based on the position anchor point, the direction consistency of the target object is judged, and the position anchor point meeting the direction consistency is regarded as the verified position anchor point. The position anchor point is used to emphasize whether the moving position of the pedestrian at the boundary of multiple subspaces maintains the direction consistency, and the direction consistency is used to filter the change of the moving direction of the pedestrian in the continuous frame recognition, so as to remove the obviously unreasonable trajectory, for example, the moving trajectory recognized at adjacent time points in the continuous time period has a direction angle difference of 180°. This case belongs to unreasonable moving trajectory and needs to be removed, so as to ensure that the currently recognized moving trajectory is correct data, and the corresponding pedestrian and non-motor vehicle are regarded as the target object.

[0039] When the direction consistency is verified, multiple continuous trajectory point coordinates labeled by the current position anchor point are extracted, the continuous trajectory point coordinates are composed into a direction vector, and the average change angle of the direction vector in the continuous frame is judged. When the average change angle is less than or equal to 30 degrees, it is indicated that it meets the normal moving rule; and when the average change angle is greater than 30°, it is considered that the current pedestrian movement is in an irregular moving situation, and the data needs to be continuously extracted and the position anchor point needs to be updated to determine the moving direction of the pedestrian relative to the position such as the lane.

[0040] It should be noted that the currently set subspace will divide the area before and after the toll station into multiple small subspaces according to the images captured by the camera, to illustrate the movement of pedestrians and non-motorized vehicles in each lane, auxiliary lane, toll booth, etc. If it is regarded as a verified position anchor point, it means that non-motorized vehicles and pedestrians are moving along the edge of the lane or moving to other lanes and toll booths, etc. In this case, it is necessary to track the position of pedestrians and non-motorized vehicles, treat them as target objects, and determine the movement trajectory of pedestrians and non-motorized vehicles.

[0041] Based on speed anchor points, the average speed range of the target object is determined. Data exceeding the upper limit of the range is filtered, and the corresponding speed anchor points are considered validated. Speed ​​anchor points are biased towards validating excessively fast-moving segments. For example, the average speed range of pedestrians and non-motorized vehicles under normal road conditions is used as the main part of the current validation. This is achieved by generating a confidence interval, such as a range of average speed ± 3 standard deviations, and filtering out anchor points for pedestrians and non-motorized vehicles that exceed the upper limit of this range to prevent the currently identified trajectory from being erroneous data such as birds in flight.

[0042] In one embodiment of the present invention, the trajectory extraction module emphasizes the movement trajectory of the target object, and combines the target object's intent label and movement trajectory into a global trajectory map according to the subspace to which the movement trajectory belongs.

[0043] Preferably, the trajectory map is constructed by: based on the movement trajectory of the target object, selecting the center point of the movement trajectory as the center of a circle, using the smallest circle enclosing the movement trajectory as the trajectory range of the current target object, determining the trajectory range within each subspace of the target object, and constructing the trajectory map of each target object based on the position of the trajectory range within each subspace. When constructing the trajectory map, the acquired images are arranged according to the shooting position, using the X and Y axes, with the center of the current toll station as the origin, the direction perpendicular to the toll station as the Y-axis, and the direction horizontal to the toll station as the X-axis, to obtain a horizontal coordinate system for the trajectory map.

[0044] The center point of the aforementioned movement trajectory can be selected by averaging the coordinates of multiple points along the X and Y axes. Then, the smallest enclosing circle encompassing the movement trajectory is calculated based on this center point. The trajectory range within multiple subspaces is calculated separately for each subspace containing the movement trajectory.

[0045] like Figure 3As shown, the trajectory extraction module monitors the movement trajectory of the target object in the subspace. Its implementation method also includes: tracking the movement trajectory of the target object in real time and mapping the movement trajectory to all involved subspaces, and performing step-by-step processing for a single subspace and centralized processing for multiple subspaces respectively.

[0046] When the target object involves a subspace, the movement trajectory of the target object is judged to determine the trajectory state of the target object, and the decision data of the target object is derived according to the trajectory state to obtain the intent label corresponding to the decision data.

[0047] When the target object involves a subspace, it means that the movement path of pedestrians and non-motorized vehicles is relatively unique, such as when a pedestrian takes a card, a non-motorized vehicle accidentally enters a toll station, or when pedestrians and non-motorized vehicles are gradually approaching the toll station without taking any further action.

[0048] When the target object involves multiple subspaces, for any subspace type, the area ratio of the current target object's trajectory range in each subspace is used as the input parameter. At the same time, the decision data of the target object is derived by combining the weights assigned by the subspace type, and the intent label corresponding to the decision data is obtained.

[0049] When multiple subspaces are involved, it means that the currently identified pedestrian is walking in multiple areas, such as passing through the areas corresponding to multiple subspaces such as the toll plaza outside the toll station, the surrounding auxiliary roads, the toll lanes, the ETC lanes, the highway ramps, and the toll booths. In this case, it is necessary to determine whether the pedestrian is walking normally based on the situation of walking in multiple spaces, and further determine the form of their intention. The same applies to non-motorized vehicles. The trajectory range under multiple areas is used to further identify the behavior status, in order to determine whether there is any behavior of entering the highway.

[0050] The decision data includes the average speed, dwell time, subspace identifier, and coordinates and orientation angles corresponding to the trajectory shape of each target object during trajectory tracking.

[0051] After completing the derivation of decision data for multiple subspaces and a single subspace, all decision data are summarized, and the movement trajectories corresponding to the summarized decision data are combined according to the derivation intent labels to form the current output trajectory map.

[0052] At this point, the decision data will normalize the current data and combine the corresponding movement trajectory with the intent label to form the current trajectory map.

[0053] Preferably, when the target object involves a subspace, the processing method includes: identifying the trajectory state of the target object based on its dwell time, trajectory shape, average speed, and orientation angle relative to the subspace boundary; the dwell time is used to mark the dwell point of pedestrians, etc., during movement, as well as the length of time they stay; the orientation angle indicates the angle with the subspace boundary, and indicates the relative position with respect to the lane or toll plaza.

[0054] Based on the trajectory shape, the subspace type pointed to by the current target object is determined. The dwell point, average speed and direction angle under each subspace type are used as derivation rules. The decision rules labeled with historical data are applied to determine the trajectory state of the current user's movement. The intent label corresponding to the trajectory state is output as decision data.

[0055] For example, the decision rule can be expressed as: IF Average speed = V1 (representing any speed) AND Direction pointing to the lane (described using the subspace type pointed to by the direction angle) THEN Intent = Potential intrusion; This potential intrusion refers to the current user moving towards a relatively dangerous area in a single subspace. If their stop point is a toll booth and the stop time is greater than a certain time, then the intent can be considered to be to take a card; and in some scenarios, when the average speed value is small and the shape represented by the movement trajectory is a hesitant shape, then the intent can be directly pointed to as hesitant or observing; at this time, by taking the stop point, average speed and direction angle as the main derivation targets, the decision rule corresponding to the data is found, and the intent label derived from the decision rule is added to the output decision data.

[0056] The trajectory status here is used to illustrate a preliminary intent derived from multiple values ​​to identify the intent bias of pedestrians identified at the current highway toll station. This intent bias data will be used as output decision data to find the intent label that is closest to the current movement trajectory from the database.

[0057] The intent labeling involves a subspace, and the main data for obtaining the intent label is the dwell point, average speed, and direction angle. The intent labels that conform to the current inference rules are used as the output decision data.

[0058] Preferably, when the target object involves multiple subspaces, the implementation method includes: calculating the area ratio of the trajectory range under each subspace, performing weighted summation based on the weight of the subspace type to obtain a risk score, using the risk score, the area ratio of the trajectory range, and the subspace type as derivation rules, applying decision rules labeled with historical data, and determining the decision data of the target object using the intent label corresponding to the decision rules.

[0059] The weights of the aforementioned subspace types will be set according to the specific type of the subspace. A preset weight will be set for each type of warning zone, danger zone, and response zone. This weight will be configured based on historical data, such as setting weights of 0.35, 0.5, and 0.15 in sequence. Then, the weight division will be refined according to the specific content of the warning zone, danger zone, and response zone, such as lanes and ramps.

[0060] The decision rules used at this time will identify the pedestrian's complete movement trajectory in multiple subspaces. Unlike the processing method for a single subspace, the comprehensive explanation under the analysis of multiple subspaces is the comprehensive intent, which emphasizes the summarization of the pedestrian's movement trajectory status in multiple subspaces. The output decision data will summarize the data in these two cases to illustrate a more complete pedestrian intent recognition.

[0061] The rules derived here are based on the risk score, combined with the area ratio of the trajectory range and the subspace type, to explain whether the current pedestrian's intention belongs to a high-risk intrusion, an emergency situation, or other trajectory states.

[0062] For example, there is a decision rule that can be expressed as follows: risk score ≥ 0.7; area ratio of the danger zone trajectory range ≥ 0.6; subspace type shows that the pedestrian quickly and directly enters the danger zone from the warning zone without obvious deceleration or observation behavior. In this case, the inferred intention is a high-risk intrusion intention.

[0063] At this point, the risk score reaches 0.7 or higher, indicating a relatively dangerous situation overall. Subsequently, if there is a large proportion of the trajectory area in the toll lanes and other locations marked in the danger zone, it indicates that the pedestrian is moving around a lot in the danger zone and has entered the danger zone directly through the warning zone. At this point, the target may pose a threat to the normal operation of the toll station, and it is necessary to record their intentions and generate a corresponding trajectory map.

[0064] Other decision-making rules will be predefined in the database to assist in the subsequent analysis of the behavioral intentions of pedestrians and non-motorized vehicles.

[0065] When multiple subspaces are involved, the output decision data format is the same as the output data format when a single subspace is involved. The difference is the intent described under the decision rule. If the target object is the same when analyzing one subspace at a time and analyzing multiple subspaces together, that is, when the movement trajectory of a pedestrian at a toll station is divided into two processing methods, one analysis of one subspace at a time and analysis of multiple subspaces together, the intent identified by both methods will be marked in the trajectory map.

[0066] The subsequent method for aggregating the decision data from the two approaches involves tracking the trajectory of each target object in real time and combining the intent labels marked under separate subspace analysis with the intent labels under centralized analysis of multiple subspaces as the intent labels for each target object. Each movement trajectory will be labeled with an intent label derived through decision rules to illustrate the movement trajectory of pedestrians and non-motorized vehicles.

[0067] In one embodiment of the present invention, a trajectory fitting module is used to receive a trajectory map from a trajectory extraction module and extract intent labels set according to decision rules. Then, these intent labels are fitted to the movement trajectory. The fitting method includes, but is not limited to, trajectory fitting for the same intent label, trajectory fitting for similar intent labels, and combining multiple intent labels identified in the current trajectory map to describe the movement intent of pedestrians and non-motorized vehicles near the toll station. Based on the movement intent, warnings or other types of intent recognition are then given to pedestrians.

[0068] like Figure 4 As shown, the implementation of the trajectory fitting module includes: extracting the intent labels marked in the trajectory map, segmenting the movement trajectory associated with the intent labels at least once to form multiple segmented local movement trajectories; when segmenting, the segmentation is based on the stationary time and hovering time of the target object during movement, or based on the length of the target object's movement trajectory for equal-sized segmentation.

[0069] A perturbation is applied to each local movement trajectory, and a random error is added to the coordinate values ​​of the local movement trajectory to form multiple perturbation trajectory points.

[0070] The purpose of applying perturbation is to simulate situations where the target is briefly occluded or detection fails. This causes the path trajectory to depend on the overall movement trend when it is being identified and processed in situations where there are obstructions such as leaves, various noises, and uncertainties. At the same time, this method also simulates the effects of non-uniform motion of pedestrians and non-motorized vehicles or changes in camera shooting angle, so that subsequent trajectory processing can cope with these changes and improve the accuracy of intention recognition of movement trajectory as much as possible.

[0071] When applying a disturbance, random errors are introduced into the coordinate values ​​of the moving trajectory, subtly altering the coordinates of points on the trajectory while maintaining the overall shape of the trajectory. The added random error can be selected from normally distributed random numbers with a mean of 0 and a standard deviation of σ. σ represents the intensity of the added noise and can be set based on the current error level of the camera device in recognizing pedestrians and non-motorized vehicle trajectories. For ordinary CMOS cameras, σ typically ranges from 0.2 to 0.4 meters, while for LiDAR, it ranges from 0.05 to 0.1 meters. The dimension of the random error being added is adjusted based on these values.

[0072] When applying the above-mentioned perturbation, in addition to directly perturbing according to the random error of the normal distribution, a certain proportion of trajectory points can be randomly discarded to form a perturbed curve or multiple trajectory points. Then, elastic deformation perturbation can be used to change the position of the entire local movement trajectory by stretching the curve.

[0073] The perturbed trajectory points are linearly fitted to the segmented local movement trajectories until all local movement trajectories are fitted, resulting in the fitted trajectory vector of the overall movement trajectory. In other words, after applying perturbation to all segmented local movement trajectories, they are combined with the perturbed trajectory points to reassemble the overall movement trajectory.

[0074] When fitting, linear least squares fitting can be used to fit the perturbation trajectory points and the trajectory points of the segmented local movement trajectory into a straight line. This fitting method is suitable when the curvature of the local fitting curve is low and the movement trajectory is monotonous, such as a straight movement trajectory. For more complex movement trajectories, such as a movement trajectory that moves back and forth while hovering, polynomial fitting can be used to identify the perturbation at the turning point and fit it into a curve. Then, principal component analysis is performed on the fitted curve to determine the orientation of the trajectory vector in space.

[0075] During principal component analysis, the main direction of motion of the local movement trajectory is obtained by treating the points in the segmented local movement trajectory as a point cloud. The trajectory vector that conforms to the perturbation deviation is used as the output trajectory vector, and the intention label corresponding to the movement trajectory is labeled.

[0076] Linear least squares fitting finds a straight line Find its slope and intercept This minimizes the sum of the squares of the perpendicular distances from all points to this line, where x and y represent the coordinates of each trajectory point. The coordinates of the trajectory points described here include the trajectory points of the segmented local movement trajectory and the trajectory points with applied perturbation, and the number of these trajectory points is denoted as N.

[0077] The slope is calculated as follows: The slope calculated at this point represents the direction of the trajectory vector after the current linear least squares fitting. Alternatively, the current trajectory vector can be normalized to form a unit vector, showing the specific form of the current trajectory vector and reflecting the average motion direction of this local moving set.

[0078] The intercept value is expressed as follows: By outputting this data, the trajectory vector processing is completed.

[0079] When fitting a polynomial, the solution is obtained using a quadratic function, such as... In the form of a, b, and c, the coefficients a, b, and c are obtained using the least squares method. The polynomial will then be fitted to a curve. The last point is selected as the key point, and the first derivative of the slope of the curve is used to construct a trajectory vector representing the direction. The fitted trajectory vector emphasizes the instantaneous direction of movement and better reflects the behavior that pedestrians are about to perform.

[0080] In principal component analysis, the coordinates of all points are subtracted from their average value so that the center of the point cloud is located at the origin. Then, a covariance matrix is ​​set for all points, and the eigenvalues ​​and eigenvectors of the covariance matrix are solved. The eigenvector corresponding to the largest eigenvalue is the first principal component, which represents the direction of the largest data variance, i.e., the main extension direction of the trajectory.

[0081] At this point, three methods are used to fit the perturbed trajectory points and the segmented, undisturbed trajectory points to obtain a trajectory vector that satisfies the overall movement trend of the pedestrian.

[0082] It should be noted that after fitting the trajectory points and forming the trajectory vector using the above method, the trajectory vector will be normalized to form a unit vector.

[0083] The fitted trajectory vector is multiplied by the latent intention direction to determine the projection coefficients in the latent intention direction. The latent intention direction with the largest projection coefficient is taken as the output intention vector.

[0084] The potential intent direction is a preset vector. A unit vector is set for each of the following directions: the direction pointing to the highway entrance, the direction pointing to the toll booth window, the direction pointing to the toll station exit or leaving the lane. These processing methods inject perturbation deviations into the direction of the trajectory after the trajectory extraction module processes the trajectory, in order to improve the handling of anomalies in the initial trajectory extraction caused by changes in camera angle and obstructions such as leaves.

[0085] As for the projection coefficient, it is the result of the dot product of the trajectory vector and the potential intention direction. The dot product method is to calculate the cosine similarity between the normalized trajectory vector and the potential intention direction. The larger the cosine value obtained by the dot product, the closer the direction of the trajectory vector is to the preset intention direction, so as to further limit the specific intention of the pedestrian. The potential intention direction at this time is then added to the intention label of the original movement trajectory to complete the analysis and processing in multiple cases.

[0086] In one embodiment of the present invention, the warning recognition module determines the aggregation or potential conflict point of pedestrians among multiple segmented movement trajectories by cross-identifying the intent vectors; at this time, the intent vectors can be extended to determine the location where multiple pedestrians and non-motorized vehicles tend to gather when moving, and this location may lead to traffic risks or group gathering phenomena in different scenarios, which is the main warning monitoring location of the current toll station.

[0087] For example, if the intersection of the intention vectors of two pedestrians in front of the toll station is close to the card dispenser, multiple pedestrians will temporarily gather at the card dispenser. At this time, it is necessary to observe the gathering situation to prevent other potential problems. If there is an intersection of the intention vectors of pedestrians and non-motorized vehicles, it indicates that collisions or interference may occur. If the intersection of the intention vectors of pedestrians and non-motorized vehicles corresponds to a vehicle, there may be a traffic safety hazard. In this case, it is still necessary to monitor the corresponding location point. Then, it is necessary to determine the minimum error value between the intersection of the intention vectors and the current identified movement trajectory to further identify whether there is a risk of intersection between pedestrians and non-motorized vehicles during movement and issue relevant warnings.

[0088] like Figure 5 As shown, the implementation of the early warning recognition module includes: for each group of target objects, detecting the intersection point of intent vectors, and labeling the intersection point according to the subspace to which the intersection point belongs; at this time, the interaction points between intent vectors are obtained by extending the intent vectors, and then the interaction points are labeled according to the type of subspace to which the interaction points fall. If the current interaction point falls into the danger zone, it is marked as a potential conflict.

[0089] By combining the marked intersection points with the average error between the moving trajectory in the trajectory map and the fitted moving trajectory, trigger conditions are constructed, and the warning level corresponding to the trigger conditions is used as the current warning result.

[0090] If there is only one target object at this time, that is, only one intention vector, or if there is no direct intersection point between each group of target objects, the point extended to the subspace by the intention vector is regarded as the intersection point, which indicates the current movement tendency and intersection position of pedestrians and non-motorized vehicles.

[0091] When calculating the average error, the average error is obtained by comparing the actual movement trajectory collected in the trajectory map with the movement trajectory after adding perturbations and fitting. This determines whether the interaction point of the current intent vector can correspond to the actual movement trajectory, so as to set early warnings for the movement of pedestrians and non-motorized vehicles in a timely manner.

[0092] When setting trigger conditions, set a condition based on the subspace type to which the interaction point belongs, and retrieve the average error to check if the average error is less than a certain set threshold. The threshold set here is based on the accuracy of identifying pedestrian movement trajectory points after the current camera captures the image. Under normal conditions, 0.5m, 1.0m and 1.5m can be used as the threshold for setting trigger conditions.

[0093] The triggering conditions can be illustrated by the following example: when the intersection is within the lane and the average error is <0.5m, it is considered a Level 1 warning, and a combination of audible and visual alarms and a linked barrier gate is used; when the intersection is within the warning zone and the average error is <1.0m, it is considered a Level 2 warning, and LED warnings and voice prompts are issued; if the average error is >1.5m, it is considered a Level 3 warning, at which point it is necessary to re-predict and update the pedestrian's movement trajectory in real time to determine the pedestrian's intention.

[0094] It should be noted that the triggering conditions are set in advance based on the situation at the highway toll station, and the warning level and measures corresponding to each triggering condition are explained, so as to provide a real-time warning response based on the movement intentions of pedestrians, etc.

[0095] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention, which are still covered within the protection scope of the present invention.

Claims

1. A high-speed toll station non-motor vehicle and pedestrian on high-speed intelligent early warning system, characterized in that, The method comprises the following steps: The target acquisition module is used for associating the toll station image with the road to divide the toll station image into a plurality of subspaces in a topological connected order, and non-motor vehicles and pedestrians in each subspace are regarded as target objects; The trajectory extraction module is used for monitoring the moving trajectory of the target objects in the subspace, deriving an intention label of the moving trajectory in each subspace according to the trajectory range of the target objects in the subspace, and combining the derived intention label and the moving trajectory to obtain a trajectory map of each target object; The trajectory fitting module is used for judging the trajectory vector of any target object under trajectory fitting according to the intention label of the trajectory map, guiding the spatial position of the fitted trajectory vector, and setting an intention vector of each target object; The early warning recognition module is used for cross-recognizing the intention vector of the target object, and obtaining an output early warning result according to the interaction point of the intention vector of each target object and the average error of the moving trajectory under cross analysis.

2. The intelligent warning system for non-motor vehicles and pedestrians on expressway at expressway toll station according to claim 1, characterized in that, The implementation of the target acquisition module comprises the following steps: The non-motor vehicles and pedestrians in each subspace are dynamically segmented in a continuous time axis according to the topological relationship of the subspaces, and verification anchor points are set for the dynamically segmented non-motor vehicles and pedestrians, including time anchor points marked by the time axis, position anchor points associated with the position, and speed anchor points of the moving speed; The verification anchor points are verified in the time anchor points, the position anchor points and the speed anchor points in sequence according to the cross verification, and the non-motor vehicles and pedestrians corresponding to the verified verification anchor points are taken as the target objects; The trajectory extraction mechanism is implanted in the obtained target objects, the target objects with consistent trajectories are classified according to the subspace type, and the classified target objects are taken as the output target objects.

3. The intelligent warning system for non-motor vehicles and pedestrians on expressway at expressway toll station according to claim 2, characterized in that, When the verification anchor points are set, the implementation comprises the following steps: The target objects in each subspace are sequentially set with the time anchor points according to the time sequence of their appearance, the position anchor points are set based on the moving path length of the target objects in each time period, and the average speed when the position anchor points are set is taken as the speed anchor point of the corresponding target object.

4. The intelligent warning system for non-motor vehicles and pedestrians on expressway of expressway toll station according to claim 2, characterized in that, When the cross verification of the verification anchor points is performed, the implementation comprises the following steps: The time continuity of the target objects is checked based on the time anchor points, for the toll station image of each subspace, if there is a frame in which the target object is not detected in the continuous frame recognition of the current toll station image, the toll station image of the adjacent subspace is called to update the time anchor point of the target object as the verified time anchor point; The direction consistency of the target objects is judged based on the position anchor points, and the position anchor points meeting the direction consistency are regarded as the verified position anchor points; The value interval of the average speed of the target objects is judged based on the speed anchor points, the data exceeding the upper limit value of the value interval is filtered, and the corresponding speed anchor point is regarded as the verified speed anchor point.

5. The intelligent warning system for non-motorized vehicles and pedestrians on expressways according to claim 1, characterized in that, When the trajectory map is constructed, the implementation comprises the following steps: Based on the moving trajectory of the target objects, the center point of the moving trajectory is selected as the center of the circle, the smallest circle surrounding the moving trajectory is taken as the trajectory range of the current target object, the trajectory range of the target objects in each subspace is determined, and the trajectory map of each target object is constructed based on the position of the trajectory range in each subspace.

6. The intelligent warning system for non-motor vehicle and pedestrian on expressway at expressway toll station according to claim 2, characterized in that, In the trajectory extraction module, the moving trajectory of the target objects in the subspace, the implementation further comprises the following steps: Real-time tracking of the moving trajectory of the target object, and mapping the moving trajectory to all involved subspaces, respectively, for single subspace step-by-step processing and multiple subspace centralized processing; When the target object involves a subspace, the moving trajectory of the target object is judged to determine the trajectory state of the target object walking, and the decision data of the target object is derived according to the trajectory state to obtain the intention label corresponding to the decision data; When the target object involves multiple subspaces, for any one subspace type, the area proportion of the trajectory range of the current target object in each subspace is taken as an input parameter, and the decision data of the target object is derived by combining the weight given by the subspace type, to obtain the intention label corresponding to the decision data; After completing the derivation of the decision data of multiple subspaces and single subspaces, all decision data is summarized, and the moving trajectory corresponding to the summarized decision data is combined according to the derived intention label to form the current output trajectory map.

7. The intelligent warning system for non-motor vehicles and pedestrians on expressway of expressway toll station according to claim 6, characterized in that, When the target object involves a subspace, the processing mode includes: According to the residence time, trajectory shape, average speed and direction angle relative to the subspace boundary of the target object during movement, the trajectory state of the target object is identified; Based on the trajectory shape, the subspace type pointed to by the current target object is judged, and the residence point, average speed and direction angle under each subspace type are taken as derivation rules, the decision rules labeled by historical data are applied, the trajectory state of the current user movement is determined, and the intention label corresponding to the trajectory state is output as the decision data.

8. The intelligent warning system for non-motorized vehicles and pedestrians on expressways at expressway toll stations according to claim 6, characterized in that, When the target object involves multiple subspaces, the implementation mode includes: The area proportion of the trajectory range under each subspace is calculated, and the risk score is obtained by weighted summation according to the weight of the subspace type, the risk score, the area proportion of the trajectory range and the subspace type are taken as derivation rules, the decision rules labeled by historical data are applied, and the intention label corresponding to the decision rule is used to determine the decision data of the target object.

9. The intelligent warning system for non-motorized vehicles and pedestrians on expressways according to claim 1, wherein, The implementation mode of the trajectory fitting module includes: Extracting the intention label labeled in the trajectory map, segmenting the moving trajectory associated with the intention label at least once to form multiple segmented local moving trajectories; Adding random errors to the coordinate values of the local moving trajectories to form multiple perturbed trajectory points; Linearly fitting the perturbed trajectory points with the trajectory points of the segmented local moving trajectories until all local moving trajectories are fitted to obtain the fitted trajectory vector of the overall moving trajectory; Vector dot product of the fitted trajectory vector and the potential intention direction is performed to determine the projection coefficient in the potential intention direction, and the potential intention direction with the maximum projection coefficient value is taken as the output intention vector.

10. The intelligent warning system for non-motorized vehicles and pedestrians on expressways according to claim 1, wherein, The implementation mode of the pre-warning recognition module includes: For each group of target objects, the intersection point of the intention vector is detected, and the intersection point is labeled according to the subspace to which the intersection point belongs; The labeled intersection point is combined with the average error of the moving trajectory and the fitted moving trajectory in the trajectory map to construct a trigger condition, and the pre-warning level corresponding to the trigger condition is taken as the current output pre-warning result.

Images