Vehicle abnormal behavior early warning method and device, equipment, storage medium

Abstract

The application provides a vehicle abnormal behavior early warning method and device, equipment and a storage medium, and belongs to the technical field of intelligent transportation. The method comprises the following steps: acquiring multi-source vehicle perception data, determining a vehicle trajectory sequence based on the multi-source vehicle perception data; extracting vehicle trajectory center point coordinates and instantaneous speed vectors from the vehicle trajectory sequence, performing smoothing processing on the vehicle trajectory center point coordinates and the instantaneous speed vectors, and obtaining standardized trajectory data; identifying vehicle abnormal behavior based on the standardized trajectory data and a pre-constructed polygon region model in combination with event determination thresholds corresponding to multiple event types; the polygon region model is a vector geometry model constructed according to road site calibrated control region boundary coordinates; and generating early warning information based on the identified vehicle abnormal behavior. The application can realize accurate identification and early warning of multiple types of vehicle abnormal behavior under multi-source data fusion.

Description

Technical Field

[0001] This application belongs to the field of intelligent transportation technology, and more specifically, relates to methods, devices, equipment, and storage media for early warning of abnormal vehicle behavior. Background Technology

[0002] With the continuous expansion of road networks, including highways, urban arterial roads, and tunnels, traffic volume is constantly increasing, leading to frequent occurrences of abnormal vehicle behavior. This can easily trigger traffic accidents and congestion, placing higher demands on road safety and the efficiency of traffic management and control. Abnormal vehicle behavior refers to the movement or encroachment of vehicles that deviate from road traffic rules during operation. Timely identification and early warning of such behavior are crucial foundational elements of intelligent traffic management.

[0003] Currently, the main monitoring and early warning solutions for abnormal vehicle behavior are automated machine monitoring. Automated machine monitoring relies on sensing devices to acquire relevant vehicle data, combines preset rules to identify abnormal vehicle behavior, and outputs alarms, which has become the mainstream development direction of smart transportation.

[0004] However, existing automated monitoring solutions mostly only identify single types of abnormal behavior and rely on perception data from a single source to complete the analysis and judgment. In complex road scenarios, they are easily affected by factors such as occlusion and detection errors, resulting in low recognition accuracy. Furthermore, they cannot simultaneously cover different types of abnormal vehicle behavior, making it difficult to meet the unified early warning requirements for multiple scenarios and types of abnormal behavior. Summary of the Invention

[0005] The purpose of this application is to provide a method, device, equipment, and storage medium for early warning of abnormal vehicle behavior, so as to solve the problem of low accuracy in abnormal behavior identification in the prior art.

[0006] A first aspect of this application provides a method for early warning of abnormal vehicle behavior, including: Acquire multi-source vehicle perception data and determine vehicle trajectory sequences based on the multi-source vehicle perception data; The coordinates of the center point of the vehicle trajectory and the instantaneous velocity vector are extracted from the vehicle trajectory sequence. The coordinates of the center point of the vehicle trajectory and the instantaneous velocity vector are then smoothed to obtain standardized trajectory data. Based on standardized trajectory data and a pre-built polygonal region model, abnormal vehicle behavior is identified by combining event judgment thresholds corresponding to multiple event types; the polygonal region model is a vector geometric model constructed based on the boundary coordinates of the control area calibrated on the road. Early warning information is generated based on the identified abnormal vehicle behavior.

[0007] A second aspect of this application provides a vehicle abnormal behavior warning device, comprising: The trajectory sequence construction module is used to acquire multi-source vehicle perception data and determine vehicle trajectory sequences based on the multi-source vehicle perception data. The data processing module is used to extract the coordinates of the center point of the vehicle trajectory and the instantaneous velocity vector from the vehicle trajectory sequence, and to smooth the coordinates of the center point of the vehicle trajectory and the instantaneous velocity vector to obtain standardized trajectory data. The abnormal behavior recognition module is used to identify abnormal vehicle behavior based on standardized trajectory data and a pre-built polygonal region model, combined with event judgment thresholds corresponding to multiple event types; the polygonal region model is a vector geometric model constructed based on the boundary coordinates of the control area calibrated on the road. The early warning module is used to generate early warning information based on the identified abnormal vehicle behavior.

[0008] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described vehicle abnormal behavior warning method.

[0009] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described vehicle abnormal behavior warning method.

[0010] The beneficial effects of the vehicle abnormal behavior early warning method, device, equipment, and storage medium provided in this application are as follows: This application embodiment acquires multi-source vehicle perception data and generates vehicle trajectory sequences. The multi-source data complements and verifies each other, solving the data deviation problem caused by a single perception data source. By performing smoothing processing on the trajectory center point coordinates and instantaneous velocity vectors to obtain standardized trajectory data, the sampling noise inherent in the original perception data can be reduced, providing higher quality basic motion data for subsequent anomaly detection.

[0011] This application embodiment uses a polygonal region vector geometric model obtained by on-site calibration of the control boundary, combined with the judgment thresholds corresponding to various events. Based on the same set of standardized trajectory data, it can be compatible with the identification of different types of abnormal vehicle behaviors such as motion deviation and area intrusion. There is no need to build an independent data processing link for each type of abnormal behavior. Finally, warning information is generated uniformly based on the identified abnormal vehicle behaviors, realizing the integrated output of multiple types of abnormal events.

[0012] Therefore, the embodiments of this application can be adapted to multiple road management business scenarios, improve the reliability of vehicle abnormal behavior identification, meet the actual business needs of unified early warning of multiple scenarios and types of abnormal behaviors on roads, facilitate traffic management personnel to obtain various traffic abnormality early warnings in a timely manner, and improve the work efficiency of road safety management. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart illustrating a vehicle abnormal behavior early warning method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the boundary division of the control area provided in an embodiment of this application; Figure 3 This is a structural block diagram of a vehicle abnormal behavior warning device provided in an embodiment of this application; Figure 4 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0015] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0016] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0017] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a vehicle abnormal behavior warning method according to an embodiment of this application. The method can be executed by an electronic device and may include: S101: Acquire multi-source vehicle perception data and determine vehicle trajectory sequences based on the multi-source vehicle perception data.

[0018] In this embodiment, the multi-source vehicle perception data includes multi-frame data streams, checkpoint vehicle passage data, and vehicle positioning data; determining the vehicle trajectory sequence based on the multi-source vehicle perception data includes: Based on multi-frame data streams, vehicle targets are identified through a target detection model. Cross-frame identity matching is performed on the identified multiple vehicle targets, and a tracking identifier is assigned to each vehicle target according to the identity matching results. Based on the tracking identifier, the trajectory association of each vehicle target is extracted to obtain an initial trajectory point set including the coordinates of the trajectory center point and the timestamp of the vehicle target frame by frame; For each vehicle target, the instantaneous velocity vector of each frame is calculated based on the coordinates of the center point of the trajectory of the adjacent trajectory points in the initial trajectory point set of the vehicle target and the timestamp. The instantaneous velocity vector is associated with the trajectory points to obtain an initial video trajectory sequence containing the coordinates of the center point of the trajectory and the instantaneous velocity vector. The vehicle passage data, vehicle positioning data and initial video trajectory sequence are aligned by spatiotemporal coordinate normalization to obtain the vehicle trajectory sequence.

[0019] In this embodiment, multi-source vehicle perception data refers to raw vehicle-related data collected from different perception channels, such as multi-frame data streams, checkpoint vehicle data, and vehicle positioning data. The target detection model is used to identify vehicle target entities within image frames; for example, it can be obtained using a deep learning detection network. Cross-frame identity matching is used to associate the same vehicle target across consecutive video frames. Tracking identifiers are used to uniquely represent a single vehicle target. The initial trajectory point set is the set of vehicle positions and times output from a single video source. The instantaneous velocity vector is used to represent the magnitude and direction of the vehicle's motion in each frame. Spatiotemporal coordinate normalization and alignment are used to unify data from different sources to the same spatial and temporal reference.

[0020] For example, this embodiment can collect multi-frame data streams output by video acquisition devices deployed on the road, simultaneously acquire vehicle passage data output by checkpoint devices deployed at road locations, and vehicle positioning data transmitted back by roadside terminals, all of which serve as multi-source vehicle perception data. This embodiment can use YOLOv5 as the target detection model to parse the multi-frame data stream frame by frame, identifying all vehicle targets within the image. Then, the ByteTrack tracking algorithm is used to complete cross-frame identity matching, associating targets belonging to the same vehicle in different video frames, assigning a unique tracking identifier to each successfully associated vehicle target, and maintaining this tracking identifier throughout the entire process from when the vehicle enters the monitoring area until it leaves the monitoring area.

[0021] This embodiment can extract the trajectory association for each vehicle target one by one based on the assigned tracking identifiers. The center point of the vehicle detection box output in each frame is used as the coordinates of the trajectory center point, and the acquisition timestamp of the corresponding frame is bound to it to form an initial trajectory point set recorded frame by frame. For the initial trajectory point set corresponding to each vehicle target, the coordinates of the trajectory center point and the timestamp of the two adjacent trajectory points are read in sequence. The instantaneous velocity vector of each frame is calculated based on the spatial displacement and time interval between the two points. The calculated instantaneous velocity vector is bound to the corresponding trajectory point, thereby generating the initial video trajectory sequence.

[0022] This embodiment can read the vehicle passage points, passage times, and location and time information output from the vehicle positioning data recorded by the checkpoint vehicle passage data, and select the road plane coordinate system obtained from on-site road calibration as a unified reference. The image pixel coordinates within the initial video trajectory sequence are transformed and mapped to the road plane coordinate system, and the location information carried by the checkpoint vehicle passage data and vehicle positioning data are also transformed to this road plane coordinate system. According to the time dimension, a time tolerance threshold of 200 milliseconds is set. Multi-source records within this time tolerance threshold range are matched and fused to complete spatiotemporal coordinate normalization and alignment. Abnormal records with time deviations exceeding the tolerance threshold are then eliminated, and finally, a complete vehicle trajectory sequence is output.

[0023] This embodiment integrates multiple types of vehicle perception data, generates an initial video trajectory sequence based on target detection and cross-frame matching, and then completes the spatiotemporal alignment of multi-source data. This can compensate for the trajectory disconnection problem caused by single video tracking, enrich the dimensions of trajectory information, improve the integrity and positional accuracy of vehicle trajectory sequences, and provide a reliable data foundation for subsequent vehicle abnormal behavior identification.

[0024] S102: Extract the coordinates of the center point of the vehicle trajectory and the instantaneous velocity vector from the vehicle trajectory sequence, and smooth the coordinates of the center point of the vehicle trajectory and the instantaneous velocity vector to obtain standardized trajectory data.

[0025] In this embodiment, the coordinates of the vehicle trajectory center point and the instantaneous velocity vector are smoothed to obtain standardized trajectory data, including: The coordinates of the vehicle trajectory center point are smoothed in the temporal domain by Kalman filtering to obtain a smoothed trajectory center point coordinate sequence. The frame-by-frame instantaneous velocity vector sequence corresponding to the smoothed trajectory center point coordinate sequence is determined. The magnitude and direction of the instantaneous velocity vector in the frame-by-frame instantaneous velocity vector sequence are jointly smoothed and corrected by a third-order Kalman filter to obtain the smoothed instantaneous velocity vector sequence. The smoothed trajectory center point coordinate sequence and the smoothed instantaneous velocity vector sequence are bound together by timestamps to obtain standardized trajectory data.

[0026] In this embodiment, the coordinates of the vehicle trajectory center point refer to the spatial position corresponding to the center point of the vehicle target detection box, which may include, for example, the position value in the road plane coordinate system. The instantaneous velocity vector is used to characterize the vehicle's motion rate and direction of travel in a single frame. Kalman filtering is a temporal state estimation method, which can output denoised position results based on time-series observation data. Third-order Kalman filtering is used to perform higher-order state estimation on the velocity vector. The amplitude is used to characterize the magnitude of the velocity vector's motion rate, and the direction is used to characterize the vehicle's direction of travel. The standardized trajectory data is a set of regular trajectories whose position and velocity are bound together by time after denoising correction.

[0027] For example, this embodiment can parse and read each trajectory record in the vehicle trajectory sequence one by one according to the timestamp order, and separate and extract two types of original observation data: the coordinates of the center point of the vehicle trajectory and the instantaneous velocity vector. The original observation data comes directly from the vehicle trajectory sequence output after multi-source perception fusion, wherein the instantaneous velocity vector is calculated from the spatial displacement and time interval of adjacent trajectory points within the trajectory sequence.

[0028] This embodiment allows configuration of process noise parameters and observation noise parameters for Kalman filtering. The process noise parameters are preset based on the normal driving state of the vehicle, while the observation noise parameters are set in conjunction with the positioning observation error of the sensing device. Position temporal smoothing is performed on the time-series observation results of the vehicle trajectory center point coordinates for each vehicle target. The filtering process iterates frame by frame in chronological order, filtering out position jumps caused by sensing sampling fluctuations, and outputting a smoothed trajectory center point coordinate sequence. This sequence maintains the original temporal order of all timestamps, and the number of frames corresponds to the original input trajectory.

[0029] This embodiment can extract the original velocity observation information at each time point based on the timestamp in the smoothed trajectory center point coordinate sequence, and organize it into a frame-by-frame instantaneous velocity vector sequence. By configuring the filtering parameters of a third-order Kalman filter, the magnitude and direction of the instantaneous velocity vector are simultaneously included in the filtering correction calculation as joint estimation objects. Noise reduction processing is performed on both the magnitude of the velocity and the direction of travel, suppressing severe jitter in velocity observations, and obtaining a smoothed instantaneous velocity vector sequence.

[0030] This embodiment can traverse all timestamps and bind the coordinates of the smoothed trajectory center point to the smoothed instantaneous velocity vector one by one at the same timestamp. The timestamp matching process sets the time tolerance threshold to 150 milliseconds, discards invalid frame records that cannot be matched due to exceeding the tolerance threshold, and summarizes the time-series records that have been bound to obtain standardized trajectory data, which is then output for subsequent abnormal behavior identification.

[0031] This embodiment uses Kalman filtering to perform noise reduction and correction on the position and velocity vectors respectively, simultaneously optimizing the position accuracy and the amplitude and direction of velocity, eliminating jitter interference from the sensing data, outputting more consistent standardized trajectory data, reducing the risk of misjudgment in the subsequent abnormal behavior identification process, and ensuring the stability of the judgment results for various traffic incidents.

[0032] S103: Based on standardized trajectory data and a pre-built polygonal region model, abnormal vehicle behavior is identified by combining event judgment thresholds corresponding to multiple event types; the polygonal region model is a vector geometric model constructed based on the boundary coordinates of the control area calibrated on the road.

[0033] In this embodiment, abnormal vehicle behavior includes abnormal flow behavior, abnormal area intrusion behavior, abnormal area statistics behavior, and abnormal cross-line driving behavior; the event judgment thresholds corresponding to multiple event types include the judgment threshold for flow-related events, the judgment threshold for area intrusion events, the judgment threshold for area statistics events, and the judgment threshold for cross-line events. In this embodiment, based on standardized trajectory data and a pre-constructed polygonal region model, and combined with event determination thresholds corresponding to multiple event types, abnormal vehicle behavior is identified, including: The coordinates of the center points of the trajectory under adjacent timestamps in the standardized trajectory data are differentially calculated to obtain the displacement per unit time; the lane direction reference vector is extracted from the polygonal region model, and the dot product operation is performed between the instantaneous velocity vector and the lane direction reference vector to obtain the vehicle driving direction angle; the motion acceleration is calculated based on the instantaneous velocity vectors of adjacent frames and the inter-frame time interval. Based on the vehicle's driving direction angle, displacement per unit time, and acceleration, the judgment result of abnormal flow behavior is determined by combining the judgment threshold of flow-related events. Abnormal flow behavior includes vehicles driving in the wrong direction, stopping, exceeding speed limits, and sudden braking. Extract the geometric boundaries of the control region from the polygonal region model; Based on the coordinates of the trajectory center point in the standardized trajectory data and the geometric boundary of the controlled area, combined with the threshold for judging intrusion events in the area, the judgment result of abnormal intrusion behavior in the area is determined. Abnormal intrusion behaviors in the area include intrusion into the emergency lane, intrusion into the escape lane, intrusion into the cooling pool, and intrusion into the construction area.

[0034] Extracting the target statistical region boundary from a polygonal region model; Based on standardized trajectory data and the boundary of the target statistical area, determine the vehicle operation indicators corresponding to all vehicles within the target statistical area; Based on vehicle operation indicators and combined with the regional statistical event judgment threshold, the judgment result of regional statistical abnormal behavior is determined; regional statistical abnormal behavior includes traffic congestion, full parking area, parking time exceeding the limit, and queue exceeding the limit.

[0035] Extract continuous trajectory segments from standardized trajectory data, and extract lane baseline segments from polygonal region models; Intersection detection is performed on continuous trajectory segments and lane baseline segments to obtain intersection detection results; Calculate the lateral displacement between the continuous trajectory segment and the lane baseline segment; Based on the intersection detection results and lateral displacement, the judgment result of abnormal cross-line driving behavior is determined by combining the cross-line event judgment threshold.

[0036] In this embodiment, the polygonal region model refers to a vector geometric model generated based on the boundary coordinates of the controlled area as determined on-site, such as vector models of areas like emergency lanes, escape lanes, and cooling pools. Abnormal flow behavior refers to anomalies caused by vehicle movement violating traffic rules, such as vehicles driving in the wrong direction, stopping, exceeding speed limits, and sudden braking. Abnormal area intrusion behavior refers to anomalies caused by vehicles entering a prohibited controlled area. Abnormal area statistical behavior refers to group traffic anomalies obtained based on multi-vehicle statistical indicators within the area. Abnormal cross-lane driving behavior refers to anomalies caused by vehicles crossing lane baseline segments. Event judgment thresholds are preset discrimination thresholds for various abnormal behaviors, which can be configured according to road management regulations. Lane direction reference vectors are used to characterize the legally permitted traffic orientation of the lane.

[0037] In this embodiment, the geometric boundary of the controlled area is the contour vector of the prohibited area in the polygonal region model. The boundary of the target statistical area is the contour vector of the area where traffic flow statistics are conducted. Vehicle operation indicators are traffic flow-related statistical parameters extracted within the statistical area. Continuous trajectory segments are trajectory segments formed by connecting standardized trajectory data in chronological order. Lane baseline segments are vector segments representing lane separation within the polygonal region model. Intersection detection results indicate whether the trajectory segments intersect with the lane baseline segments. Lateral displacement is used to characterize the degree of lateral offset of the vehicle relative to the lane baseline segment.

[0038] For example, this embodiment can read a pre-constructed polygonal region model. When calibrating this polygonal region model on the road, the actual boundary coordinates of the emergency lane, escape lane, construction control zone, cooling pool, and parking area are collected. Vector construction is completed using geometric processing tools, and all kinds of boundary vectors are uniformly mapped to the road plane coordinate system. At the same time, standardized trajectory data that has undergone noise reduction processing is read. The standardized trajectory data stores the coordinates of the trajectory center point of each vehicle arranged in an orderly manner according to the timestamp, as well as the instantaneous speed vector. Multiple sets of event judgment thresholds that have been pre-configured according to road control requirements are retrieved, including thresholds for flow-related events, intrusion-related events, statistical events, and cross-line events. Different thresholds can be configured for different road sections, and the threshold parameters for highway sections and tunnel sections can be set independently.

[0039] For example, this embodiment can traverse the standardized trajectory data of a single vehicle across all time sequences, and perform a difference operation on the coordinates of the trajectory center points corresponding to adjacent timestamps to obtain the displacement per unit time. The lane direction reference vector of the corresponding lane is retrieved from the polygonal region model, and a vector dot product operation is performed between the instantaneous velocity vector and the lane direction reference vector to obtain the vehicle's driving direction angle. Motion acceleration is calculated using the instantaneous velocity vectors of adjacent frames and the inter-frame time interval. The obtained vehicle driving direction angle, displacement per unit time, and motion acceleration are compared with the threshold values ​​for determining flow-related events. For example, in the threshold values ​​for determining flow-related events, the acceleration threshold for emergency braking is set to -5 m / s², the minimum safe driving distance threshold and the stationary duration threshold are set for stopping, and high-speed overspeed threshold and low-speed driving threshold are configured respectively. Combining the angle discrimination rules for different scenarios in tunnels and highways, the various parameters are comprehensively compared with the thresholds to output the judgment result of abnormal flow behavior, identifying vehicle reversing, stopping, speed exceeding limits, and emergency braking events.

[0040] For example, this embodiment can extract the geometric boundary of the control area stored in the polygonal region model. The geometric boundary of the control area includes the vector contours of the emergency lane, the escape lane, the cooling pool, and the construction area. Figure 2 As shown, Figure 2 This is a schematic diagram of the boundary division of the control area provided in one embodiment of this application. Figure 2 This image presents a simplified road scene, where the main lane area is located in the center of the image and serves as the primary passage for vehicles to travel normally. Figure 2 A black rectangle indicates that a "vehicle" is located within this area. The emergency lane is located below the main lane area, the parking area is located to the right of the main lane area, and the cooling pool is located below the parking area.

[0041] The coordinates of the center point of the standardized trajectory data are sequentially compared with the geometric boundary of the controlled area to perform spatial relationship calculations, determining whether the vehicle's center point falls within the controlled area. At the same time, the duration of the vehicle's continuous stay within the controlled area is counted. The location determination result and the stay duration are compared with the threshold for determining area intrusion events. The threshold for determining area intrusion events can be set with a stay duration threshold. For example, if the vehicle's center point falls inside the emergency escape lane and the duration exceeds 300 milliseconds, the determination is triggered. The result of the abnormal behavior determination of area intrusion is output, identifying intrusion events such as intrusion into the emergency lane, intrusion into the emergency escape lane, intrusion into the cooling pool, and intrusion into the construction area.

[0042] For example, this embodiment can extract the boundary of the target statistical region from a polygonal region model. The boundary of the target statistical region includes the vector contour of the parking area of ​​the road cross-section. The standardized trajectory data of all vehicles within the boundary of the target statistical region are traversed to obtain the vehicle operation indicators corresponding to all vehicles within the target statistical region. The vehicle operation indicators include the total number of vehicles in the region, the average vehicle dwell time, and the queue length. Each vehicle operation indicator is compared with the regional statistical event judgment thresholds. For example, the regional statistical event judgment thresholds can be set as follows: average dwell time threshold for congestion judgment and regional vehicle number threshold; maximum parking space limit threshold for parking area full; maximum allowed dwell time for a single vehicle for parking timeout; and maximum queue length limit threshold for queue exceeding the limit. Based on the comparison results, the regional statistical abnormal behavior judgment results are output to identify traffic congestion, parking area full, parking timeout, and queue exceeding the limit events.

[0043] For example, this embodiment can read coordinate points from standardized trajectory data in chronological order, stitch them together to generate continuous trajectory segments, and simultaneously extract lane baseline segments from the polygonal region model. The lane baseline segments correspond to the vectors of the lane dividing lines. Intersection detection is performed on the continuous trajectory segments and the lane baseline segments to obtain intersection detection results. If intersection is confirmed, the lateral displacement of the continuous trajectory segments relative to the lane baseline segments is further calculated. The intersection detection results and the lateral displacement are matched and verified against a threshold value for cross-lane events. This threshold value is configured with a lateral displacement threshold; when trajectories intersect and the lateral displacement exceeds the threshold, cross-lane driving abnormal behavior is determined, and the cross-lane driving abnormal behavior determination result is output. The above four types of abnormal behavior determination results are summarized to complete the overall identification of abnormal vehicle behavior.

[0044] This embodiment relies on the same set of standardized trajectory data and a unified polygonal region vector geometric model to complete the identification of four types of abnormal vehicle behaviors through multi-dimensional calculations. It eliminates the need to build independent data processing links for each type of abnormal behavior, is compatible with various scenarios such as movement behavior regions intruding into traffic flow statistics and crossing lanes, fully reuses trajectory and geometric model resources, and the unified discrimination logic reduces parameter inconsistencies caused by multiple independent systems, improving the synergy and overall recognition efficiency of various traffic event identifications.

[0045] S104: Generate early warning information based on the identified abnormal vehicle behavior.

[0046] In this embodiment, generating early warning information based on the identified abnormal vehicle behavior includes: The event type and risk level are determined based on the identified abnormal vehicle behavior. Obtain standardized trajectory data corresponding to abnormal vehicle behavior, and extract the timestamp, duration, and location information corresponding to the abnormal vehicle behavior from the standardized trajectory data; The location information is matched with the polygonal region model to obtain the control area identifier corresponding to the abnormal vehicle behavior. Determine the early warning mode based on the event type and event risk level; Warning information is generated based on the warning mode and the timestamp, duration, location information, and control area identifier corresponding to the abnormal vehicle behavior.

[0047] In this embodiment, the event type refers to the specific traffic event category corresponding to the identified abnormal vehicle behavior, such as a vehicle driving in the wrong direction and occupying an emergency lane. The event risk level is a risk hierarchy divided according to the severity of the event, which can be obtained by evaluating the event type and behavioral parameters. The control area identifier is the identity tag corresponding to each control area within the polygonal area model, used to distinguish different road control sections. The warning mode is the alarm output rule adapted to the event risk configuration, which may include a reporting priority message rule suppression strategy. The warning information is the alarm message data output after encapsulating various event elements.

[0048] For example, this embodiment can receive the vehicle abnormal behavior recognition result output in the previous step, read the recognized vehicle abnormal behavior, retrieve pre-stored event mapping rules, match the recognized abnormal behavior with the corresponding event type, and simultaneously retrieve preset risk level classification rules. The risk level classification rules combine the severity of the event itself with parameters such as the duration of the abnormal behavior for comprehensive evaluation. For example, a fire and vehicle collision are classified as the highest risk level, while low-speed vehicle driving is classified as a lower risk level, thereby determining the event type and event risk level.

[0049] This embodiment can retrieve the standardized trajectory data associated with the abnormal vehicle behavior, extract the timestamp of the start time from the standardized trajectory data along the time interval corresponding to the occurrence of the abnormal behavior, calculate the duration by combining the start and end times of the abnormal behavior, and read the coordinates of the trajectory center point as the location information.

[0050] This embodiment can perform spatial matching calculations between the extracted location information and the pre-constructed polygonal region model to determine which control vector region the location information falls within, and read the control area identifier bound to that vector region, such as the area number corresponding to the emergency lane or escape lane, to complete the control area identifier matching and acquisition.

[0051] This embodiment can retrieve a pre-stored warning mode configuration table. The configuration table stores the configuration content corresponding to different event types and event risk levels, including alarm reporting priority and alarm repetition suppression strategy. For example, a short suppression interval is set for high-risk events, and a longer suppression interval is set for low-risk events. The warning mode is determined by looking up the table based on the current event type and event risk level.

[0052] This embodiment can organize and encapsulate the timestamp, duration, location information and control area identifier according to the message organization format set in the early warning mode to obtain complete early warning information, and output it to the monitoring platform to complete the event reporting.

[0053] This embodiment categorizes abnormal behaviors and assesses their risk levels. It then matches control area identifiers using a polygonal region model and selects an appropriate early warning mode based on the event conditions. All key elements are uniformly encapsulated to generate early warning information. This ensures the completeness of alarm elements and that alarm output rules are adapted to event risks, facilitating tiered handling by the backend monitoring platform and improving the efficiency of traffic incident response.

[0054] Corresponding to the vehicle abnormal behavior warning method in the above embodiment, Figure 3 This is a structural block diagram of a vehicle abnormal behavior warning device according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 3 The vehicle abnormal behavior warning device 20 includes: a trajectory sequence construction module 21, a data processing module 22, an abnormal behavior recognition module 23, and a warning module 24.

[0055] Among them, the trajectory sequence construction module 21 is used to acquire multi-source vehicle perception data and determine the vehicle trajectory sequence based on the multi-source vehicle perception data. Data processing module 22 is used to extract the coordinates of the center point of the vehicle trajectory and the instantaneous velocity vector from the vehicle trajectory sequence, and to smooth the coordinates of the center point of the vehicle trajectory and the instantaneous velocity vector to obtain standardized trajectory data. The abnormal behavior recognition module 23 is used to identify abnormal vehicle behavior based on standardized trajectory data and a pre-built polygonal region model, combined with event judgment thresholds corresponding to multiple event types; the polygonal region model is a vector geometric model constructed based on the boundary coordinates of the control area calibrated on the road. The early warning module 24 is used to generate early warning information based on the identified abnormal vehicle behavior.

[0056] In one embodiment of this application, the multi-source vehicle perception data includes multi-frame data streams, checkpoint vehicle passage data, and vehicle positioning data; the trajectory sequence construction module 21, when determining the vehicle trajectory sequence based on the multi-source vehicle perception data, is specifically used for: Based on multi-frame data streams, vehicle targets are identified through a target detection model. Cross-frame identity matching is performed on the identified multiple vehicle targets, and a tracking identifier is assigned to each vehicle target according to the identity matching results. Based on the tracking identifier, the trajectory association of each vehicle target is extracted to obtain an initial trajectory point set including the coordinates of the trajectory center point and the timestamp of the vehicle target frame by frame; For each vehicle target, the instantaneous velocity vector of each frame is calculated based on the coordinates of the center point of the trajectory of the adjacent trajectory points in the initial trajectory point set of the vehicle target and the timestamp. The instantaneous velocity vector is associated with the trajectory points to obtain an initial video trajectory sequence containing the coordinates of the center point of the trajectory and the instantaneous velocity vector. The vehicle passage data, vehicle positioning data and initial video trajectory sequence are aligned by spatiotemporal coordinate normalization to obtain the vehicle trajectory sequence.

[0057] In one embodiment of this application, when the data processing module 22 smooths the coordinates of the vehicle trajectory center point and the instantaneous velocity vector to obtain standardized trajectory data, it is specifically used for: The coordinates of the vehicle trajectory center point are smoothed in the temporal domain by Kalman filtering to obtain a smoothed trajectory center point coordinate sequence. The frame-by-frame instantaneous velocity vector sequence corresponding to the smoothed trajectory center point coordinate sequence is determined. The magnitude and direction of the instantaneous velocity vector in the frame-by-frame instantaneous velocity vector sequence are jointly smoothed and corrected by a third-order Kalman filter to obtain the smoothed instantaneous velocity vector sequence. The smoothed trajectory center point coordinate sequence and the smoothed instantaneous velocity vector sequence are bound together by timestamps to obtain standardized trajectory data.

[0058] In one embodiment of this application, abnormal vehicle behavior includes abnormal flow behavior and abnormal area intrusion behavior; the event judgment thresholds corresponding to multiple event types include flow event judgment thresholds and area intrusion event judgment thresholds; when the abnormal behavior recognition module 23 identifies abnormal vehicle behavior based on standardized trajectory data and a pre-built polygonal region model, combined with the event judgment thresholds corresponding to multiple event types, it is specifically used for: The coordinates of the center points of the trajectory under adjacent timestamps in the standardized trajectory data are differentially calculated to obtain the displacement per unit time; the lane direction reference vector is extracted from the polygonal region model, and the dot product operation is performed between the instantaneous velocity vector and the lane direction reference vector to obtain the vehicle driving direction angle; the motion acceleration is calculated based on the instantaneous velocity vectors of adjacent frames and the inter-frame time interval. Based on the vehicle's driving direction angle, displacement per unit time, and acceleration, the judgment result of abnormal flow behavior is determined by combining the judgment threshold of flow-related events. Abnormal flow behavior includes vehicles driving in the wrong direction, stopping, exceeding speed limits, and sudden braking. Extract the geometric boundaries of the control region from the polygonal region model; Based on the coordinates of the trajectory center point in the standardized trajectory data and the geometric boundary of the controlled area, combined with the threshold for judging intrusion events in the area, the judgment result of abnormal intrusion behavior in the area is determined. Abnormal intrusion behaviors in the area include intrusion into the emergency lane, intrusion into the escape lane, intrusion into the cooling pool, and intrusion into the construction area.

[0059] In one embodiment of this application, abnormal vehicle behavior further includes regional statistical abnormal behavior; the event judgment thresholds corresponding to multiple event types also include regional statistical event judgment thresholds; when the abnormal behavior identification module 23 identifies abnormal vehicle behavior based on standardized trajectory data and a pre-built polygonal region model, combined with the event judgment thresholds corresponding to multiple event types, it is specifically used for: Extracting the target statistical region boundary from a polygonal region model; Based on standardized trajectory data and the boundary of the target statistical area, determine the vehicle operation indicators corresponding to all vehicles within the target statistical area; Based on vehicle operation indicators and combined with the regional statistical event judgment threshold, the judgment result of regional statistical abnormal behavior is determined; regional statistical abnormal behavior includes traffic congestion, full parking area, parking time exceeding the limit, and queue exceeding the limit.

[0060] In one embodiment of this application, abnormal vehicle behavior further includes cross-lane driving abnormal behavior; the event judgment thresholds corresponding to multiple event types also include cross-lane event judgment thresholds; when the abnormal behavior recognition module 23 identifies abnormal vehicle behavior based on standardized trajectory data and a pre-built polygonal region model, combined with the event judgment thresholds corresponding to multiple event types, it is specifically used for: Extract continuous trajectory segments from standardized trajectory data, and extract lane baseline segments from polygonal region models; Intersection detection is performed on continuous trajectory segments and lane baseline segments to obtain intersection detection results; Calculate the lateral displacement between the continuous trajectory segment and the lane baseline segment; Based on the intersection detection results and lateral displacement, the judgment result of abnormal cross-line driving behavior is determined by combining the cross-line event judgment threshold.

[0061] In one embodiment of this application, when the warning module 24 generates warning information based on the identified abnormal vehicle behavior, it is specifically used for: The event type and risk level are determined based on the identified abnormal vehicle behavior. Obtain standardized trajectory data corresponding to abnormal vehicle behavior, and extract the timestamp, duration, and location information corresponding to the abnormal vehicle behavior from the standardized trajectory data; The location information is matched with the polygonal region model to obtain the control area identifier corresponding to the abnormal vehicle behavior. Determine the early warning mode based on the event type and event risk level; Warning information is generated based on the warning mode and the timestamp, duration, location information, and control area identifier corresponding to the abnormal vehicle behavior.

[0062] See Figure 4 , Figure 4 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 4 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of each module / unit in the above-described device embodiments, for example... Figure 3 The functions of the trajectory sequence construction module 21, data processing module 22, abnormal behavior recognition module 23, and early warning module 24 are shown.

[0063] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0064] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0065] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store information about vehicle trajectories.

[0066] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation method described in the vehicle abnormal behavior warning method provided in the embodiments of this application, or they can execute the implementation method of the electronic device described in the embodiments of this application, which will not be repeated here.

[0067] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0068] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0069] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0070] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0071] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces or units, or they may be electrical, mechanical, or other forms of connection.

[0072] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0073] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0074] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for early warning of abnormal vehicle behavior, characterized in that, include: Acquire multi-source vehicle perception data, and determine vehicle trajectory sequences based on the multi-source vehicle perception data; The coordinates of the vehicle trajectory center point and the instantaneous velocity vector are extracted from the vehicle trajectory sequence. The coordinates of the vehicle trajectory center point and the instantaneous velocity vector are then smoothed to obtain standardized trajectory data. Based on the standardized trajectory data and the pre-constructed polygonal region model, abnormal vehicle behavior is identified by combining event judgment thresholds corresponding to multiple event types. The polygonal region model is a vector geometric model constructed based on the boundary coordinates of the control area as determined on-site. Early warning information is generated based on the identified abnormal vehicle behavior.

2. The vehicle abnormal behavior early warning method as described in claim 1, characterized in that, The multi-source vehicle perception data includes multi-frame data streams, checkpoint vehicle passage data, and vehicle positioning data. Determining the vehicle trajectory sequence based on the multi-source vehicle perception data includes: Based on the multi-frame data stream, vehicle targets are identified through a target detection model, cross-frame identity matching is performed on the identified vehicle targets, and a tracking identifier is assigned to each vehicle target according to the identity matching result. Based on the tracking identifier, trajectory association is extracted for each vehicle target to obtain an initial trajectory point set including the coordinates of the trajectory center point and the timestamp of the vehicle target frame by frame; For each vehicle target, the instantaneous velocity vector of each frame is calculated based on the coordinates of the center point of the trajectory of the adjacent trajectory points in the initial trajectory point set of the vehicle target and the timestamp. The instantaneous velocity vector is associated with the trajectory points to obtain an initial video trajectory sequence containing the coordinates of the center point of the trajectory and the instantaneous velocity vector. The vehicle passage data at the checkpoint, the vehicle positioning data, and the initial video trajectory sequence are aligned by spatiotemporal coordinate normalization to obtain the vehicle trajectory sequence.

3. The vehicle abnormal behavior early warning method as described in claim 1, characterized in that, The process of smoothing the coordinates of the vehicle trajectory center point and the instantaneous velocity vector to obtain standardized trajectory data includes: The coordinates of the vehicle trajectory center point are smoothed in the temporal domain by Kalman filtering to obtain a smoothed trajectory center point coordinate sequence. The frame-by-frame instantaneous velocity vector sequence corresponding to the smoothed trajectory center point coordinate sequence is determined, and the magnitude and direction of the instantaneous velocity vector in the frame-by-frame instantaneous velocity vector sequence are jointly smoothed and corrected by a third-order Kalman filter to obtain the smoothed instantaneous velocity vector sequence. The smoothed trajectory center point coordinate sequence and the smoothed instantaneous velocity vector sequence are bound together by timestamps to obtain the standardized trajectory data.

4. The vehicle abnormal behavior early warning method as described in claim 1, characterized in that, The abnormal vehicle behavior includes abnormal movement behavior and abnormal area intrusion behavior; the event judgment thresholds corresponding to the multiple event types include the judgment threshold for movement events and the judgment threshold for area intrusion events. The method of identifying abnormal vehicle behavior based on the standardized trajectory data and a pre-constructed polygonal region model, combined with event determination thresholds corresponding to multiple event types, includes: The coordinates of the trajectory center points under adjacent timestamps in the standardized trajectory data are differentially calculated to obtain the displacement per unit time; the lane direction reference vector is extracted from the polygonal region model, and the dot product of the instantaneous velocity vector and the lane direction reference vector is performed to obtain the vehicle driving direction angle; the motion acceleration is calculated based on the instantaneous velocity vectors of adjacent frames and the inter-frame time interval. Based on the vehicle's driving direction angle, the displacement per unit time, and the acceleration, the abnormal flow behavior determination result is determined in conjunction with the flow event determination threshold. The abnormal flow behavior includes vehicle driving in the wrong direction, stopping, exceeding speed limits, and sudden braking. Extract the geometric boundary of the control region from the polygonal region model; Based on the coordinates of the trajectory center point in the standardized trajectory data and the geometric boundary of the controlled area, combined with the threshold for judging intrusion events in the area, the judgment result of abnormal intrusion behavior in the area is determined. The abnormal intrusion behavior in the area includes intrusion into the emergency lane, intrusion into the escape lane, intrusion into the cooling pool, and intrusion into the construction area.

5. The vehicle abnormal behavior early warning method as described in claim 4, characterized in that, The abnormal vehicle behavior also includes regional statistical abnormal behavior; the event judgment thresholds corresponding to the multiple event types also include regional statistical event judgment thresholds. The method of identifying abnormal vehicle behavior based on the standardized trajectory data and the pre-constructed polygonal region model, combined with event determination thresholds corresponding to multiple event types, further includes: Extract the target statistical region boundary from the polygonal region model; Based on the standardized trajectory data and the boundary of the target statistical area, determine the vehicle operation indicators corresponding to all vehicles within the target statistical area; Based on the vehicle operation indicators, the regional statistical abnormal behavior judgment result is determined by combining the regional statistical event judgment threshold; the regional statistical abnormal behavior includes traffic congestion, full parking area, parking time exceeding the limit and queue exceeding the limit.

6. The vehicle abnormal behavior early warning method as described in claim 5, characterized in that, The abnormal vehicle behavior also includes abnormal cross-line driving behavior; the event determination thresholds corresponding to the multiple event types also include cross-line event determination thresholds. The method of identifying abnormal vehicle behavior based on the standardized trajectory data and the pre-constructed polygonal region model, combined with event determination thresholds corresponding to multiple event types, further includes: Extract continuous trajectory segments from the standardized trajectory data, and extract lane baseline segments from the polygonal region model; Intersection detection is performed between the continuous trajectory line segment and the lane baseline line segment to obtain the intersection detection result; Calculate the lateral displacement between the continuous trajectory segment and the lane baseline segment; Based on the intersection detection results and the lateral displacement, the abnormal behavior judgment result of crossing the line is determined by combining the cross-line event judgment threshold.

7. The vehicle abnormal behavior early warning method as described in claim 1, characterized in that, The generation of early warning information based on the identified abnormal vehicle behavior includes: The event type and risk level are determined based on the identified abnormal vehicle behavior. Obtain standardized trajectory data corresponding to the abnormal vehicle behavior, and extract the timestamp, duration, and location information corresponding to the abnormal vehicle behavior from the standardized trajectory data; The location information is matched with the polygonal region model to obtain the control area identifier corresponding to the abnormal vehicle behavior; The early warning mode is determined based on the event type and the event risk level; Warning information is generated based on the warning mode and the timestamp, duration, location information, and control area identifier corresponding to the abnormal vehicle behavior.

8. A vehicle abnormal behavior early warning device, characterized in that, include: The trajectory sequence construction module is used to acquire multi-source vehicle perception data and determine vehicle trajectory sequences based on the multi-source vehicle perception data. The data processing module is used to extract the coordinates of the center point of the vehicle trajectory and the instantaneous velocity vector from the vehicle trajectory sequence, and to smooth the coordinates of the center point of the vehicle trajectory and the instantaneous velocity vector to obtain standardized trajectory data. The abnormal behavior recognition module is used to identify abnormal vehicle behavior based on the standardized trajectory data and the pre-built polygonal region model, combined with event judgment thresholds corresponding to multiple event types. The polygonal region model is a vector geometric model constructed based on the boundary coordinates of the control area as determined on-site. The early warning module is used to generate early warning information based on the identified abnormal vehicle behavior.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images