A vehicle video data analysis method based on traffic responsibility definition

Abstract

The present application relates to the technical field of vehicle video data analysis, and discloses a vehicle video data analysis method based on traffic responsibility definition. The method first builds a video library containing various traffic scene videos, and each video has an event sequence diagram. Then, the same scene event sequence diagrams are merged into a comprehensive sequence diagram. Next, the responsibility requirements and analysis requirements of the accident are obtained, and the comprehensive sequence diagram is selected and the basic sequence diagram is extracted according to the requirements, and the part to be identified is labeled. Then, a responsibility judgment model is constructed, and after the complete sequence diagram is obtained by analyzing the part to be identified, the key frame nodes are checked, and the nodes with a dispute probability greater than a critical threshold are defined as dispute nodes and a label reminder is generated. In addition, when the video library is built, the video is collected and preprocessed, and the key frame nodes are extracted by the optical flow method or the convolutional neural network model. The method can improve the efficiency and accuracy of traffic responsibility definition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle video data analysis, in particular to a vehicle video data analysis method based on traffic responsibility definition. BACKGROUND

[0002] With the continuous expansion of urban traffic networks, the frequency of traffic accidents is rising year by year, and the complexity of accident responsibility definition is increasingly prominent. Currently, traffic accident responsibility determination mainly relies on the comprehensive analysis of on-site investigation records, statements of parties involved and limited monitoring videos by law enforcement personnel, but this approach has many limitations.

[0003] In actual operation, monitoring videos often fail to clearly present key details of accident occurrence due to problems such as shooting angle, light condition or device resolution. For example, in intersection accidents, core information such as vehicle driving trajectory, signal light status and avoidance actions of both parties may not be accurately captured due to blurred pictures, making it difficult for law enforcement personnel to form an objective judgment. At the same time, accident features in different traffic scenarios differ significantly, such as the responsibility determination logic for highway rear-end collisions and urban intersection collisions is completely different, and existing analysis methods lack effective adaptation to scene specificity, which may lead to incorrect application of determination standards.

[0004] In addition, in the traditional process of responsibility determination, the processing of video data mainly relies on manual frame-by-frame inspection, which not only consumes a lot of time and labor cost, but also may lead to biased determination results due to individual experience differences. For example, for accidents caused by continuous lane changes, manual analysis needs to repeatedly check details such as vehicle turn signal opening time and lane line crossing nodes, and key information may be missed due to visual fatigue during the process. And when the accident involves multiple vehicles or complex road conditions, the descriptions of the accident process by each party often contradict each other, and existing technologies lack systematic marking and tracing mechanisms for controversial points in video data, leading to situations where responsibility determination is stuck.

[0005] In existing technologies, some automated analysis tools can extract basic parameters such as vehicle position and speed, but cannot accurately associate these parameters with responsibility clauses in traffic regulations. For example, tools can identify vehicle speeding behavior, but cannot determine the direct relevance of this behavior to accident consequences in combination with the relative position relationship at the time of the accident. At the same time, these tools have difficulty in integrating picture information from different devices when processing multi-source video data, resulting in insufficient completeness of accident sequences and further increasing the difficulty of responsibility determination.

[0006] In judicial practice, disputes over responsibility identification caused by inadequate video data analysis are common. Parties often object to the judgment and request re-examination, but since the original video data has not been structured, the re-examination process needs to start from scratch, greatly prolonging the dispute resolution period. This inefficient processing mode not only affects the efficiency of law enforcement, but also easily leads to doubts about the fairness of the judgment, undermining the credibility of law enforcement.

[0007] With the development of intelligent transportation technology, the volume of vehicle video data has grown explosively, and traditional analysis methods have been unable to cope with the rapid processing needs of massive data. How to accurately extract key event sequences from complex video information and establish an analysis framework that matches the responsibility determination logic has become a problem that needs to be solved in the field of traffic management. SUMMARY

[0008] The purpose of the present application is to provide a vehicle video data analysis method based on traffic responsibility definition to solve the problems raised in the background art.

[0009] To achieve the above purpose, the present application provides a vehicle video data analysis method based on traffic responsibility definition, which comprises:

[0010] A video library is built, which includes multiple traffic scene videos, each traffic scene video having an event sequence graph, and the event sequence graph including multiple key frame nodes. The event sequence graphs of traffic scene videos of the same scene are merged into a comprehensive sequence graph. The responsibility requirements and analysis requirements of an accident are obtained, a corresponding comprehensive sequence graph is selected based on the responsibility requirements, and a corresponding basic sequence graph is extracted in the comprehensive sequence graph based on the analysis requirements. The to-be-identified part in the basic sequence graph is labeled, which is the part of the basic sequence graph that needs to be identified. A responsibility determination model is constructed, and a complete sequence graph is obtained after the analysis of the to-be-identified part is completed. Each key frame node in the complete sequence graph is checked based on the responsibility determination model, the key frame nodes that fail the check are defined as controversial nodes, and a marker pointing to the controversial nodes is generated to remind.

[0011] Preferably, the steps of merging the event sequence diagrams into a comprehensive sequence diagram include the following steps: marking time flow lines between key frame nodes in the event sequence diagrams, numbering each key frame node based on the time flow lines, dividing the event sequence diagrams into multiple stages based on the numbering; taking event sequence diagrams corresponding to the same scene traffic scene video as aggregation objects, taking an aggregation object containing the most stages as a main object, taking key frame nodes included in the main object as main nodes, locating reference nodes in the remaining aggregation objects, merging the key frame nodes of each stage of the aggregation objects into the main object from the reference nodes, and when merging, if the main nodes and the key frame nodes meet the association condition, merging the key frame nodes into the main nodes, and if not, linking the key frame nodes into the main object as branch nodes.

[0012] Preferably, locating the reference nodes includes the following steps: locating the stage with the smallest value in the aggregation object, and taking the key frame node in the stage as a first node, if a main node with an association condition with the first node is found in the main object, the first node is determined as a reference node, otherwise, continue to extract key frame nodes as second nodes in other stages of the aggregation object, and determine whether the second nodes are reference nodes, repeat the step until the reference nodes are located.

[0013] Preferably, extracting the corresponding basic sequence diagram in the comprehensive sequence diagram includes the following steps: obtaining a demand sequence diagram based on the analysis requirements of the accident, locating the start node and the end node in the demand sequence diagram, locating the key frame nodes in the comprehensive sequence diagram that can be associated with the start node and the end node, and taking them as top end nodes and bottom end nodes, respectively, generating multiple candidate sequence diagrams in the comprehensive sequence diagram with the top end nodes and the bottom end nodes as the starting point and the ending point, calculating the matching degree of the demand sequence diagram and each candidate sequence diagram, and taking the candidate sequence diagram with the largest matching degree as the basic sequence diagram.

[0014] Preferably, locating the start node and the end node includes the following steps: the demand sequence diagram includes multiple flow nodes, taking both ends of the demand sequence diagram as the starting point, and determining in sequence whether the flow nodes are target nodes in the demand sequence diagram, wherein if there are key frame nodes in the comprehensive sequence diagram that meet the association condition with the flow nodes, the flow nodes are determined as target nodes, and the target nodes determined first at both ends of the demand sequence diagram are defined as the start node and the end node, respectively.

[0015] Preferably, determining whether to meet the association condition includes the following steps: setting attribute information for each key frame node, the attribute information including its own event label, the event label of the time source node, and the event label of the time output node, when the event labels of the main nodes and the reference nodes are the same, and at least one of the event labels of the time source node and the time output node is the same, the main nodes and the key frame nodes are defined to meet the association condition.

[0016] Preferably, the matching degree calculation comprises the following steps: generating corresponding feature values based on the event labels of the key frame nodes themselves, taking the key frame node adjacent to the starting node or the top node as the first adjacent node, performing XOR operation on the feature values of the first adjacent node and the starting node itself, and performing XOR operation on the feature values of the first adjacent node and the top node itself to obtain the label values of the starting node and the top node; obtaining the second adjacent node of the first adjacent node, the second adjacent node does not contain the starting node or the top node, performing XOR operation on the feature values of the first adjacent node and the second adjacent node to obtain the label value of the first adjacent node, and repeating the step until all key frame nodes in the sequence diagram are traversed; generating a first label set and a second label set corresponding to the demand sequence diagram and the alternative sequence diagram based on the label values, and calculating the matching degree based on the number of the same label values contained in the first label set and the second label set.

[0017] Preferably, the responsibility determination model is constructed based on a neural network, the attribute information of the parsed key frame nodes is input into the responsibility determination model, the responsibility determination model outputs the controversy probability of the key frame nodes, and when the controversy probability is greater than a critical threshold, the key frame node is determined as a controversial node.

[0018] Preferably, the construction of the video library comprises the following steps: collecting traffic scene videos in real time through road monitoring cameras and vehicle-mounted event data recorders, and importing historical accident video data; and storing the collected videos into the video library after denoising and resolution standardization preprocessing.

[0019] Preferably, the key frame nodes in the event sequence diagram are extracted in the following manner: detecting motion changes in the video based on an optical flow method, taking a frame as a key frame node when the motion vector of adjacent frames exceeds a preset threshold; or extracting key frame nodes based on feature similarity clustering by using a convolutional neural network model to extract features of video frames.

[0020] Compared with the prior art, the present application has the following advantages:

[0021] The vehicle video data analysis method based on traffic responsibility definition provides rich scene references for accident analysis by constructing a video library containing various traffic scene videos. Each traffic scene video in the video library has an event sequence diagram, and the event sequence diagrams of the same scene are merged into a comprehensive sequence diagram. This processing method can integrate various accident characteristics under the same scene to form a comprehensive scene analysis basis. In actual application, when facing a specific accident, the corresponding basic sequence diagram can be accurately selected from the comprehensive sequence diagram according to the responsibility demand and analysis demand, so that the analysis process can closely revolve around the actual situation of the accident, avoiding the interference of irrelevant information.

[0022] The identified part is marked in the basic sequence diagram, the focus direction of analysis is clear, and the subsequent analysis work is more targeted. The responsibility judgment model constructed can check each key frame node in the complete sequence diagram after analyzing the identified part. This checking mechanism can go deep into each important moment of the accident, ensuring a comprehensive review of the accident process. When a key frame node that does not pass the check appears, it is defined as a controversial node and a label reminder is generated, which can focus on the doubts in the accident and provide clear attention objects for law enforcement personnel.

[0023] For different types of traffic accidents, this method realizes accurate matching of analysis logic and scene characteristics through scenario-based sequence diagram processing. For example, in the processing of highway accidents, the comprehensive sequence diagram contains key information such as vehicle spacing and speed change, and the basic sequence diagram extracted based on this can accurately reflect the core elements of this type of accident. In the analysis of urban intersection accidents, key frame nodes such as signal light status and pedestrian crossing behavior are included in the sequence diagram to ensure that the analysis process meets the actual needs of the scene.

[0024] In addition, this method breaks the traditional responsibility judgment mode of fuzzy processing of controversial points by marking controversial nodes. Law enforcement personnel can directly investigate controversial nodes based on the label reminder, without the need to blindly search in massive video data, effectively shortening the time for dispute resolution. At the same time, the structured sequence diagram presentation method presents the key links of the accident process in the form of clear nodes, so that different law enforcement personnel can obtain consistent information basis when viewing, reducing the judgment differences caused by individual understanding differences.

[0025] In the face of complex multi-party accidents, the complete sequence diagram can systematically present the dynamic changes of each vehicle, and the responsibility judgment model can verify the compliance of each party's behavior one by one, making the basis for responsibility division more clear. For the case where the video data is incomplete or the key information is unclear, the marking and analysis process of the identified part can guide technical personnel to conduct targeted supplementary collection or enhanced processing, improving the utilization efficiency of video data. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 The working principle diagram of the vehicle video data analysis method based on traffic responsibility definition described in the present application;

[0027] Figure 2 The flowchart for merging event sequence diagrams into a comprehensive sequence diagram;

[0028] Figure 3 The flowchart for extracting a basic sequence diagram;

[0029] Figure 4A flowchart for locating the start node and the end node;

[0030] Figure 5 A flowchart for a vehicle video data analysis system. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0032] Please refer to Figures 1-5 The present application provides a vehicle video data analysis method based on traffic responsibility definition, and the specific implementation steps are as follows:

[0033] A video library is built, which includes various traffic scene videos, each traffic scene video having an event sequence diagram, and the event sequence diagram including multiple key frame nodes. The building steps of the video library are as follows: real-time collection of traffic scene videos through road monitoring cameras and vehicle-mounted event data recorders, and import of historical accident video data; storage of the collected videos in the video library after denoising and resolution standardization preprocessing. The key frame nodes in the event sequence diagram are extracted in the following ways: detection of motion changes in the video based on the optical flow method, and when the motion vectors of adjacent frames exceed a preset threshold, the frame is taken as a key frame node; or feature extraction of video frames using a convolutional neural network model, and extraction of key frame nodes based on feature similarity clustering.

[0034] The event sequence diagrams of the same scene traffic scene videos are merged into a comprehensive sequence diagram.

[0035] The responsibility requirements and analysis requirements of the accident are obtained, the corresponding comprehensive sequence diagram is selected based on the responsibility requirements, and the corresponding basic sequence diagram is extracted in the comprehensive sequence diagram based on the analysis requirements.

[0036] The to-be-identified part is labeled in the basic sequence diagram, and the to-be-identified part is the part that needs to be identified in the basic sequence diagram.

[0037] A responsibility determination model is constructed, a complete sequence diagram is obtained after the analysis of the to-be-identified part, each key frame node in the complete sequence diagram is checked based on the responsibility determination model, a key frame node that fails to pass the check is defined as a disputed node, and a marker pointing to the disputed node is generated. The responsibility determination model is constructed based on a neural network, the attribute information of the analyzed key frame node is input into the responsibility determination model, the responsibility determination model outputs the dispute probability of the key frame node, and when the dispute probability is greater than a critical threshold, the key frame node is determined as a disputed node.

[0038] Embodiment 1

[0039] In the process of merging event sequence graphs into a comprehensive sequence graph, the basic composition of the event sequence graph, i.e. the key frame nodes marked with time flow lines between them, is made clear. Based on these time flow lines, each key frame node can be numbered, and through the numbering, the event sequence graph can be divided into multiple stages. The numbering here is in the order of time flow, and each stage corresponds to a different period of event development. Such division is helpful for subsequent integration and analysis of event sequences.

[0040] The event sequence graphs corresponding to the same scene traffic scene video are taken as aggregation objects. Among these aggregation objects, the one containing the most stages needs to be found and determined as the main object. The key frame nodes contained in the main object become the main nodes, which play a leading role in the subsequent merging process.

[0041] The reference nodes are located in the remaining aggregation objects. The specific positioning steps are as follows: first, find the stage with the smallest value in the aggregation object, i.e. the stage where the event occurred earliest, and take the key frame node in this stage as the first node. Then, check whether there is a main node in the main object that has an association condition with the first node. The association condition is that attribute information is set for each key frame node, including its own event label, the event label of the time source node, and the event label of the time output node. When the event label of the main node is the same as that of the first node, and the event label of at least one of the time source node and the time output node is also the same, it is considered that the main node meets the association condition with the first node, and the first node is determined as the reference node.

[0042] If no main node that meets the association condition with the first node is found in the main object, then key frame nodes are extracted as second nodes in other stages of the aggregation object, and then it is judged whether the second nodes can become reference nodes. The judgment method is the same as that for the first node, i.e. checking whether there is a main node in the main object that meets the association condition with the second node. If there is, the second node becomes the reference node; if not, the key frame nodes in other stages are continuously extracted as the next nodes for judgment, and this process is repeated until the reference node is located.

[0043] After finding the reference node, the key frame nodes of each stage of the aggregation object are merged into the main object starting from the reference node. During the merging process, it is necessary to determine whether the main node and the key frame node meet the association condition. If they meet the condition, the key frame node is merged into the main node, making the main node contain more information. If they do not meet the condition, the key frame node is linked as a branch node into the main object. In this way, without changing the structure of the main object, the additional key frame node information is preserved, thereby forming a more comprehensive integrated sequence diagram.

[0044] During the entire merging process, each step is closely connected. Through the numbering of key frame nodes, the division of stages, the positioning of reference nodes, and the judgment of association conditions, it is ensured that the event sequence diagrams of the same scene traffic scene video can be accurately merged into integrated sequence diagrams. This merging method can integrate the information of multiple event sequence diagrams to form a more complete and comprehensive integrated sequence diagram, providing a solid foundation for subsequent selection of integrated sequence diagrams based on responsibility requirements and extraction of basic sequence diagrams in integrated sequence diagrams. In this way, the integrated sequence diagram can more comprehensively reflect the development process and key nodes of different traffic events under the same scene, providing more rich and accurate information support for subsequent traffic responsibility definition and video data analysis. In this process, each step is strictly followed by established rules and procedures, ensuring the accuracy and reliability of the merging results, so that the integrated sequence diagram can be effectively applied to subsequent analysis work. From the preliminary processing of the event sequence diagram to the determination of the aggregation object, to the accurate positioning of the reference node, and finally to the merging of the key frame node, each step is carefully designed and strictly executed to ensure the smooth progress of the entire merging process and the effectiveness of the merging results. This method of merging event sequence diagrams of the same scene into integrated sequence diagrams can make full use of existing video data information and provide more comprehensive analysis basis for traffic responsibility definition, which helps to more accurately determine the responsibility attribution in traffic events.

[0045] Example 2:

[0046] When extracting the corresponding basic sequence diagram in the integrated sequence diagram, the demand sequence diagram needs to be obtained based on the analysis requirements of the accident. The demand sequence diagram reflects the event flow framework set according to the analysis purpose, which contains multiple flow nodes representing the key links that need to be focused on in the analysis process.

[0047] The starting node and the ending node in the requirement sequence diagram are located. Specifically, the two ends of the requirement sequence diagram are taken as starting points, and whether each flow node in the requirement sequence diagram is a target node is determined in sequence. The criterion for judging the target node is that if there is a key frame node in the comprehensive sequence diagram that meets the association condition with the flow node, the flow node is determined as the target node. The association condition here refers to the attribute information set for each key frame node, including its own event label, the event label of the time source node, and the event label of the time output node. When the key frame node in the comprehensive sequence diagram and the flow node in the requirement sequence diagram meet a certain correspondence in these attribute information, it is considered to meet the association condition. The target nodes are determined at the two ends of the requirement sequence diagram respectively, and the target nodes determined first at the two ends are defined as the starting node and the ending node respectively.

[0048] The key frame nodes associated with the starting node and the ending node of the requirement sequence diagram are located in the comprehensive sequence diagram, and the key frame nodes are taken as the top end node and the bottom end node respectively. The top end node and the bottom end node are the key nodes in the comprehensive sequence diagram corresponding to the starting node and the ending node of the requirement sequence diagram, and they constitute the starting point and the terminal point of extracting the basic sequence diagram in the comprehensive sequence diagram.

[0049] Taking the top end node and the bottom end node as the starting point and the terminal point, a plurality of candidate sequence diagrams are generated in the comprehensive sequence diagram. These candidate sequence diagrams are different paths from the top end node to the bottom end node in the comprehensive sequence diagram, and each path represents a possible event development sequence.

[0050] The matching degree of the requirement sequence diagram and each candidate sequence diagram is calculated. The process of calculating the matching degree is as follows: first, the corresponding feature value is generated based on the event label of the key frame node, and the event label of each key frame node corresponds to a unique feature value, which is used to represent the event attribute of the node. The key frame node adjacent to the starting node or the top end node is defined as the first adjacent node, and the feature values of the first adjacent node and the starting node itself are subjected to XOR operation, and the feature values of the first adjacent node and the top end node itself are also subjected to XOR operation, and the label values of the starting node and the top end node are obtained through the two XOR operations. The XOR operation here is a logical operation for comparing the difference between two feature values, so as to obtain a label value that can reflect the degree of association between nodes.

[0051] The second adjacent node of the first adjacent node is obtained, the second adjacent node does not contain the starting node or the top node, the characteristic values of the first adjacent node and the second adjacent node are subjected to XOR operation, and the label value of the first adjacent node is obtained. In this way, each adjacent node is processed in turn until all key frame nodes in the sequence diagram are traversed. During the traversal process, the label value corresponding to each node is obtained by continuously performing XOR operation on the characteristic values of adjacent nodes, and the label values reflect the position and association relationship of the nodes in the sequence diagram.

[0052] Based on the obtained label values, a first label set and a second label set corresponding to the demand sequence diagram and the alternative sequence diagram are generated. The first label set and the second label set respectively contain the label values of all nodes in the demand sequence diagram and the alternative sequence diagram. Finally, the matching degree is calculated based on the number of the same label values contained in the first label set and the second label set. The more the number of the same label values, the more similar the structure and node association relationship of the two sequence diagrams, and the higher the matching degree.

[0053] After calculating the matching degrees of all alternative sequence diagrams and the demand sequence diagram, the alternative sequence diagram with the highest matching degree is taken as the basic sequence diagram. The basic sequence diagram is the event sequence that best meets the analysis demand extracted from the comprehensive sequence diagram, which can provide an accurate analysis object for subsequent annotation of the to-be-identified part in the basic sequence diagram and responsibility checking using the responsibility determination model. The entire extraction process ensures that the extracted basic sequence diagram can accurately reflect the accident analysis demand and provide effective support for video data analysis of traffic responsibility definition through node positioning of the demand sequence diagram and the comprehensive sequence diagram, alternative sequence generation, and matching degree calculation.

[0054] Embodiment 3:

[0055] When building the video library, first, traffic scene videos are collected in real time through various devices, and historical accident video data is imported. Specifically, road monitoring cameras are used, which are usually installed at key positions on the road, such as intersections, curves, bridges, etc., and can continuously monitor and shoot the traffic situation on the road from a fixed perspective, obtaining videos containing information such as the overall condition of the road, vehicle trajectory, traffic signal changes, etc. Vehicle-mounted event data recorders are installed on vehicles and move with the vehicles to record, capturing real-time situations in front of the vehicle and inside the vehicle, including the interaction details between the vehicle and surrounding vehicles and pedestrians. The combination of these two devices collects videos from different angles and positions, making the obtained traffic scene videos more comprehensive and diverse.

[0056] After collecting the videos, they need to be preprocessed, which includes two main steps: denoising and resolution standardization. Denoising is to eliminate the noise interference in the video. During the collection process, due to environmental light changes, device performance, and other factors, various noises such as Gaussian noise and salt and pepper noise may be generated, which will affect the clarity of the video and subsequent analysis. By using denoising algorithms to process the video, these noises can be removed, making the video clearer and the key information more prominent, which is convenient for subsequent identification and analysis of events and objects in the video.

[0057] Resolution standardization preprocessing is because videos collected by different devices may have different resolutions, such as standard definition resolution, high definition resolution, and other different resolution specifications. If these videos with different resolutions are directly stored in the video library, it will bring many inconveniences in subsequent analysis due to inconsistent resolutions. For example, when performing video frame processing, feature extraction, and other operations, different resolutions need to be processed differently, which will increase the complexity and workload of analysis. Therefore, the collected videos need to be converted to a standard resolution. When performing resolution conversion, appropriate algorithms and techniques need to be used to maintain the original information and quality of the video as much as possible, avoiding the loss or distortion of important information due to resolution conversion.

[0058] After preprocessing, the processed videos are stored in the video library. The video library is a system or storage medium used to store and manage these traffic scene videos, which needs to have sufficient storage capacity to accommodate a large amount of video data. At the same time, the video library should have good organization and management functions, and be able to store videos in a classified manner, such as by traffic scene type (e.g. urban road, highway, rural road, etc.), accident type (e.g. rear-end collision, scratch, collision with pedestrians, etc.), time and date, etc., so as to facilitate subsequent quick and accurate retrieval and retrieval of required videos according to different responsibility requirements and analysis requirements.

[0059] Each link is crucial in the whole process of building the video library. From the video collection, ensure the normal operation and reasonable arrangement of the collection equipment to obtain high-quality and comprehensive video data. The noise removal and resolution standardization processing in the preprocessing link provide a good foundation for subsequent video analysis, making the video data more standardized and easy to process. The storage and management of the video library ensure the security and accessibility of the video data, making it easy to quickly find the required video for analysis when needed. Through such steps and methods, the video library can contain various traffic scene videos, each of which can further generate corresponding event sequence graphs, providing rich data support for subsequent vehicle video data analysis based on traffic responsibility definition. The whole process strictly follows the specified process to ensure that the data in the video library is accurate, standardized, and usable, thereby providing a reliable video data basis for traffic responsibility definition. From video collection, preprocessing to storage, each step is carefully designed and executed to ensure the quality and practicality of the video library, so that accurate video information can be obtained from the video library when performing traffic responsibility definition, assisting in responsibility determination and analysis.

[0060] Example 4:

[0061] When extracting key frame nodes in the event sequence graph, there are two specific implementations, which are described in detail as follows.

[0062] The first way is to detect the motion change in the video based on the optical flow method to extract the key frame node. The core idea of the optical flow method is to calculate the motion vector of each pixel point between adjacent frames of video to represent the motion of objects in the video. In specific operation, first, the video needs to be processed frame by frame. For each frame of image, calculate the motion vector of each pixel point in the adjacent next frame image. The motion vector here is a two-dimensional vector that represents the displacement direction and size of the pixel point from the current frame to the next frame.

[0063] When the motion vector of adjacent frames exceeds the preset threshold, the frame is taken as a key frame node. The preset threshold is a pre-set value used to judge whether the degree of motion change is large enough. Assuming that the current processing is the i-th frame image, after calculating the motion vector of all pixel points between the i-th frame and the i+1-th frame, statistical analysis needs to be performed on these motion vectors, such as calculating the average value or maximum value of the motion vector. If the average value or maximum value exceeds the preset threshold T, it means that there is a large motion change between the i-th frame and the i+1-th frame, and a key event may have occurred, such as sudden acceleration, deceleration, turning of the vehicle, or sudden appearance of pedestrians, etc. At this time, the i-th frame is extracted as a key frame node.

[0064] The second way is to use a convolutional neural network model to extract features from video frames, and then extract key frame nodes based on feature similarity clustering. Convolutional Neural Network (CNN) is a deep learning model that has strong feature extraction capability and can automatically extract valuable features from video frames.

[0065] First, input each frame of the video into the convolutional neural network model, and the model will perform forward propagation calculation on each frame to output the feature vector of the frame. The feature vector is a high-dimensional numerical vector that contains various feature information in the video frame, such as image color, texture, shape, etc. Assume that the input video frame is F j , after processing by the convolutional neural network model, the obtained feature vector is V j , where j represents the sequence number of the video frame.

[0066] Next, the feature vectors need to be clustered. The purpose of clustering is to classify video frames with high feature similarity into a class, because these video frames may contain similar scenes or events, and only a representative frame needs to be selected in each class. Common clustering algorithms include K-means clustering algorithm, etc. In the clustering process, the similarity between each two feature vectors needs to be calculated. Here, the cosine similarity is used to measure the similarity of feature vectors, and the calculation formula of cosine similarity is:

[0067]

[0068] where V a and V b represent the feature vectors of two different video frames, V a ·V b represents the dot product of the two feature vectors, |V a | and |V b | represent the length of the two feature vectors. The cosine similarity value ranges between [-1, 1], and the larger the value, the higher the similarity between the two feature vectors.

[0069] All feature vectors are divided into several clustering clusters by clustering algorithm, and the video frames in each clustering cluster have high feature similarity. Then, a representative video frame is selected as a key frame node in each clustering cluster. The method of selecting a representative frame can be to select the frame with the highest average similarity to other frames in the clustering cluster, or to select the center frame of the clustering cluster, etc.

[0070] In practical applications, the two key frame node extraction methods can be selected for use according to specific circumstances, or can be used in combination to improve the accuracy and comprehensiveness of key frame node extraction. For example, in some scenes sensitive to motion changes, the optical flow method can be mainly used to extract key frame nodes; while in some scenes that need to consider the content features of images, more reliance can be placed on the method of combining clustering with a convolutional neural network model.

[0071] The key frame nodes extracted by the two methods can effectively represent the key events and scene changes in the video, providing a basis for subsequent construction of event sequence graphs. Each key frame node corresponds to a key time point in the video. Arranging these key frame nodes in chronological order and marking the time flow lines between them forms an event sequence graph. The event sequence graph can intuitively show the development process and key nodes of events in the video, providing an important basis for subsequent work such as merging event sequence graphs of the same scene into comprehensive sequence graphs, and extracting and determining responsibility based on comprehensive sequence graphs.

[0072] During the entire key frame node extraction process, each step needs to be strictly executed to ensure that the extracted key frame nodes accurately reflect the key information in the video. For the optical flow method, the preset threshold needs to be adjusted reasonably according to the actual video scene and requirements; for the convolutional neural network model, appropriate network structure and training parameters need to be selected to ensure the accuracy of feature extraction. At the same time, in the clustering process, the determination of the number of clusters also affects the extraction effect of key frame nodes, which needs to be reasonably selected according to the length and content complexity of the video. By paying attention to and handling these details, the quality of key frame node extraction can be improved, thereby laying a solid foundation for the entire vehicle video data analysis method based on traffic responsibility determination.

[0073] Embodiment 5:

[0074] In constructing the responsibility determination model and applying it to the analysis of vehicle video data for traffic responsibility determination, it is necessary to first clarify that the responsibility determination model is based on a neural network. The structure of the neural network can include an input layer, a hidden layer, and an output layer, where the input layer is used to receive attribute information of key frame nodes, the hidden layer performs feature extraction and complex relationship learning, and the output layer outputs the dispute probability of key frame nodes.

[0075] When the analysis of the to-be-identified part in the basic sequence diagram is completed, a complete sequence diagram is obtained. The complete sequence diagram contains all key frame nodes of the traffic event from the beginning to the end, and each key frame node has detailed attribute information, including its own event label, such as "vehicle lane change", "pedestrian crossing", "red light on", etc., used to describe the event content represented by the key frame node; and the event label of the time source node, i.e. the event label of the previous key frame node of the key frame node, and the event label of the time output node, i.e. the event label of the next key frame node of the key frame node. These attribute information constitutes the input data of the responsibility determination model.

[0076] The attribute information is input into the responsibility determination model, and the model processes the input data. Taking a specific traffic event as an example, suppose there is a key frame node A in the complete sequence diagram, whose own event label is "vehicle speeding at intersection", the event label of the time source node is "green light on vehicle starts", and the event label of the time output node is "vehicle collision with oncoming vehicle". When these attribute information is input into the model, the input layer receives these data and passes them to the hidden layer. The neurons in the hidden layer calculate and process the data through weights and activation functions, learn the association between different attribute information, such as the probability relationship between speeding and collision, and the influence of previous and subsequent event labels on the controversy of the current event.

[0077] After multiple layers of processing in the hidden layer, the output layer generates the controversy probability of the key frame node. The controversy probability is a value between 0 and 1, which represents the possibility of the key frame node being controversial in the responsibility determination process. At this time, a critical threshold needs to be set, which can be set according to the actual traffic responsibility definition requirements and historical data experience, for example, set to 0.6. When the output controversy probability is greater than the critical threshold, the key frame node is determined to be a controversial node, and a marker pointing to the controversial node is generated to remind.

[0078] The setting of the controversial probability threshold needs to be determined comprehensively in combination with various actual situations. The probability distribution characteristics of the controversial nodes confirmed by artificial review in historical accident video data are referred to as the basis for reference. At the same time, considering the strictness difference of responsibility determination in different traffic scenes, for example, in the highway scene, because the accident consequences are usually more serious, the determination of controversial points needs to be more rigorous, and the setting of the critical threshold needs to adapt to the accuracy requirement of responsibility definition in this scene; in the urban ordinary road scene, the determination efficiency and accuracy can be balanced according to the controversial characteristics of common accidents. In addition, combined with the conventional standards for identifying controversial points in law enforcement practice, the controversial and non-controversial limits generally recognized by law enforcement personnel in past responsibility determination are collected. At the same time, in the training process of the responsibility determination model, by verifying the identification accuracy and misjudgment rate of the model for known controversial nodes under different thresholds, the threshold value that can make the model accurately identify the actual controversial nodes while reducing the misjudgment of non-controversial nodes as controversial nodes is selected, to ensure that the setting of the critical threshold can effectively mark the controversial nodes that need to be checked, and will not cause the omission of key controversial points due to the threshold being too high, or produce too many invalid marks due to the threshold being too low, so as to adapt to the actual needs of responsibility definition in various traffic scenes.

[0079] Continuing with the above-mentioned key frame node A as an example, if the controversial probability output by the responsibility determination model is 0.7, which is greater than the set critical threshold 0.6, the node is defined as a controversial node. The system will automatically generate a mark reminder, which can be color labeling of the node in the complete sequence diagram, such as marking red, or generating a reminder list containing detailed information of the node, clearly indicating the event label, attribute information and controversial probability of the node, so that relevant personnel can quickly locate the controversial node and further check and handle it.

[0080] In practical application, the construction of the responsibility determination model needs to be trained based on a large amount of historical traffic event data. In the training process, the attribute information of the key frame nodes with known responsibility determination results is input into the model, and by comparing with the actual responsibility determination results, the weights and parameters of the model are adjusted, so that the model can gradually learn the correct responsibility determination logic and rules, and improve the accuracy of the controversial probability output by the model.

[0081] For example, in the training data, there are multiple key frame nodes similar to node A, whose attribute information is "vehicle overspeed driving" and subsequent collision. These nodes may be identified as the responsible party in actual responsibility determination. The model will gradually increase the output value of the controversial probability of such nodes by learning these data. When encountering a new traffic event, the model can make a more accurate prediction of the controversial probability of the key frame node based on the learned knowledge.

[0082] After completing the checking of each key frame node in the complete sequence graph, the responsibility determination model generates a marked reminder containing multiple controversial nodes. The staff can focus on the event link represented by the controversial nodes according to the marked reminders, and combine specific pictures in the video and other related information such as traffic rules, vehicle driving trajectory, etc. to conduct in-depth analysis and judgment on the responsibility attribution of the controversial nodes.

[0083] For example, in the above example, the event corresponding to the controversial node A is "vehicle driving at speed in the intersection" and causing a collision. After seeing the marked reminder, the staff will check the video frame corresponding to the node to confirm whether the vehicle is indeed driving at speed, the traffic signal condition of the intersection, and the driving state of the transverse vehicle, etc. to more accurately determine the responsibility division in the event.

[0084] Through the application of the responsibility determination model, controversial nodes in the complete sequence graph can be automatically identified, reducing the workload of manual checking of all key frame nodes and improving the efficiency of traffic responsibility determination. At the same time, based on the strong learning ability of neural network, the model can process complex attribute information and association relationship, providing a more objective and comprehensive reference basis for responsibility determination, which helps to improve the accuracy and fairness of responsibility determination.

[0085] It should be noted that in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment.

[0086] Although the embodiments of the present application have been shown and described, it can be understood by those skilled in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for analyzing vehicle video data based on traffic liability definition, characterized in that, The method includes: A video library is built, containing videos of various traffic scenarios. Each traffic scenario video has an event sequence diagram, which includes multiple keyframe nodes. Event sequence diagrams of traffic scenario videos of the same scenario are merged into a comprehensive sequence diagram. The responsibility requirements and analysis requirements for the accident are obtained. Based on the responsibility requirements, the corresponding comprehensive sequence diagram is selected, and based on the analysis requirements, the corresponding basic sequence diagram is extracted from the comprehensive sequence diagram. The parts to be identified are marked on the basic sequence diagram, which are the parts that need to be identified in the basic sequence diagram. A responsibility determination model is constructed. After the parts to be identified are parsed, a complete sequence diagram is obtained. Based on the responsibility determination model, each keyframe node in the complete sequence diagram is checked. Keyframe nodes that fail the check are defined as dispute nodes, and a marker reminder pointing to the dispute nodes is generated. Merging event sequence diagrams into a comprehensive sequence diagram involves the following steps: Keyframe nodes in the event sequence diagrams are marked with time flow lines; each keyframe node is numbered based on these time flow lines; the event sequence diagrams are divided into multiple stages based on these numbers; event sequence diagrams corresponding to traffic scene videos of the same scenario are used as aggregation objects; the aggregation object containing the most stages is used as the main object; the keyframe nodes included in the main object are used as main nodes; reference nodes are located in the remaining aggregation objects; starting from the reference nodes, the keyframe nodes of each stage of the aggregation objects are merged into the main object; during merging, if the main node and keyframe nodes satisfy the association condition, the keyframe nodes are merged into the main node; otherwise, the keyframe nodes are linked into the main object as branch nodes. Extracting the corresponding base sequence diagram from the composite sequence diagram includes the following steps: obtaining a requirement sequence diagram based on the analysis requirements of the incident; locating the start and end nodes in the requirement sequence diagram; locating keyframe nodes that can be associated with the start and end nodes in the composite sequence diagram, and using them as the top and bottom nodes respectively; generating multiple candidate sequence diagrams in the composite sequence diagram with the top and bottom nodes as the start and end points; calculating the matching degree between the requirement sequence diagram and each candidate sequence diagram; and using the candidate sequence diagram with the highest matching degree as the base sequence diagram.

2. The vehicle video data analysis method based on traffic liability definition according to claim 1, characterized in that, Locating the reference node involves the following steps: Locate the stage with the smallest value in the aggregated object and take its keyframe node as the first node. If a main node with a related condition to the first node is found in the main object, then the first node is determined as the reference node. Otherwise, continue to extract keyframe nodes as the second nodes in other stages of the aggregated object and determine whether the second node is a reference node. Repeat this step until the reference node is located.

3. The vehicle video data analysis method based on traffic liability definition according to claim 2, characterized in that, Locating the start and end nodes includes the following steps: The requirement sequence diagram includes multiple process nodes. Starting from both ends of the requirement sequence diagram, it is determined whether each process node is a target node. If a key frame node in the integrated sequence diagram satisfies the association condition with a process node, the process node is determined as the target node. The target nodes first determined at both ends of the requirement sequence diagram are defined as the start node and the end node, respectively.

4. The vehicle video data analysis method based on traffic liability definition according to claim 1, characterized in that, Determining whether the association conditions are met includes the following steps: setting attribute information for each keyframe node, including its own event label, the event label of the time source node, and the event label of the time output node. When the event labels of the main node and the reference node are the same, and at least one event label of the time source node and the time output node are the same, then the main node and the keyframe node are defined as meeting the association conditions.

5. The vehicle video data analysis method based on traffic liability determination according to claim 3, characterized in that, The matching degree calculation includes the following steps: First, generate corresponding feature values ​​based on the event labels of the keyframe nodes themselves. Second, identify the keyframe nodes adjacent to the start or top node as the first neighboring nodes. Third, perform an XOR operation between the feature values ​​of the first neighboring nodes and the start node, and then perform an XOR operation between the feature values ​​of the first neighboring nodes and the top node to obtain the label values ​​of the start and top nodes. Fourth, obtain the second neighboring nodes of the first neighboring nodes, excluding the start or top node. Perform an XOR operation between the feature values ​​of the first and second neighboring nodes to obtain the label value of the first neighboring node. Repeat this step until all keyframe nodes in the sequence graph have been traversed. Fifth, generate a first label set and a second label set corresponding to the required sequence graph and the alternative sequence graph based on the label values. Sixth, calculate the matching degree based on the number of identical label values ​​in the first and second label sets.

6. The vehicle video data analysis method based on traffic liability determination according to claim 5, characterized in that, The responsibility determination model is built on a neural network. The attribute information of the parsed keyframe nodes is input into the responsibility determination model, and the responsibility determination model outputs the dispute probability of the keyframe nodes. When the dispute probability is greater than the critical threshold, the keyframe node is determined to be a dispute node.

7. The vehicle video data analysis method based on traffic liability determination according to claim 1, characterized in that, The construction of the video library includes the following steps: real-time collection of traffic scene videos through road monitoring cameras and vehicle dashcams, and import of historical accident video data; the collected videos are preprocessed by denoising and resolution standardization before being stored in the video library.

8. The vehicle video data analysis method based on traffic liability determination according to claim 1, characterized in that, Keyframe nodes in the event sequence graph are extracted in the following ways: based on optical flow detection of motion changes in the video, when the motion vector of an adjacent frame exceeds a preset threshold, the frame is taken as a keyframe node; or a convolutional neural network model is used to extract features from video frames, and keyframe nodes are extracted based on feature similarity clustering.

Images